AI Coding Kya Hota Hai

AI Coding Kya Hota Hai — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  List of large language models

  A large language model (LLM) is a type of machine learning model designed for natural language processing tasks such as language generation. LLMs are language models with many parameters, and are trained with self-supervised learning on a vast amount of text. == List == For the training cost column, 1 petaFLOP-day equals 1 petaFLOP/sec × 1 day, or 8.64×1019 FLOP (floating point operations). Only the cost of the largest model is shown. The number of parameters is measured in billions, and the training cost is measured in petaFLOP-days. === 2018 === === 2019 === === 2020 === === 2021 === === 2022 === === 2023 === === 2024 === === 2025 === === 2026 ===

  Read more →
• 

  FrameNet

  FrameNet is a group of online lexical databases based upon the theory of meaning known as Frame semantics, developed by linguist Charles J. Fillmore. The project's fundamental notion is simple: most words' meanings may be best understood in terms of a semantic frame, which is a description of a certain kind of event, connection, or item and its actors. As an illustration, the act of cooking usually requires the following: a cook, the food being cooked, a container to hold the food while it is being cooked, and a heating instrument. Within FrameNet, this act is represented by a frame named Apply_heat, and its components (Cook, Food, Container, and Heating_instrument), are referred to as frame elements (FEs). The Apply_heat frame also lists a number of words that represent it, known as lexical units (LUs), like fry, bake, boil, and broil. Other frames are simpler. For example, Placing only has an agent or cause, a theme—something that is placed—and the location where it is placed. Some frames are more complex, like Revenge, which contains more FEs (offender, injury, injured party, avenger, and punishment). As in the examples of Apply_heat and Revenge below, FrameNet's role is to define the frames and annotate sentences to demonstrate how the FEs fit syntactically around the word that elicits the frame. == Concepts == === Frames === A frame is a schematic representation of a situation involving various participants, props, and other conceptual roles. Examples of frame names are Being_born and Locative_relation. A frame in FrameNet contains a textual description of what it represents (a frame definition), associated frame elements, lexical units, example sentences, and frame-to-frame relations. === Frame elements === Frame elements (FE) provide additional information to the semantic structure of a sentence. Each frame has a number of core and non-core FEs which can be thought of as semantic roles. Core FEs are essential to the meaning of the frame while non-core FEs are generally descriptive (such as time, place, manner, etc.) For example: The only core FE of the Being_born frame is called Child; non-core FEs Time, Place, Means, etc. Core FEs of the Commerce_goods-transfer frame include the Seller, Buyer, and Goods, while non-core FEs include a Place, Purpose, etc. FrameNet includes shallow data on syntactic roles that frame elements play in the example sentences. For example, for a sentence like "She was born about AD 460", FrameNet would mark She as a noun phrase referring to the Child frame element, and "about AD 460" as a noun phrase corresponding to the Time frame element. Details of how frame elements can be realized in a sentence are important because this reveals important information about the subcategorization frames as well as possible diathesis alternations (e.g. "John broke the window" vs. "The window broke") of a verb. === Lexical units === Lexical units (LUs) are lemmas, with their part of speech, that evoke a specific frame. In other words, when an LU is identified in a sentence, that specific LU can be associated with its specific frame(s). For each frame, there may be many LUs associated to that frame, and also there may be many frames that share a specific LU; this is typically the case with LUs that have multiple word senses. Alongside the frame, each lexical unit is associated with specific frame elements by means of the annotated example sentences. For example, lexical units that evoke the Complaining frame (or more specific perspectivized versions of it, to be precise), include the verbs complain, grouse, lament, and others. === Example sentences === Frames are associated with example sentences and frame elements are marked within the sentences. Thus, the sentence She was born about AD 460 is associated with the frame Being_born, while She is marked as the frame element Child and "about AD 460" is marked as Time. From the start, the FrameNet project has been committed to looking at evidence from actual language use as found in text collections like the British National Corpus. Based on such example sentences, automatic semantic role labeling tools are able to determine frames and mark frame elements in new sentences. === Valences === FrameNet also exposes statistics on the valence of each frame; that is, the number and position of the frame elements within example sentences. The sentence She was born about AD 460 falls in the valence pattern NP Ext, INI --, NP Dep which occurs twice in the FrameNet's annotation report for the born.v lexical unit, namely: She was born about AD 460, daughter and granddaughter of Roman and Byzantine emperors, whose family had been prominent in Roman politics for over 700 years. He was soon posted to north Africa, and never met their only child, a daughter born 8 June 1941. === Frame relations === FrameNet additionally captures relationships between different frames using relations. These include the following: Inheritance: When one frame is a more specific version of another, more abstract, parent frame. Anything that is true about the parent frame must also be true about the child frame, and a mapping is specified between the frame elements of the parent and the frame elements of the child. Perspectivization: A neutral frame is connected to a frame with a specific perspective of the same scenario. For example, Commerce_transfer-goods is considered from the perspective of the buyer in Commerce_buy and from that of the seller in Commerce_sell. Subframe: Some frames refer to complex scenarios that consist of several individual states or events that can be described by separate frames. For example, Criminal_process is composed of Arrest, Trial, and so on. Precedence: This relation captures the temporal order that holds between subframes of a complex frame. For example, within the Cycle_of_life_and_death frame, the subframe Death is preceded by the subframe Being_born. Causative and Inchoative: These two relations mark, for causative- and inchoative-aspect frames, the separate stative frame they refer to. For example, the stative Position_on_a_scale (e.g. "She had a high salary") is described by the causative Cause_change_of_scalar_position (e.g. "She raised his salary") and by the inchoative Change_position_on_a_scale frame (e.g. "Her salary increased"). Using: This relation marks a frame that in some way involves another frame. For example, Judgment_communication uses both Judgment and Statement, but does not inherit from either of them because there is no clear correspondence of frame elements. See also: Connects frames that bear some resemblance but need to be distinguished carefully. == Applications == FrameNet has proven to be useful in a number of computational applications, because computers need additional knowledge in order to recognize that "John sold a car to Mary" and "Mary bought a car from John" describe essentially the same situation, despite using two quite different verbs, different prepositions and a different word order. FrameNet has been used in applications like question answering, paraphrasing, recognizing textual entailment, and information extraction, either directly or by means of Semantic Role Labeling tools. The first automatic system for Semantic Role Labeling (SRL, sometimes also referred to as "shallow semantic parsing") was developed by Daniel Gildea and Daniel Jurafsky based on FrameNet in 2002. Semantic Role Labeling has since become one of the standard tasks in natural language processing, with the latest version (1.7) of FrameNet now fully supported in the Natural Language Toolkit. Since frames are essentially semantic descriptions, they are similar across languages, and several projects have arisen over the years that have relied on the original FrameNet as the basis for additional non-English FrameNets, for Spanish, Japanese, German, and Polish, among others.

  Read more →
• 

  AI Pair Programmers: Free vs Paid (2026)

  Trying to pick the best AI pair programmer? An AI pair programmer is software that uses machine learning to help you get more done — it scales effortlessly from a single task to thousands. The best picks balance beginner-friendly simplicity with the depth power users need, and they ship updates often. Whether you are a beginner or a pro, the right AI pair programmer slots into your workflow and pays for itself fast. This guide breaks down the top picks, their pros and cons, and who each one is best for.

  Read more →
• 

  Arthur Zimek

  Arthur Zimek is a professor in data mining, data science and machine learning at the University of Southern Denmark in Odense, Denmark. He graduated from LMU Munich in Germany, where he worked with Prof. Hans-Peter Kriegel. His dissertation on "Correlation Clustering" was awarded the "SIGKDD Doctoral Dissertation Award 2009 Runner-up" by the Association for Computing Machinery. He is well known for his work on outlier detection, density-based clustering, correlation clustering, and the curse of dimensionality. He is one of the founders and core developers of the open-source ELKI data mining framework.

  Read more →
• 

  List of security hacking incidents

  This list of security hacking incidents covers important or noteworthy events in the history of security hacking and cracking. == 1900 == === 1903 === Magician and inventor Nevil Maskelyne disrupts John Ambrose Fleming's public demonstration of Guglielmo Marconi's purportedly secure wireless telegraphy technology, sending insulting Morse code messages through the auditorium's projector. == 1930s == === 1932 === Polish cryptologists Marian Rejewski, Henryk Zygalski and Jerzy Różycki broke the Enigma machine code. === 1939 === Alan Turing, Gordon Welchman and Harold Keen worked together to develop the codebreaking device Bombe (based off of Rejewski's work on Bomba). The Enigma machine's use of a reliably small key space makes it vulnerable to brute force attacks. == 1940s == === 1943 === René Carmille, comptroller general of the Vichy French Army, hacked the punch card system used by the Nazis to locate Jews. === 1949 === The theory that underlies computer viruses was first made public in 1949, when computer pioneer John von Neumann presented a paper titled "Theory and Organization of Complicated Automata". In the paper, von Neumann speculated that computer programs could reproduce themselves. == 1950s == === 1955 === At MIT, "hack" first came to mean playing with machines. An April 1955 meeting of the Tech Model Railroad Club has one say that "Mr. Eccles requests that anyone working or hacking on the electrical system turn the power off to avoid fuse blowing." === 1957 === Joe "Joybubbles" Engressia, a blind seven-year-old boy with perfect pitch, discovered that whistling the fourth E above middle C (a frequency of 2600 Hz) would interfere with AT&T's automated telephone systems, thereby inadvertently opening the door for phreaking. == 1960s == Various phreaking boxes are used to interact with automated telephone systems. === 1963 === The first ever reference to malicious hacking is 'phreaking' in MIT's student newspaper, The Tech, containing hackers tying up the lines with Harvard, configuring the PDP-1 to make free calls, war dialing and accumulating large phone bills. === 1965 === William D. Mathews from MIT finds a vulnerability in a CTSS running on an IBM 7094. The standard text editor on the system was designed to be used by one user at a time, working in one directory, and so it created a temporary file with a constant name for all instances of the editor. The flaw was discovered when two system programmers were editing at the same time and the temporary files for the message of the day and the password file became swapped, causing the contents of the system CTSS password file to display to any user logging into the system. === 1967 === The first known incidence of network penetration hacking took place when members of a computer club at a suburban Chicago high school were provided access to IBM's APL network. In the fall of 1967, IBM (through Science Research Associates) approached Evanston Township High School with the offer of four 2741 Selectric teletypewriter-based terminals with dial-up modem connectivity to an experimental computer system which implemented an early version of the APL programming language. The APL network system was structured into workspaces which were assigned to various clients using the system. Working independently, the students quickly learned the language and the system. They were free to explore the system, often using existing code available in public workspaces as models for their own creations. Eventually, curiosity drove the students to explore the system's wider context. This first informal network penetration effort was later acknowledged as helping harden the security of one of the first publicly accessible networks:Science Research Associates undertook to write a full APL system for the IBM 1500. They modeled their system after APL/360, which had by that time been developed and seen substantial use inside of IBM, using code borrowed from MAT/1500 where possible. In their documentation, they acknowledge their gratitude to "a number of high school students for their compulsion to bomb the system". This was an early example of a kind of sportive, but very effective, debugging that was often repeated in the evolution of APL systems. == 1970s == === 1971 === John T. Draper (later nicknamed Captain Crunch), his friend Joe Engressia (also known as Joybubbles), and blue box phone phreaking hit the news with an Esquire magazine feature story. === 1979 === Kevin Mitnick breaks into his first major computer system, the Ark, which was the computer system Digital Equipment Corporation (DEC) used for developing their RSTS/E operating system software. == 1980s == === 1980 === The FBI investigates a breach of security at National CSS (NCSS). The New York Times, reporting on the incident in 1981, describes hackers as: Technical experts, skilled, often young, computer programmers who almost whimsically probe the defenses of a computer system, searching out the limits and the possibilities of the machine. Despite their seemingly subversive role, hackers are a recognized asset in the computer industry, often highly prized. The newspaper describes white hat activities as part of a "mischievous but perversely positive 'hacker' tradition". When a National CSS employee revealed the existence of his password cracker, which he had used on customer accounts, the company chastised him not for writing the software but for not disclosing it sooner. The letter of reprimand stated that "The Company realizes the benefit to NCSS and in fact encourages the efforts of employees to identify security weaknesses to the VP, the directory, and other sensitive software in files". === 1981 === Chaos Computer Club forms in Germany. Ian Murphy, aka Captain Zap, was the first cracker to be tried and convicted as a felon. Murphy broke into AT&T's computers in 1981 and changed the internal clocks that metered billing rates. People were getting late-night discount rates when they called at midday. Of course, the bargain-seekers who waited until midnight to call long distance were hit with high bills. === 1983 === The 414s break into 60 computer systems at institutions ranging from the Los Alamos National Laboratory to Manhattan's Memorial Sloan-Kettering Cancer Center. The incident appeared as the cover story of Newsweek with the title "Beware: Hackers at play". As a result, the U.S. House of Representatives held hearings on computer security and passed several laws. The group KILOBAUD is formed in February, kicking off a series of other hacker groups that formed soon after. The movie WarGames introduces the wider public to the phenomenon of hacking and creates a degree of mass paranoia about hackers and their supposed abilities to bring the world to a screeching halt by launching nuclear ICBMs. The U.S. House of Representatives begins hearings on computer security hacking. In his Turing Award lecture, Ken Thompson mentions "hacking" and describes a security exploit that he calls a "Trojan horse". === 1984 === Someone calling himself Lex Luthor founds the Legion of Doom. Named after a Saturday morning cartoon, the LOD had the reputation of attracting "the best of the best"—until one of the most talented members called Phiber Optik feuded with Legion of Doomer Erik Bloodaxe and got 'tossed out of the clubhouse'. Phiber's friends formed a rival group, the Masters of Deception. The Comprehensive Crime Control Act gives the Secret Service jurisdiction over computer fraud. The Cult of the Dead Cow forms in Lubbock, Texas, and begins publishing its underground ezine. The hacker magazine 2600 begins regular publication, right when TAP was putting out its final issue. The editor of 2600, "Emmanuel Goldstein" (whose real name is Eric Corley), takes his handle from the leader of the resistance in George Orwell's Nineteen Eighty-Four. The publication provides tips for would-be hackers and phone phreaks, as well as commentary on the hacker issues of the day. Today, copies of 2600 are sold at most large retail bookstores. The Chaos Communication Congress, the annual European hacker conference organized by the Chaos Computer Club, is held in Hamburg, Germany. William Gibson's groundbreaking science fiction novel Neuromancer, about "Case", a futuristic computer hacker, is published. Considered the first major cyberpunk novel, it brought into hacker jargon such terms as "cyberspace", "the matrix", "simstim", and "ICE". === 1985 === KILOBAUD is re-organized into P.H.I.R.M. and begins sysopping hundreds of bulletin board systems (BBSs) throughout the United States, Canada, and Europe. The online 'zine Phrack is established. The Hacker's Handbook is published in the UK. The FBI, Secret Service, Middlesex County NJ Prosecutor's Office and various local law enforcement agencies execute seven search warrants concurrently across New Jersey on July 12, 1985, seizing equipment from BBS operators and users alike for "complicity in computer theft", under a n

  Read more →
• 

  Xuedong Huang

  Xuedong David Huang (born October 20, 1962) is a Chinese-American computer scientist and technology executive who has made contributions to spoken language processing and artificial intelligence, including Azure AI Services. He is Zoom's chief technology officer after serving as Microsoft's Technical Fellow and Azure AI Chief Technology Officer for 30 years. Huang is a strong advocate of AI for Accessibility, and AI for Cultural Heritage. == Education == Huang received his PhD from the University of Edinburgh in 1989 (sponsored by the British ORS and Edinburgh University Scholarship), his MS from Tsinghua University in 1984, and BS from Hunan University in 1982. == Career == After receiving his PhD in 1989, Huang joined Carnegie Mellon University and worked with Raj Reddy and Kai-Fu Lee on speech recognition. At CMU, he directed the Sphinx-II speech system research which achieved the best performance in every category of DARPA's 1992 benchmarking. Microsoft Research recruited him to found and lead Microsoft's spoken language initiatives in 1993. His co-authored book Spoken Language Processing and his Historical speech recognition review succinctly summarize several generations of spoken language research. As Microsoft's Mr. Speech for three decades, Huang has been instrumental in creating Microsoft's Speech Application Programming Interface (SAPI), shipping Microsoft Speech Server, and modernizing spoken language and integrative AI services via Azure AI, which not only enables millions of 3rd party customers but also powers up Microsoft's Windows, Office, Teams, and Azure OpenAI Services. Huang helped Microsoft and Azure Cognitive Services achieve multiple industry's first human parity milestones on the following open research tasks: transcribing conversational speech, machine translation, conversational QnA, and computer vision image captioning. Huang has made significant contributions to the software and AI industry through his executive leadership and his scientific publications, owning more than 170 US patents and impacting billions through Azure AI enabled products and services. In 2016, Wired magazine named him one of 25 Geniuses. In 2021, Azure AI was named the winner of InfoWorld's Technology of the Year Award. Huang was awarded the Allen Newell research excellence medal in 1992, and IEEE Speech Processing Best Paper in 1993. He was recognized as an IEEE Fellow by Institute of Electrical and Electronics Engineers in 2000, named ACM Fellow by Association for Computing Machinery in 2017, and a member of Washington State Academy of Sciences. Huang received 2022 Asian American Corporate Leadership Award, and IEEE Amar Bose Industrial Leader Award. In 2023, he was elected a member of the US National Academy of Engineering (NAE), and a member of the American Academy of Arts and Sciences.

  Read more →
• 

  P4-metric

  The P4 metric (also known as FS or Symmetric F ) enables performance evaluation of a binary classifier. The P4 metric is calculated from precision, recall, specificity, and NPV (negative predictive value). The definition of the P4 metric is similar to that of the F1 metric, however the P4 metric definition addresses criticisms leveled against the definition of the F1 metric. The definition of the P4 metric may, therefore, be understood as an extension of the F1 metric. Like the other known metrics, the P4 metric is a function of: TP (true positives), TN (true negatives), FP (false positives), FN (false negatives). == Justification == The key concept of the P4 metric is to leverage the four key conditional probabilities: P ( + ∣ C + ) {\displaystyle P(+\mid C{+})} — the probability that the sample is positive, provided the classifier result was positive. P ( C + ∣ + ) {\displaystyle P(C{+}\mid +)} — the probability that the classifier result will be positive, provided the sample is positive. P ( C − ∣ − ) {\displaystyle P(C{-}\mid -)} — the probability that the classifier result will be negative, provided the sample is negative. P ( − ∣ C − ) {\displaystyle P(-\mid C{-})} — the probability the sample is negative, provided the classifier result was negative. The main assumption behind this metric is that all the probabilities mentioned above are close to 1 for a properly designed binary classifier. Indeed, P 4 = 1 {\displaystyle \mathrm {P} _{4}=1} if, and only if, all of the probabilities above are equal to 1. Another important feature is that P 4 {\displaystyle \mathrm {P} _{4}} tends to zero any of the above probabilities tend to zero. == Definition == P4 is defined as a harmonic mean of four key conditional probabilities: P 4 = 4 1 P ( + ∣ C + ) + 1 P ( C + ∣ + ) + 1 P ( C − ∣ − ) + 1 P ( − ∣ C − ) = 4 1 p r e c i s i o n + 1 r e c a l l + 1 s p e c i f i c i t y + 1 N P V . {\displaystyle \mathrm {P} _{4}={\frac {4}{{\frac {1}{P(+\mid C{+})}}+{\frac {1}{P(C{+}\mid +)}}+{\frac {1}{P(C{-}\mid -)}}+{\frac {1}{P(-\mid C{-})}}}}={\frac {4}{{\frac {1}{\mathit {precision}}}+{\frac {1}{\mathit {recall}}}+{\frac {1}{\mathit {specificity}}}+{\frac {1}{\mathit {NPV}}}}}.} In terms of TP,TN,FP,FN it can be calculated as follows: P 4 = 4 ⋅ T P ⋅ T N 4 ⋅ T P ⋅ T N + ( T P + T N ) ⋅ ( F P + F N ) . {\displaystyle \mathrm {P} _{4}={\frac {4\cdot \mathrm {TP} \cdot \mathrm {TN} }{4\cdot \mathrm {TP} \cdot \mathrm {TN} +(\mathrm {TP} +\mathrm {TN} )\cdot (\mathrm {FP} +\mathrm {FN} )}}.} == Evaluation of the binary classifier performance == Evaluating the performance of binary classifiers is a multidisciplinary concept. It spans from the evaluation of medical tests, psychiatric tests to machine learning classifiers from a variety of fields. Thus, many of the metrics in use exist under several names, some defined independently. == Properties of P4 metric == Symmetry — contrasting to the F1 metric, P4 is symmetrical. It means - it does not change its value when dataset labeling is changed - positives named negatives and negatives named positives. Range: P 4 ∈ [ 0 , 1 ] {\displaystyle \mathrm {P} _{4}\in [0,1]} . Achieving P 4 ≈ 1 {\displaystyle \mathrm {P} _{4}\approx 1} requires all the key four conditional probabilities being close to 1. For P 4 ≈ 0 {\displaystyle \mathrm {P} _{4}\approx 0} it is sufficient that one of the key four conditional probabilities is close to 0. == Examples, comparing with the other metrics == Dependency table for selected metrics ("true" means depends, "false" - does not depend): Metrics that do not depend on a given probability are prone to misrepresentation when the probability approaches 0. === Example 1: Rare disease detection test === Let us consider a medical test used to detect a rare disease. Suppose a population size of 100000 and 0.05% of the population is infected. Further suppose the following test performance: 95% of all positive individuals are classified correctly (TPR=0.95) and 95% of all negative individuals are classified correctly (TNR=0.95). In such a case, due to high population imbalance and in spite of having high test accuracy (0.95), the probability that an individual who has been classified as positive is in fact positive is very low: P ( + ∣ C + ) = 0.0095. {\displaystyle P(+\mid C{+})=0.0095.} We can observe how this low probability is reflected in some of the metrics: P 4 = 0.0370 {\displaystyle \mathrm {P} _{4}=0.0370} , F 1 = 0.0188 {\displaystyle \mathrm {F} _{1}=0.0188} , J = 0.9100 {\displaystyle \mathrm {J} =\mathbf {0.9100} } (Informedness / Youden index), M K = 0.0095 {\displaystyle \mathrm {MK} =0.0095} (Markedness). === Example 2: Image recognition — cats vs dogs === Consider the problem of training a neural network based image classifier with only two types of images: those containing dogs (labeled as 0) and those containing cats (labeled as 1). Thus, the goal is to distinguish between the cats and dogs. Suppose that the classifier overpredicts in favour of cats ("positive" samples): 99.99% of cats are classified correctly and only 1% of dogs are classified correctly. Further, suppose that the image dataset consists of 100000 images, 90% of which are pictures of cats and 10% are pictures of dogs. In this situation, the probability that the picture containing dog will be classified correctly is pretty low: P ( C − | − ) = 0.01. {\displaystyle P(C-|-)=0.01.} Not all metrics are notice this low probability: P 4 = 0.0388 {\displaystyle \mathrm {P} _{4}=0.0388} , F 1 = 0.9478 {\displaystyle \mathrm {F} _{1}=\mathbf {0.9478} } , J = 0.0099 {\displaystyle \mathrm {J} =0.0099} (Informedness / Youden index), M K = 0.8183 {\displaystyle \mathrm {MK} =\mathbf {0.8183} } (Markedness).

  Read more →
• 

  Language model

  A language model is a computational model that predicts sequences in natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, information retrieval and disaster response. Large language models (LLMs), currently their most advanced form as of 2026, are predominantly based on transformers trained on larger datasets (frequently using texts scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as the word n-gram language model. == History == Noam Chomsky did pioneering work on language models in the 1950s by developing a theory of formal grammars. In 1980, statistical approaches were explored and found to be more useful for many purposes than rule-based formal grammars. Discrete representations like word n-gram language models, with probabilities for discrete combinations of words, made significant advances. In the 2000s, continuous representations for words, such as word embeddings, began to replace discrete representations. Typically, the representation is a real-valued vector that encodes a word’s meaning such that words closer in vector space are similar in meaning and common relationships between words, such as plurality or gender, are preserved. == Pure statistical models == In 1980, the first significant statistical language model was proposed, and during the decade IBM performed 'Shannon-style' experiments, in which potential sources for language modeling improvement were identified by observing and analyzing the performance of human subjects in predicting or correcting text. === Models based on word n-grams === === Exponential === Maximum entropy language models encode the relationship between a word and the n-gram history using feature functions. The equation is P ( w m ∣ w 1 , … , w m − 1 ) = 1 Z ( w 1 , … , w m − 1 ) exp ⁡ ( a T f ( w 1 , … , w m ) ) {\displaystyle P(w_{m}\mid w_{1},\ldots ,w_{m-1})={\frac {1}{Z(w_{1},\ldots ,w_{m-1})}}\exp(a^{T}f(w_{1},\ldots ,w_{m}))} where Z ( w 1 , … , w m − 1 ) {\displaystyle Z(w_{1},\ldots ,w_{m-1})} is the partition function, a {\displaystyle a} is the parameter vector, and f ( w 1 , … , w m ) {\displaystyle f(w_{1},\ldots ,w_{m})} is the feature function. In the simplest case, the feature function is just an indicator of the presence of a certain n-gram. It is helpful to use a prior on a {\displaystyle a} or some form of regularization. The log-bilinear model is another example of an exponential language model. === Skip-gram model === == Neural models == === Recurrent neural network === Continuous representations or embeddings of words are produced in recurrent neural network-based language models (known also as continuous space language models). Such continuous space embeddings help to alleviate the curse of dimensionality, which is the consequence of the number of possible sequences of words increasing exponentially with the size of the vocabulary, further causing a data sparsity problem. Neural networks avoid this problem by representing words as non-linear combinations of weights in a neural net. === Large language models === Although sometimes matching human performance, it is not clear whether they are plausible cognitive models. At least for recurrent neural networks, it has been shown that they sometimes learn patterns that humans do not, but fail to learn patterns that humans typically do. == Evaluation and benchmarks == Evaluation of the quality of language models is mostly done by comparison to human created sample benchmarks created from typical language-oriented tasks. Other, less established, quality tests examine the intrinsic character of a language model or compare two such models. Since language models are typically intended to be dynamic and to learn from data they see, some proposed models investigate the rate of learning, e.g., through inspection of learning curves. Various data sets have been developed for use in evaluating language processing systems. These include: Massive Multitask Language Understanding (MMLU) Corpus of Linguistic Acceptability GLUE benchmark Microsoft Research Paraphrase Corpus Multi-Genre Natural Language Inference Question Natural Language Inference Quora Question Pairs Recognizing Textual Entailment Semantic Textual Similarity Benchmark SQuAD question answering Test Stanford Sentiment Treebank Winograd NLI BoolQ, PIQA, SIQA, HellaSwag, WinoGrande, ARC, OpenBookQA, NaturalQuestions, TriviaQA, RACE, BIG-bench hard, GSM8k, RealToxicityPrompts, WinoGender, CrowS-Pairs

  Read more →
• 

  List of publications in data science

  This is a list of publications in data science, generally organized by order of use in a data analysis workflow. See the list of publications in statistics for more research-based and fundamental publications; while this list is more applied, business oriented, and cross-disciplinary. General article inclusion criteria are: Papers from notable practitioners or notable professors, either with a Wikipedia page or reference to their notability Common knowledge all data professionals should know, with references validating this claim Highly cited applied statistics and machine learning publications Discussion-facilitating papers on the field of data science as a whole (for example, the Attention Is All You Need paper is arguably a landmark paper that can be added here, but it is specific to generative artificial intelligence, not for all practitioners of data) Some reasons why a particular publication might be regarded as important: Topic creator – A publication that created a new topic Breakthrough – A publication that changed scientific knowledge significantly Influence – A publication which has significantly influenced the world or has had a massive impact on the teaching of data science. When possible, a reference is used to validate the inclusion of the publication in this list. == History == Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) Author: Leo Breiman Publication data: Online version: https://projecteuclid.org/journals/statistical-science/volume-16/issue-3/Statistical-Modeling--The-Two-Cultures-with-comments-and-a/10.1214/ss/1009213726.pdf Description: Describes two cultures of statistics, one using a parsimonious and generative stochastic model, while the other is an algorithmic model with no known mechanism for how the data is generated. Breiman argues that while statistics has traditionally favored using the stochastic model, there is value in expanding the methods that statisticians can use to study phenomenon. Importance: Influence on the philosophies of statisticians right before the increased use of machine learning and deep learning methods. In a 20-year retrospective on this article, "Breiman's words are perhaps more relevant than ever". Notable statisticians at the time wrote opinion pieces about the publication. Although overall critical of the publication, David Cox writes that the publication "contains enough truth and exposes enough weaknesses to be thought-provoking." Bradley Efron commented that this publication is a "stimulating paper". Emanuel Parzen also comments about this publication that "Breiman alerts us to systematic blunders (leading to wrong conclusions) that have been committed applying current statistical practice of data modeling". Data Scientist: The Sexiest Job of the 21st Century Author: Thomas H. Davenport and DJ Patil Publication data: Online version: hbr.org/2022/07/is-data-scientist-still-the-sexiest-job-of-the-21st-century Description: Describes the new role at companies that is coined "Data scientist", what they do, how an organization might recruit one to their organization, and how to work with one effectively. Importance: This publication has been an influence on the data community as mentioned near the time it was published in 2012 by institutions like IEEE Spectrum, but also mentioned nearly a decade later asking the same question the title poses. In a retrospective response to their own publication 10 years earlier, authors Davenport and Patil have reflected that the role of a data scientist has "become better institutionalized, the scope of the job has been redefined, the technology it relies on has made huge strides, and the importance of non-technical expertise, such as ethics and change management, has grown". 50 Years of Data Science Author: David Donoho Publication data: Online version: https://www.tandfonline.com/doi/full/10.1080/10618600.2017.1384734 Description: Retrospective discussion paper on the history and origins of data science, with a number of commentary from notable statisticians. Importance: This has been described as "the first in the field to present such a comprehensive and in-depth survey and overview", and helps to define the field that has many definitions. The Composable Data Management System Manifesto Author: Pedro Pedreira, Orri Erling, Konstantinos Karanasos, Scott Schneider, Wes McKinney, Satya R Valluri, Mohamed Zait, Jacques Nadeau Publication data: Online version: https://www.vldb.org/pvldb/vol16/p2679-pedreira.pdf Description: The vision paper advocating for a paradigm shift in how data management systems are designed using standard, composable, interoperable tools rather than siloed software tools. Importance: A paradigm shifting view on how future data science software tools should be designed for more efficient workflows, the principles of which "will be especially crucial for addressing fragmentation, improving interoperability, and promoting user-centricity as data ecosystems grow increasingly complex". == Data collection and organization == Tidy Data Author: Hadley Wickham Publication data: Online version: https://www.jstatsoft.org/article/view/v059i10/ https://vita.had.co.nz/papers/tidy-data.pdf Description: Describes a framework for data cleaning that is summarized in the quote, "each variable is a column, each observation is a row, and each type of observational unit is a table". This allows a standard data structure for which data analysis tools can be consistently built around. Importance: Cited over 1,500 times, this effort for tidy data has been described by David Donoho as having "more impact on today's practice of data analysis than many highly regarded theoretical statistics articles". In the context of data visualization, this publication is said to support "efficient exploration and prototyping because variables can be assigned different roles in the plot without modifying anything about the original dataset". Data Organization in Spreadsheets Author: Karl W. Broman and Kara H. Woo Publication data: Online version: https://www.tandfonline.com/doi/full/10.1080/00031305.2017.1375989 Description: This article offers practical recommendations for organizing data in spreadsheets, like Microsoft Excel and Google Sheets, to reduce errors and lower the barrier for later analyses due to limitations in spreadsheets or quirks in the software. Importance: Influences teaching both data and non-data practitioners to create more analysis-friendly spreadsheets, and has been described to outline "spreadsheet best practices". == Data visualizations == Quantitative Graphics in Statistics: A Brief History Author: James R. Beniger and Dorothy L. Robyn Publication data: Online version: https://www.jstor.org/stable/2683467 Description: Outlines history and evolution of quantitative graphics in statistics, going through spatial organization (17th and 18th centuries), discrete comparison (18th and 19th centuries), continuous distribution (19th century), and multivariate distribution and correlation (late 19th and 20th centuries). Importance: Helps put into perspective for learning data practitioners the recency of graphics that are used. A later publication "Graphical Methods in Statistics" by Stephen Fienberg in 1979 writes that his publication "owes much to the work of Beniger and Robyn". == Practice == Data Science for Business Author: Foster Provost and Tom Fawcett Publication data: Online version: N/A Description: Broadly outlines principles of data science and data-analytic thinking for businesses. Importance: Cited over 3,000 times, it is "highly recommended for students" but also it is also recommended due to its "relevance to senior management leaders who want to build and lead a team of data scientists and implement data science in solving complex business problems". == Tooling == Hidden Technical Debt in Machine Learning Systems Author: D. Sculley, Gary Holy, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-François Crespo, Dan Dennison Publication data: Online version: https://proceedings.neurips.cc/paper_files/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf Description: This paper argues that it is "dangerous to think of [complex machine learning] quick wins as coming for free" and overviews risk factors to account for when implementing a machine learning system. Importance: All authors worked for Google, article is cited over 2,000 times, and helped practitioners thinking about quickly implementing a machine learning tool without understanding the long-term maintenance of the tool. A few useful things to know about machine learning Author: Pedro Domingos Publication data: Online version: https://dl.acm.org/doi/10.1145/2347736.2347755 https://homes.cs.washington.edu/~pedrod/papers/cacm12.pdf Description: The purpose of this paper is to distill inaccessible "folk knowledge" to effectively implement machine learning projects because "machin

  Read more →
• 

  How to Choose an AI Video Editor

  In search of the best AI video editor? An AI video editor is software that uses machine learning to help you get more done — it turns a rough idea into a polished result in seconds. When choosing one, weigh output quality, pricing, export formats, and how well it fits the tools you already use. Whether you are a beginner or a pro, the right AI video editor slots into your workflow and pays for itself fast. We tested the leading options and ranked them by quality, value, and ease of use.

  Read more →
• 

  Is an AI Image Generator Worth It in 2026?

  Comparing the best AI image generator? An AI image generator is software that uses machine learning to help you get more done — it lowers the barrier so anyone can produce professional output. Privacy matters too: check whether your data trains the model and whether a no-log or enterprise tier is available. Whether you are a beginner or a pro, the right AI image generator slots into your workflow and pays for itself fast. We tested the leading options and ranked them by quality, value, and ease of use.

  Read more →
• 

  Stefan Schaal

  Stefan Schaal (born 1961) is a German-American computer scientist specializing in robotics, machine learning, autonomous systems, and computational neuroscience. == Education and career == Schaal was born in Frankfurt am Main in Germany, Schaal grew up in the North Bavarian town of Nürnberg. After graduating from school, he served in the German army in the Ski Patrol Division of Bad Reichenhall, where he honorably discharged with the rank of a Lieutenant. Schaal studied mechanical engineering at the Technical University of Munich, graduating in 1987 with a Diploma degree (summa cum laude). Subsequently, Schaal did his Ph.D. in computer aided design and artificial intelligence at the Technical University of Munich and the Massachusetts Institute of Technology, receiving his Ph.D. in 1991 (Summa Cum Laude) under Klaus Ehrlenspiel. In 1991, Schaal was a Postdoctoral Fellow at the Department and Brain and Cognitive Science and the Artificial Intelligence Lab at the Massachusetts Institute of Technology, funded by the Alexander von Humboldt Foundation and the German Academic Scholarship Foundation. Starting from 1992, he became an invited researcher at the ATR Computational Neuroscience Labs in Japan, where he created a robotics lab focusing on biological principles of motor control and learning. In 1994, Schaal moved to the Georgia Institute of Technology as an adjunct assistant professor, and also held the same rank at the Pennsylvania State University. In 1996, Schaal assumed a group leader position in the ERATO Kawato Dynamic Brain Project in Japan. Schaal joined the University of Southern California (USC) in 1997, where he advanced from the ranks of assistant professor, to associate professor, to full professor. In 2009, Schaal became a founder in defining and creating the Max Planck Institute for Intelligent Systems in Tübingen and Stuttgart, Germany, an institute focusing on principles of perception-action-learning systems in synthetic intelligence. In 2012, Schaal founded the Autonomous Motion Department (AMD) at this institute, while maintaining a partial appointment at USC. Stefan Schaal joined Google X as lead of a robotics research team in late 2018. == Research == Stefan Schaal's interests focus on autonomous perception-action-learning systems, in particular anthropomorphic robotic systems. He works on topics of machine learning for control, control theory, computational neuroscience for neuromotor control, experimental robotics, reinforcement learning, artificial intelligence, and nonlinear dynamical systems. Stefan has co-authored more than 400 publications in top conferences and journals, and served as organizer on various top conferences in machine learning and robotics. He has received numerous best paper awards and honors in his scientific community. Stefan Schaal has been noted as one of the five leaders in robotics in 2011, and among the top robotics experts in the world. == Controversy == In 2018, the German newsjournal Der Spiegel published an article reporting on his double affiliation with USC and the Max-Planck Society, both with full salaries, which was apparently unknown to either party. Schaal rejected the allegations, but was forced to leave his position at the Max Planck Institute.

  Read more →
• 

  Cache language model

  A cache language model is a type of statistical language model. These occur in the natural language processing subfield of computer science and assign probabilities to given sequences of words by means of a probability distribution. Statistical language models are key components of speech recognition systems and of many machine translation systems: they tell such systems which possible output word sequences are probable and which are improbable. The particular characteristic of a cache language model is that it contains a cache component and assigns relatively high probabilities to words or word sequences that occur elsewhere in a given text. The primary, but by no means sole, use of cache language models is in speech recognition systems. To understand why it is a good idea for a statistical language model to contain a cache component one might consider someone who is dictating a letter about elephants to a speech recognition system. Standard (non-cache) N-gram language models will assign a very low probability to the word "elephant" because it is a very rare word in English. If the speech recognition system does not contain a cache component, the person dictating the letter may be annoyed: each time the word "elephant" is spoken another sequence of words with a higher probability according to the N-gram language model may be recognized (e.g., "tell a plan"). These erroneous sequences will have to be deleted manually and replaced in the text by "elephant" each time "elephant" is spoken. If the system has a cache language model, "elephant" will still probably be misrecognized the first time it is spoken and will have to be entered into the text manually; however, from this point on the system is aware that "elephant" is likely to occur again – the estimated probability of occurrence of "elephant" has been increased, making it more likely that if it is spoken it will be recognized correctly. Once "elephant" has occurred several times, the system is likely to recognize it correctly every time it is spoken until the letter has been completely dictated. This increase in the probability assigned to the occurrence of "elephant" is an example of a consequence of machine learning and more specifically of pattern recognition. There exist variants of the cache language model in which not only single words but also multi-word sequences that have occurred previously are assigned higher probabilities (e.g., if "San Francisco" occurred near the beginning of the text subsequent instances of it would be assigned a higher probability). The cache language model was first proposed in a paper published in 1990, after which the IBM speech-recognition group experimented with the concept. The group found that implementation of a form of cache language model yielded a 24% drop in word-error rates once the first few hundred words of a document had been dictated. A detailed survey of language modeling techniques concluded that the cache language model was one of the few new language modeling techniques that yielded improvements over the standard N-gram approach: "Our caching results show that caching is by far the most useful technique for perplexity reduction at small and medium training data sizes". The development of the cache language model has generated considerable interest among those concerned with computational linguistics in general and statistical natural language processing in particular: recently, there has been interest in applying the cache language model in the field of statistical machine translation. The success of the cache language model in improving word prediction rests on the human tendency to use words in a "bursty" fashion: when one is discussing a certain topic in a certain context, the frequency with which one uses certain words will be quite different from their frequencies when one is discussing other topics in other contexts. The traditional N-gram language models, which rely entirely on information from a very small number (four, three, or two) of words preceding the word to which a probability is to be assigned, do not adequately model this "burstiness". Recently, the cache language model concept – originally conceived for the N-gram statistical language model paradigm – has been adapted for use in the neural paradigm. For instance, recent work on continuous cache language models in the recurrent neural network (RNN) setting has applied the cache concept to much larger contexts than before, yielding significant reductions in perplexity. Another recent line of research involves incorporating a cache component in a feed-forward neural language model (FN-LM) to achieve rapid domain adaptation.

  Read more →
• 

  Nick Frosst

  Nicholas M. W. Frosst is a Canadian computer scientist and musician. He co-founded Cohere, a Toronto-based artificial intelligence company. He is also the lead singer in the indie rock band Good Kid. == Early life and education == Frosst was born on January 5, 1993. Frosst earned a Bachelor of Science degree in computer science and cognitive science from the University of Toronto in 2015. He was a student of Geoffrey Hinton, who also hired Frosst at Google Brain. == Career == Frosst was among Geoffrey Hinton's earliest hires at Google Brain in Toronto, working as a machine learning researcher on deep learning and neural network architectures. He worked there from 2016 to 2020. Frosst co-founded Cohere with Aidan Gomez and Ivan Zhang in 2019. The company builds large language models and enterprise AI tools. Frosst has publicly explained Cohere's focus on industries like finance and health, where there are privacy and other regulatory considerations. Frosst has also spoken openly about his belief that artificial intelligence will not replace humans, but rather streamline and automate mundane tasks, and his belief that AGI is less "imminent" than many in the field claim. Frosst and the other Cohere co-founders were listed first on Maclean's AI Trailblazers Power List and The Logic's Innovation Leaders. == Music == After spending time in a prior band which played "weird" music featuring a glockenspiel, Frosst and fellow computer science students at the University of Toronto formed the indie rock band Good Kid in 2015. Frosst is the lead vocalist for the band. While on tour with the band, Frosst continues his work in the tech industry remotely. Frosst has described the band as way for him to relax and not constantly think about tech. His vocals have been compared to that of Kele Okereke. As of 2026, the band, which has performed at Lollapalooza, has 3.1 million monthly Spotify listeners. In 2024, the band was nominated for the Juno Awards Breakthrough Group of the Year. == Discography == === Good Kid === Can We Hang Out Sometime? (2026)

  Read more →
• 

  Transfer-based machine translation

  Transfer-based machine translation is a type of machine translation (MT). It is currently one of the most widely used methods of machine translation. In contrast to the simpler direct model of MT, transfer MT breaks translation into three steps: analysis of the source language text to determine its grammatical structure, transfer of the resulting structure to a structure suitable for generating text in the target language, and finally generation of this text. Transfer-based MT systems are thus capable of using knowledge of the source and target languages. == Design == Both transfer-based and interlingua-based machine translation have the same idea: to make a translation it is necessary to have an intermediate representation that captures the "meaning" of the original sentence in order to generate the correct translation. In interlingua-based MT this intermediate representation must be independent of the languages in question, whereas in transfer-based MT, it has some dependence on the language pair involved. The way in which transfer-based machine translation systems work varies substantially, but in general they follow the same pattern: they apply sets of linguistic rules which are defined as correspondences between the structure of the source language and that of the target language. The first stage involves analysing the input text for morphology and syntax (and sometimes semantics) to create an internal representation. The translation is generated from this representation using both bilingual dictionaries and grammatical rules. It is possible with this translation strategy to obtain fairly high quality translations, with accuracy in the region of 90% (although this is highly dependent on the language pair in question, for example the distance between the two). == Operation == In a rule-based machine translation system the original text is first analysed morphologically and syntactically in order to obtain a syntactic representation. This representation can then be refined to a more abstract level putting emphasis on the parts relevant for translation and ignoring other types of information. The transfer process then converts this final representation (still in the original language) to a representation of the same level of abstraction in the target language. These two representations are referred to as "intermediate" representations. From the target language representation, the stages are then applied in reverse. == Analysis and transformation == Various methods of analysis and transformation can be used before obtaining the final result. Along with these statistical approaches may be augmented generating hybrid systems. The methods which are chosen and the emphasis depends largely on the design of the system, however, most systems include at least the following stages: Morphological analysis. Surface forms of the input text are classified as to part-of-speech (e.g. noun, verb, etc.) and sub-category (number, gender, tense, etc.). All of the possible "analyses" for each surface form are typically made output at this stage, along with the lemma of the word. Lexical categorisation. In any given text some of the words may have more than one meaning, causing ambiguity in analysis. Lexical categorisation looks at the context of a word to try to determine the correct meaning in the context of the input. This can involve part-of-speech tagging and word sense disambiguation. Lexical transfer. This is basically dictionary translation; the source language lemma (perhaps with sense information) is looked up in a bilingual dictionary and the translation is chosen. Structural transfer. While the previous stages deal with words, this stage deals with larger constituents, for example phrases and chunks. Typical features of this stage include concordance of gender and number, and re-ordering of words or phrases. Morphological generation. From the output of the structural transfer stage, the target language surface forms are generated. == Transfer types == One of the main features of transfer-based machine translation systems is a phase that "transfers" an intermediate representation of the text in the original language to an intermediate representation of text in the target language. This can work at one of two levels of linguistic analysis, or somewhere in between. The levels are: Superficial transfer (or syntactic). This level is characterised by transferring "syntactic structures" between the source and target languages. It is suitable for languages in the same family or of the same type, for example in the Romance languages between Spanish, Catalan, French, Italian, etc. Deep transfer (or semantic). This level constructs a semantic representation that is dependent on the source language. This representation can consist of a series of structures which represent the meaning. In these transfer systems predicates are typically produced. The translation also typically requires structural transfer. This level is used to translate between more distantly related languages (e.g. Spanish-English or Spanish-Basque, etc.)

  Read more →