Referring expression generation (REG) is the subtask of natural language generation (NLG) that received most scholarly attention. While NLG is concerned with the conversion of non-linguistic information into natural language, REG focuses only on the creation of referring expressions (noun phrases) that identify specific entities called targets. This task can be split into two sections. The content selection part determines which set of properties distinguish the intended target and the linguistic realization part defines how these properties are translated into natural language. A variety of algorithms have been developed in the NLG community to generate different types of referring expressions. == Types of referring expressions == A referring expression (RE), in linguistics, is any noun phrase, or surrogate for a noun phrase, whose function in discourse is to identify some individual object (thing, being, event...) The technical terminology for identify differs a great deal from one school of linguistics to another. The most widespread term is probably refer, and a thing identified is a referent, as for example in the work of John Lyons. In linguistics, the study of reference relations belongs to pragmatics, the study of language use, though it is also a matter of great interest to philosophers, especially those wishing to understand the nature of knowledge, perception and cognition more generally. Various devices can be used for reference: determiners, pronouns, proper names... Reference relations can be of different kinds; referents can be in a "real" or imaginary world, in discourse itself, and they may be singular, plural, or collective. === Pronouns === The simplest type of referring expressions are pronoun such as he and it. The linguistics and natural language processing communities have developed various models for predicting anaphor referents, such as centering theory, and ideally referring-expression generation would be based on such models. However most NLG systems use much simpler algorithms, for example using a pronoun if the referent was mentioned in the previous sentence (or sentential clause), and no other entity of the same gender was mentioned in this sentence. === Definite noun phrases === There has been a considerable amount of research on generating definite noun phrases, such as the big red book. Much of this builds on the model proposed by Dale and Reiter. This has been extended in various ways, for example Krahmer et al. present a graph-theoretic model of definite NP generation with many nice properties. In recent years a shared-task event has compared different algorithms for definite NP generation, using the TUNA corpus. === Spatial and temporal reference === Recently there has been more research on generating referring expressions for time and space. Such references tend to be imprecise (what is the exact meaning of tonight?), and also to be interpreted in different ways by different people. Hence it may be necessary to explicitly reason about false positive vs false negative tradeoffs, and even calculate the utility of different possible referring expressions in a particular task context. === Criteria for good expressions === Ideally, a good referring expression should satisfy a number of criteria: Referential success: It should unambiguously identify the referent to the reader. Ease of comprehension: The reader should be able to quickly read and understand it. Computational complexity: The generation algorithm should be fast No false inferences: The expression should not confuse or mislead the reader by suggesting false implicatures or other pragmatic inferences. For example, a reader may be confused if he is told Sit by the brown wooden table in a context where there is only one table. == History == === Pre-2000 era === REG goes back to the early days of NLG. One of the first approaches was done by Winograd in 1972 who developed an "incremental" REG algorithm for his SHRDLU program. Afterwards researchers started to model the human abilities to create referring expressions in the 1980s. This new approach to the topic was influenced by the researchers Appelt and Kronfeld who created the programs KAMP and BERTRAND and considered referring expressions as parts of bigger speech acts. Some of their most interesting findings were the fact that referring expressions can be used to add information beyond the identification of the referent as well as the influence of communicative context and the Gricean maxims on referring expressions. Furthermore, its skepticism concerning the naturalness of minimal descriptions made Appelt and Kronfeld's research a foundation of later work on REG. The search for simple, well-defined problems changed the direction of research in the early 1990s. This new approach was led by Dale and Reiter who stressed the identification of the referent as the central goal. Like Appelt they discuss the connection between the Gricean maxims and referring expressions in their culminant paper in which they also propose a formal problem definition. Furthermore, Reiter and Dale discuss the Full Brevity and Greedy Heuristics algorithms as well as their Incremental Algorithm(IA) which became one of the most important algorithms in REG. === Later developments === After 2000 the research began to lift some of the simplifying assumptions, that had been made in early REG research in order to create more simple algorithms. Different research groups concentrated on different limitations creating several expanded algorithms. Often these extend the IA in a single perspective for example in relation to: Reference to Sets like "the t-shirt wearers" or "the green apples and the banana on the left" Relational Descriptions like "the cup on the table" or "the woman who has three children" Context Dependency, Vagueness and Gradeability include statements like "the older man" or "the car on the left" which are often unclear without a context Salience and Generation of Pronouns are highly discourse dependent making for example "she" a reference to "the (most salient) female person" Many simplifying assumptions are still in place or have just begun to be worked on. Also a combination of the different extensions has yet to be done and is called a "non-trivial enterprise" by Krahmer and van Deemter. Another important change after 2000 was the increasing use of empirical studies in order to evaluate algorithms. This development took place due to the emergence of transparent corpora. Although there are still discussions about what the best evaluation metrics are, the use of experimental evaluation has already led to a better comparability of algorithms, a discussion about the goals of REG and more task-oriented research. Furthermore, research has extended its range to related topics such as the choice of Knowledge Representation(KR) Frameworks. In this area the main question, which KR framework is most suitable for the use in REG remains open. The answer to this question depends on how well descriptions can be expressed or found. A lot of the potential of KR frameworks has been left unused so far. Some of the different approaches are the usage of: Graph search which treats relations between targets in the same way as properties. Constraint Satisfaction which allows for a separation between problem specification and the implementation. Modern Knowledge Representation which offers logical inference in for example Description Logic or Conceptual Graphs. == Problem definition == Dale and Reiter (1995) think about referring expressions as distinguishing descriptions. They define: The referent as the entity that should be described The context set as set of salient entities The contrast set or potential distractors as all elements of the context set except the referent A property as a reference to a single attribute–value pair Each entity in the domain can be characterised as a set of attribute–value pairs for example ⟨ {\displaystyle \langle } type, dog ⟩ {\displaystyle \rangle } , ⟨ {\displaystyle \langle } gender, female ⟩ {\displaystyle \rangle } or ⟨ {\displaystyle \langle } age, 10 years ⟩ {\displaystyle \rangle } . The problem then is defined as follows: Let r {\displaystyle r} be the intended referent, and C {\displaystyle C} be the contrast set. Then, a set L {\displaystyle L} of attribute–value pairs will represent a distinguishing description if the following two conditions hold: Every attribute–value pair in L {\displaystyle L} applies to r {\displaystyle r} : that is, every element of L {\displaystyle L} specifies an attribute–value that r {\displaystyle r} possesses. For every member c {\displaystyle c} of C {\displaystyle C} , there is at least one element l {\displaystyle l} of L {\displaystyle L} that does not apply to c {\displaystyle c} : that is, there is an l {\displaystyle l} in L {\displaystyle L} that specifies an attribute–value that c {\displaystyle c} does not possess. l {\displaystyle l} is said
Corpus of Linguistic Acceptability
Corpus of Linguistic Acceptability (CoLA) is a dataset the primary purpose of which is to serve as a benchmark for evaluating the ability of artificial neural networks, including large language models, to judge the grammatical correctness of sentences. It consists of 10,657 English sentences from published linguistics literature that were manually labeled either as grammatical or ungrammatical. == Public version == The publicly available version of CoLA contains 9,594 sentences that belong to training and development sets. It excludes 1,063 sentences reserved for a held-out test set.
Turing test
The Turing test, originally called the imitation game by Alan Turing in 1949, is a test of a machine's ability to exhibit intelligent behaviour equivalent to that of a human. In the test, a human evaluator judges a text transcript of a natural-language conversation between a human and a machine. The evaluator tries to identify the machine, and the machine passes if the evaluator cannot reliably tell them apart. The results would not depend on the machine's ability to answer questions correctly, only on how closely its answers resembled those of a human. Since the Turing test is a test of indistinguishability in performance capacity, the verbal version generalizes naturally to all of human performance capacity, verbal as well as nonverbal (robotic). The test was introduced by Turing in his 1950 paper "Computing Machinery and Intelligence" while working at the University of Manchester. It opens with the words: "I propose to consider the question, 'Can machines think?'." Because "thinking" is difficult to define, Turing chooses to "replace the question by another, which is closely related to it and is expressed in relatively unambiguous words". Turing describes the new form of the problem in terms of a three-person party game called the "imitation game", in which an interrogator asks questions of a man and a woman in another room in order to determine the correct sex of the two players. Turing's new question is: "Are there imaginable digital computers which would do well in the imitation game?" This question, Turing believed, was one that could actually be answered. In the remainder of the paper, he argued against the major objections to the proposition that "machines can think". Since Turing introduced his test, it has been highly influential in the philosophy of artificial intelligence, resulting in substantial discussion and controversy, as well as criticism from philosophers like John Searle, who argue against the test's ability to detect consciousness. == History == === Philosophical background === The question of whether it is possible for machines to think has a long history, which is firmly entrenched in the distinction between dualist and materialist views of the mind. René Descartes prefigures aspects of the Turing test in his 1637 Discourse on the Method when he writes: [H]ow many different automata or moving machines could be made by the industry of man ... For we can easily understand a machine's being constituted so that it can utter words, and even emit some responses to action on it of a corporeal kind, which brings about a change in its organs; for instance, if touched in a particular part it may ask what we wish to say to it; if in another part it may exclaim that it is being hurt, and so on. But it never happens that it arranges its speech in various ways, in order to reply appropriately to everything that may be said in its presence, as even the lowest type of man can do. Here Descartes notes that automata are capable of responding to human interactions but argues that such automata cannot respond appropriately to things said in their presence in the way that any human can. Descartes therefore prefigures the Turing test by defining the insufficiency of appropriate linguistic response as that which separates the human from the automaton. Descartes fails to consider the possibility that future automata might be able to overcome such insufficiency, and so does not propose the Turing test as such, even if he prefigures its conceptual framework and criterion. Denis Diderot formulates in his 1746 book Pensées philosophiques a Turing-test criterion, though with the important implicit limiting assumption maintained, of the participants being natural living beings, rather than considering created artifacts: If they find a parrot who could answer to everything, I would claim it to be an intelligent being without hesitation. This does not mean he agrees with this, but that it was already a common argument of materialists at that time. According to dualism, the mind is non-physical (or, at the very least, has non-physical properties) and, therefore, cannot be explained in purely physical terms. According to materialism, the mind can be explained physically, which leaves open the possibility of minds that are produced artificially. In 1936, philosopher Alfred Ayer considered the standard philosophical question of other minds: how do we know that other people have the same conscious experiences that we do? In his book, Language, Truth and Logic, Ayer suggested a protocol to distinguish between a conscious man and an unconscious machine: "The only ground I can have for asserting that an object which appears to be conscious is not really a conscious being, but only a dummy or a machine, is that it fails to satisfy one of the empirical tests by which the presence or absence of consciousness is determined". (This suggestion is very similar to the Turing test, but it is not certain that Ayer's popular philosophical classic was familiar to Turing.) In other words, a thing is not conscious if it fails the consciousness test. === Cultural background === A rudimentary idea of the Turing test appears in the 1726 novel Gulliver's Travels by Jonathan Swift. When Gulliver is brought before the king of Brobdingnag, the king thinks at first that Gulliver might be a "a piece of clock-work (which is in that country arrived to a very great perfection) contrived by some ingenious artist". Even when he hears Gulliver speaking, the king still doubts whether Gulliver was taught "a set of words" to make him "sell at a better price". Gulliver tells that only after "he put several other questions to me, and still received rational answers" the king became satisfied that Gulliver was not a machine. Tests where a human judges whether a computer or an alien is intelligent were an established convention in science fiction by the 1940s, and it is likely that Turing would have been aware of these. Stanley G. Weinbaum's "A Martian Odyssey" (1934) provides an example of how nuanced such tests could be. Earlier examples of machines or automatons attempting to pass as human include the Ancient Greek myth of Pygmalion who creates a sculpture of a woman that is animated by Aphrodite, Carlo Collodi's novel The Adventures of Pinocchio, about a puppet who wants to become a real boy, and E. T. A. Hoffmann's 1816 story "The Sandman," where the protagonist falls in love with an automaton. In all these examples, people are fooled by artificial beings that—up to a point—pass as human. === Alan Turing and the imitation game === Researchers in the United Kingdom had been exploring "machine intelligence" for up to ten years prior to the founding of the field of artificial intelligence (AI) research in 1956. It was a common topic among the members of the Ratio Club, an informal group of British cybernetics and electronics researchers that included Alan Turing. Turing, in particular, had been running the notion of machine intelligence since at least 1941 and one of the earliest-known mentions of "computer intelligence" was made by him in 1947. In Turing's report, "Intelligent Machinery," he investigated "the question of whether or not it is possible for machinery to show intelligent behaviour" and, as part of that investigation, proposed what may be considered the forerunner to his later tests: It is not difficult to devise a paper machine which will play a not very bad game of chess. Now get three men A, B and C as subjects for the experiment. A and C are to be rather poor chess players, B is the operator who works the paper machine. ... Two rooms are used with some arrangement for communicating moves, and a game is played between C and either A or the paper machine. C may find it quite difficult to tell which he is playing. "Computing Machinery and Intelligence" (1950) was the first published paper by Turing to focus exclusively on machine intelligence. Turing begins the 1950 paper with the claim, "I propose to consider the question 'Can machines think?'" As he highlights, the traditional approach to such a question is to start with definitions, defining both the terms "machine" and "think". Turing chooses not to do so; instead, he replaces the question with a new one, "which is closely related to it and is expressed in relatively unambiguous words". In essence he proposes to change the question from "Can machines think?" to "Can machines do what we (as thinking entities) can do?" The advantage of the new question, Turing argues, is that it draws "a fairly sharp line between the physical and intellectual capacities of a man". To demonstrate this approach Turing proposes a test inspired by a party game, known as the "imitation game", in which a man and a woman go into separate rooms and guests try to tell them apart by writing a series of questions and reading the typewritten answers sent back. In this game, both the man and the woman aim to convince the guests that they ar
Script theory
Script theory is a psychological theory which posits that human behaviour largely falls into patterns called scripts because they function the way a written script does, by providing a program for action. Silvan Tomkins created script theory as a further development of his affect theory, which regards human beings' emotional responses to stimuli as falling into categories called affects: he noticed that the purely biological response of affect may be followed by awareness and by what we cognitively do in terms of acting on that affect, so that more was needed to produce a complete explanation of what he called human being theory. These scripts fall under the larger cognitive concept called schemas, which are organized chunks of information. A schema is a script that has the potential to lack the specificity of the sequence of events. A schema becomes a script is when there is an ordering to it that requires action, such as the process of starting a car (get in, put on the seatbelt, turn the car on, release the emergency brake, etc.). In script theory, the basic unit of analysis is called a scene, defined as a sequence of events linked by the affects triggered during the experience of those events. Tomkins recognized that affective experiences fall into patterns that we may group together according to criteria, such as the types of persons and places involved and the degree of intensity of the effect experienced—the patterns of which constitute scripts that inform behavior in an effort to maximize positive affect and to minimize negative affect. == In artificial intelligence == Roger Schank, Robert P. Abelson and their research group extended Tomkins' scripts and used them in early artificial intelligence work as a method of representing procedural knowledge. In their work, scripts are very much like frames, except the values that fill the slots must be ordered. A script is a structured representation describing a stereotyped sequence of events in a particular context. Scripts are used in natural-language understanding systems to organize a knowledge base in terms of the situations that the system should understand. The classic example of a script involves the typical sequence of events that occur when a person drinks in a restaurant: finding a seat, reading the menu, ordering drinks from the waitstaff, etc. In the script form, these would be decomposed into conceptual transitions, such as MTRANS and PTRANS, which refer to mental transitions [of information] and physical transitions [of things]. Schank, Abelson and their colleagues tackled some of the most difficult problems in artificial intelligence (i.e., story understanding), but ultimately their line of work ended without tangible success. This type of work received little attention after the 1980s, but became very influential in later knowledge representation techniques, such as case-based reasoning. Scripts can be inflexible. To deal with inflexibility, smaller modules called memory organization packets (MOP) can be combined in a way that is appropriate for the situation.
OpenAI Operator
OpenAI Operator was an AI agent developed by OpenAI, capable of autonomously performing tasks through web browser interactions, including filling forms, placing online orders, scheduling appointments, and other repetitive browser-based tasks. It uses OpenAI's advanced models to expand practical automation capabilities for users in daily activities. Operator was launched on January 23, 2025. It was released as a limited-access research preview to ChatGPT Pro-tier subscribers in the United States on February 1, 2025, with future plans to broaden availability. Operator was deprecated after the release of ChatGPT agent, and shut down on August 31, 2025. == Performance and limitations == In benchmark assessments, Operator achieved notable success, scoring 38.1% on OSWorld benchmarks (OS-level tasks) and 58.1% on WebArena benchmarks (web interactions). However, it did not reach human-level accuracy and faced limitations with intricate user interfaces and extended workflows. == Safety and privacy == OpenAI emphasized privacy and safety measures within Operator, including stringent data protection protocols and built-in safety checks designed to prevent unauthorized sensitive actions or information misuse. == Availability == Initially, Operator was only available to ChatGPT Pro subscribers in the U.S., with plans for broader availability to Plus, Team, and Enterprise users in the future.
Grammatik
Grammatik was the first grammar-checking program for home computers. Aspen Software of Albuquerque, NM, released the earliest version of this diction and style checker for personal computers. It was first released no later than 1981, and was inspired by the Writer's Workbench. Grammatik was first available for the TRS-80, and soon had versions for CP/M and the IBM PC. Reference Software International of San Francisco, California, acquired Grammatik in 1985. Development of Grammatik continued, and it became an actual grammar checker that could detect writing errors beyond simple style checking. Subsequent versions were released for MS-DOS, Windows, Macintosh, and Unix. Grammatik was ultimately acquired by WordPerfect Corporation and is integrated into the WordPerfect word processor.
Colossus (supercomputer)
Colossus is a supercomputer developed by xAI. Construction began in 2024 in Memphis, Tennessee; the system became operational in July 2024. It is currently the world's largest AI supercomputer. Colossus's primary purpose is to train the company's chatbot, Grok. In addition, Colossus provides computing support to the social-media platform X and to other projects of Elon Musk, such as SpaceX. In 2025, it expanded to neighboring Southaven, Mississippi across the Tennessee–Mississippi border. As of May 6, 2026, Anthropic has agreed to rent all compute capacity at the Colossus 1 data center. == Background == Colossus was launched in September 2024 at a former Electrolux site in South Memphis to train the AI language model Grok. Within 19 days of the project's conception, xAI was ready to begin construction. The site was chosen because the abandoned Electrolux building could be repurposed to expedite construction and its proximity to a nearby wastewater treatment facility provided a water source. As of February 2025, xAI plans to build an $80 million facility to process additional wastewater for use at the supercomputer. === xAI === Musk incorporated xAI in March 2023 with the stated purpose of understanding the "nature of the universe". The team includes former members of OpenAI, DeepMind, Microsoft, and Tesla. Musk was one of the founding members of the company OpenAI, investing up to US$45 million in 2015. He left OpenAI in 2018, reportedly to avoid conflicts of interest with Tesla. It has also been reported that he had made a bid for leadership at OpenAI and left when his proposal was rejected. The exact reasons for his departure from the company are unclear. Both Dell Technologies and Supermicro partnered with xAI to build the supercomputer. It was originally powered by 100,000 Nvidia graphics processing units (GPUs) and was constructed in 122 days. 3 months after the first 100,000 GPUs were deployed, xAI announced that they had increased the system to 200,000 GPUs and that they intended to continue increasing the computer's processing power to 1 million GPUs. As of April 2025, xAI claimed Colossus was the largest AI training platform in the world. == Choice of location == xAI selected Memphis, in southwestern Tennessee, as the site for Colossus in part because an existing industrial facility allowed the project to proceed more quickly than constructing a new data center. Elon Musk was initially told that building a data center would take 18–24 months. The company instead searched for a vacant facility and selected the former Electrolux factory in Memphis. Electrolux opened the facility in 2012 and operated it for about eight years before closing it in 2020 after relocating operations to Springfield, Tennessee. The building covered 785,000 sq ft (72,900 m2) and had been purchased by Phoenix Investors in December 2023 for $35 million . Because the structure was already in place, work on the supercomputer could begin immediately rather than waiting for a new facility to be constructed. According to Forbes, xAI considered seven or eight other sites before selecting Memphis, and Musk finalized the decision to build in Memphis in about a week. The decision was finalized in March 2024, after which construction began. xAI publicly announced in June 2024 that Colossus would be built in Memphis. The building itself was not the only reason xAI selected Memphis. According to the Greater Memphis Chamber, the company chose the city because of its "reliable power grid, ability to create a water recycling facility, proximity to the Mississippi River and ample land". The city was also able to provide the large amounts of electricity and water needed to operate the supercomputer. At full capacity, the system was expected to require 150 megawatts of electricity and millions of gallons of water per day. The project also relied on partnerships with local and regional organizations including Memphis Light, Gas and Water (MLGW), Tennessee Valley Authority (TVA), the City of Memphis, and Shelby County. The city also provided financial incentives for the project. == Environmental impact == AI data centers consume large amounts of energy. At the site of Colossus in South Memphis, the grid connection was only 8 MW, so xAI applied to temporarily set up more than a dozen gas turbines (Voltagrid’s 2.5 MW units and Solar Turbines’ 16 MW SMT-130s) which would steadily burn methane gas from a 16-inch natural gas main. Aerial imagery in April 2025 showed 35 gas turbines had been set up at a combined 422 MW. These turbines have been estimated to generate about "72 megawatts, which is approximately 3% of the (TVA) power grid". The higher number of gas turbines and the subsequent emissions requires xAI to have a major source permit. In Memphis, xAI was able to avoid some environmental rules in the construction of Colossus, such as operating without permits for the on-site methane gas turbines because they are "portable". The Shelby County Health Department told NPR that "it only regulates gas-burning generators if they're in the same location for more than 364 days". However, in a January 2026 ruling, the EPA revised its New Source Performance Standard and announced that large methane gas turbines require permits even for temporary operations. In November 2024, the grid connection was upgraded to 150 MW, and some turbines were removed. Along with high electricity needs, the expected water demand is over five million gallons of water per day. While xAI has stated they plan to work with MLGW on a wastewater treatment facility and the installation of 50 megawatts of large battery storage facilities, there are currently no concrete plans in place aside from a one-page factsheet shared by MLGW. == Community response == The plan to build Colossus in Memphis was unknown to residents, City Council members, and environmental agencies. Many did not find out about the project until the day before, or the day of, as they watched the announcement on the local news. Keshaun Pearson, president of Memphis Community Against Pollution, stated that there is a historical lack of transparency and communication surrounding environmental issues in Memphis. Some community members in Memphis have expressed concern about the potential for additional air and water pollution caused by the supercomputer. In a letter to the Shelby County Health Department, the Southern Environmental Law Center stated the emissions from the turbines make the facility "...likely the largest industrial emitter of NOx in Memphis..." This is due to data supplied by the manufacturer showing that "...xAI emits between 1,200 and 2,000 tons of smog-forming nitrogen oxides (NOx)..." At a public Shelby County Commissioner's hearing on April 9, 2025, residents living near the site of Colossus voiced complaints about air quality, noting that they have chronic respiratory issues related to living in a polluted section of Memphis. One woman said she smells "everything but the right thing and the right thing is the clean air." Other residents voiced frustration that Brent Mayo, the senior xAI official responsible for building out xAI's infrastructure, did not attend the meeting to discuss community concerns. Keshaun Pearson also stated that "We're getting more and more days a year where it is unhealthy for us to go outside." People living near the site of Colossus have said they were not offered the opportunity for a public review of the plans, nor were they provided with information on how their community could potentially benefit. The community is also concerned about the strain on the power grid. Memphis's peak demand is around 3 GW. In November 2024, TVA approved xAI's request for access to more than 100 megawatts of power to Colossus which is supplied by MLGW. In December 2022, MLGW imposed (then rescinded) rolling blackouts during several days of extreme cold, straining the power grid. In a letter to the TVA, the SELC "urged the agency to 'prioritize Memphis families' access to reliable power over the 'secondary purpose' of serving xAI". == Current progress == In early December 2024, Ted Townsend detailed how the power of Colossus doubled in its processing capability. When it first went online in September 2024, it was using "100,000 Nvidia H100 processing chips". This initial launch demonstrated Colossus to be the largest supercomputer globally. The maximum power consumption increased from 150 to 250 MW. As of June 2025, the supercomputer consists of 150,000 H100 GPUs, 50,000 H200 GPUs, and 30,000 GB200 GPUs. Another 110,000 GB200 GPUs are to be brought online at a second data center, also in the Memphis area. The expansion of this supercomputer has already been discussed and will be the second phase of the project. xAI also plans to increase Colossus to 1 million GPUs. Because the supercomputer currently utilizes gas turbines for power, alongside 168 Tesla Megapack battery storage units. xAI is also looking to add more