AI Content Improver Free Online

AI Content Improver Free Online — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

Umple

Umple is a language for both object-oriented programming and modelling with class diagrams and state diagrams. The name Umple is a portmanteau of "UML", "ample" and "Simple", indicating that it is designed to provide ample features to extend programming languages with UML capabilities. == History and philosophy == The design of Umple started in 2008 at the University of Ottawa. Umple was open-sourced and its development was moved to Google Code in early 2011 and to GitHub in 2015. Umple was developed, in part, to address certain problems observed in the modelling community. Most specifically, it was designed to bring modelling and programming into alignment, It was intended to help overcome inhibitions against modelling common in the programmer community. It was also intended to reduce some of the difficulties of model-driven development that arise from the need to use large, expensive or incomplete tools. One design objective is to enable programmers to model in a way they see as natural, by adding modelling constructs to programming languages. == Features and capabilities == Umple can be used to represent in a textual manner many UML modelling entities found in class diagrams and state diagrams. Umple can generate code for these in various programming languages. Currently Umple fully supports Java, C++ and PHP as target programming languages and has functional, but somewhat incomplete support for Ruby. Umple also incorporates various features not related to UML, such as the singleton pattern, keys, immutability, mixins and aspect-oriented code injection. The class diagram notations Umple supports includes classes, interfaces, attributes, associations, generalizations and operations. The code Umple generates for attributes include code in the constructor, 'get' methods and 'set' methods. The generated code differs considerably depending on whether the attribute has properties such as immutability, has a default value, or is part of a key. Umple generates many methods for manipulating, querying and navigating associations. It supports all combinations of UML multiplicity and enforces referential integrity. Umple supports the vast majority of UML state machine notation, including arbitrarily deep nested states, concurrent regions, actions on entry, exit and transition, plus long-lasting activities while in a state. A state machine is treated as an enumerated attribute where the value is controlled by events. Events encoded in the state machine can be methods written by the user, or else generated by the Umple compiler. Events are triggered by calling the method. An event can trigger transitions (subject to guards) in several different state machines. Since a program can be entirely written around one or more state machines, Umple enables automata-based programming. The bodies of methods are written in one of the target programming languages. The same is true for other imperative code such as state machine actions and guards, and code to be injected in an aspect-oriented manner. Such code can be injected before many of the methods in the code Umple generates, for example before or after setting or getting attributes and associations. The Umple notation for UML constructs can be embedded in any of its supported target programming languages. When this is done, Umple can be seen as a pre-processor: The Umple compiler expands the UML constructs into code of the target language. Code in a target language can be passed to the Umple compiler directly; if no Umple-specific notation is found, then the target-language code is emitted unchanged by the Umple compiler. Umple, combined with one of its target languages for imperative code, can be seen and used as a complete programming language. Umple plus Java can therefore be seen as an extension of Java. Alternatively, if imperative code and Umple-specific concepts are left out, Umple can be seen as a way of expressing a large subset of UML in a purely textual manner. Code in one of the supported programming languages can be added in the same manner as UML envisions adding action language code. == License == Umple is licensed under an MIT-style license. == Examples == Here is the classic Hello world program written in Umple (extending Java): This example looks just like Java, because Umple extends other programming languages. With the program saved in a file named HelloWorld.ump, it can be compiled from the command line: $ java -jar umple.jar HelloWorld.ump To run it: $ java HelloWorld The following is a fully executable example showing embedded Java methods and declaration of an association. The following example describes a state machine called status, with states Open, Closing, Closed, Opening and HalfOpen, and with various events that cause transitions from one state to another. class GarageDoor { status { Open { buttonOrObstacle -> Closing; } Closing { buttonOrObstacle -> Opening; reachBottom -> Closed; } Closed { buttonOrObstacle -> Opening; } Opening { buttonOrObstacle -> HalfOpen; reachTop -> Open; } HalfOpen { buttonOrObstacle -> Opening; } } } == Umple use in practice == The first version of the Umple compiler was written in Java, Antlr and Jet (Java Emitter Templates), but in a bootstrapping process, the Java code was converted to Umple following a technique called Umplification. The Antlr and Jet were also later converted to native Umple. Umple is therefore now written entirely in itself, in other words it is self-hosted and serves as its own largest test case. Umple and UmpleOnline have been used in the classroom by several instructors to teach UML and modelling. In one study it was found to help speed up the process of teaching UML, and was also found to improve the grades of students. == Tools == Umple is available as a Jar file so it can be run from the command line, and as an Eclipse plugin. There is also an online tool for Umple called UmpleOnline , which allows a developer to create an Umple system by drawing a UML class diagram, editing Umple code or both. Umple models created with UmpleOnline are stored in the cloud. Currently UmpleOnline only supports Umple programs consisting of a single input file. In addition to code, Umple's tools can generate a variety of other types of output, including user interfaces based on the Umple model.
Read more →
Computational theory of mind

In philosophy of mind, the computational theory of mind (CTM), also known as computationalism, is a family of views that hold that the human mind is an information processing system and that cognition and consciousness together are a form of computation. It is closely related to functionalism, a broader theory that defines mental states by what they do rather than what they are made of. == History == Warren McCulloch and Walter Pitts (1943) were the first to suggest that neural activity is computational. They argued that neural computations explain cognition. A version of the theory was put forward by Peter Putnam and Robert W. Fuller in 1964. The theory was proposed in its modern form by Hilary Putnam in 1960 and 1961, aided by his then PhD student, philosopher and cognitive scientist Jerry Fodor, who continued the research as a post-doc in the 1960s, 1970s, and 1980s. It was later criticized by Putnam himself, John Searle, and others. == Classical computational theory of mind == The CTM holds that the human mind is a computational system that is realized (i.e., physically implemented) by neural activity in the brain. The theory can be elaborated in many ways and varies largely based on how the term computation is understood. In classical computational theory of mind (CCTM), computation is modeled in terms of Turing machines which manipulate symbols according to a rule, in combination with the internal state of the machine. A Turing machine is an abstract machine with unlimited time and storage. CCTM does not pretend that the mind looks like a Turing machine, but instead uses Turing machines as a formalism. Alan Turing argued that any symbolic algorithm executed by a human brain can in theory be replicated on a Turing machine. The critical aspect of such a computational model is that it allows to abstract away from particular physical details of the machine that is implementing the computation. For example, the appropriate computation could be implemented either by silicon chips or biological neural networks, so long as there is a series of outputs based on manipulations of inputs and internal states, performed according to a rule. Computational theories of mind are often said to require mental representation because 'input' into a computation comes in the form of symbols or representations of other objects. A computer cannot compute an actual object but must interpret and represent the object in some form and then compute the representation. Unlike CTM, the representational theory of mind shifts the focus to the symbols being manipulated. This approach better accounts for systematicity and productivity. In Fodor's view, the mind is a computational system that processes the language of thought. == Variants == Connectionist computationalism models the mind as a neural network. Steven Pinker and Alan Prince distinguish two types of connectionists: eliminative and implementationist. Eliminative connectionists generally reject classical CTMs and the idea of a structured, symbolic mind, whereas implementationists view neural networks and Turing machines as two potentially complementary levels of analysis. It is indeed possible in theory to implement a neural network in a Turing machine, or a Turing machine in a neural network. Building from the tradition of McCulloch and Pitts, the computational theory of cognition (CTC) states that neural computations explain cognition. The computational theory of mind asserts that not only cognition, but also phenomenal consciousness or qualia, are computational. That is to say, CTM entails CTC. While phenomenal consciousness could fulfill some other functional role, computational theory of cognition leaves open the possibility that some aspects of the mind could be non-computational. CTC, therefore, provides an important explanatory framework for understanding neural networks, while avoiding counter-arguments that center around phenomenal consciousness. == "Computer metaphor" == Computational theory of mind is not the same as the computer metaphor, comparing the mind to a modern-day digital computer. While the computer metaphor draws an analogy between the mind as software and the brain as hardware, CTM is the claim that the mind is literally a computational system. "Computational system" is not intended to mean a modern-day electronic computer. == Pancomputationalism == CTM raises a question that remains a subject of debate: what does it take for a physical system (such as a mind, or an artificial computer) to perform computations? A very straightforward account is based on a simple mapping between abstract mathematical computations and physical systems: a system performs computation C if and only if there is a mapping between a sequence of states individuated by C and a sequence of states individuated by a physical description of the system. Putnam (1988) and Searle (1992) argue that this simple mapping account (SMA) trivializes the empirical import of computational descriptions. As Putnam put it, "everything is a Probabilistic Automaton under some Description". Even rocks, walls, and buckets of water—contrary to appearances—are computing systems. Gualtiero Piccinini identifies different versions of pancomputationalism. Searle wrote:the wall behind my back is right now implementing the WordStar program, because there is some pattern of molecule movements that is isomorphic with the formal structure of WordStar. But if the wall is implementing WordStar, if it is a big enough wall it is implementing any program, including any program implemented in the brain.In response to the trivialization criticism, and to restrict SMA, philosophers of mind have offered different accounts of computational systems. These typically include causal account, semantic account, syntactic account, and mechanistic account. Instead of a semantic restriction, the syntactic account imposes a syntactic restriction. The mechanistic account was first introduced by Gualtiero Piccinini in 2007. == Criticism == A range of arguments have been proposed against physicalist conceptions used in computational theories of mind. An early, though indirect, criticism of the computational theory of mind comes from philosopher John Searle. In his thought experiment known as the Chinese room, Searle attempts to refute the claims that artificially intelligent agents can be said to have intentionality and understanding and that these systems, because they can be said to be minds themselves, are sufficient for the study of the human mind. Searle asks us to imagine that there is a man in a room with no way of communicating with anyone or anything outside of the room except for a piece of paper with symbols written on it that is passed under the door. With the paper, the man is to use a series of provided rule books to return paper containing different symbols. Unknown to the man in the room, these symbols are of a Chinese language, and this process generates a conversation that a Chinese speaker outside of the room can actually understand. Searle contends that the man in the room does not understand the Chinese conversation. This was originally written as a repudiation of the idea that computers work like minds. Objections like Searle's might be called insufficiency objections. They claim that computational theories of mind fail because computation is insufficient to account for some capacity of the mind. Arguments from qualia, such as Frank Jackson's knowledge argument, can be understood as objections to computational theories of mind in this way—though they take aim at physicalist conceptions of the mind in general, and not computational theories specifically. Objections have also been put forth that are directly tailored for computational theories of mind. Jerry Fodor himself argues that the mind is still a very long way from having been explained by the computational theory of mind. The main reason for this shortcoming is that most cognition is abductive and global, hence sensitive to all possibly relevant background beliefs to (dis)confirm a belief. This creates, among other problems, the frame problem for the computational theory, because the relevance of a belief is not one of its local, syntactic properties but context-dependent. Putnam himself (see in particular Representation and Reality and the first part of Renewing Philosophy) became a prominent critic of computationalism for a variety of reasons, including ones related to Searle's Chinese room arguments, questions of world-word reference relations, and thoughts about the mind-body problem. Regarding functionalism in particular, Putnam has claimed along lines similar to, but more general than Searle's arguments, that the question of whether the human mind can implement computational states is not relevant to the question of the nature of mind, because "every ordinary open system realizes every abstract finite automaton." Computationalists have responded by aiming to develop criteri
Read more →
Sarah Guo

Sarah Guo is an American tech investor. She is the founder of the venture capital firm Conviction and formerly a general partner at Greylock Partners. == Early life and education == Guo grew up in Wisconsin. Her parents worked for Bell Labs. After attending Phillips Academy, she graduated from the University of Pennsylvania and its Wharton School. She received a Bachelor of Arts, a Bachelor of Science, a Master of Business Administration (M.B.A.), and a Master of Arts from the University of Pennsylvania. == Career == As a teenager, Guo worked at Casa Systems, a cloud networking company founded by her parents that launched in 2003 and went public in 2017. She then worked at Goldman Sachs. In 2013, Guo joined Greylock Partners. While still in her twenties, she became the firm's youngest General Partner. Guo left Greylock in July 2022, and in October of that year, launched a new early-stage venture capital firm focused on AI with $101 million. In 2025, Conviction raised a second fund in late 2024 with Mike Vernal. Conviction's investments include early investments in Baseten, Cognition AI, OpenEvidence, Harvey, HeyGen, Mistral AI, Sierra Platform, Sunday Robotics, and Thinking Machines Lab. Guo appears in media outlets, as an expert in AI, infrastructure, business software, cybersecurity, technology policy and software engineering. Guo is on the Midas List and the Midas Seed List of top investors. She co-hosts the podcast No Priors with tech founder and super angel Elad Gil. == Personal life == Guo is married to Pat Grady of Sequoia Capital.
Read more →
REEM

REEM is a prototype humanoid robot built by PAL Robotics in Spain. It is a 1.70 m high humanoid robot with 22 degrees of freedom, with a mobile base with wheels, allowing it to move at 4 km/hour. The upper part of the robot consists of a torso with a touch screen, two motorized arms, which give it a high degree of expression, and a head, which is also motorized. REEM-A and REEM-B are the first and second prototypes of humanoid robots created by PAL Robotics. REEM-B can recognize, grasp and lift objects and walk by itself, avoiding obstacles through simultaneous localization and mapping. The robot accepts voice commands and can recognize faces. == Specifications ==
Read more →
Emergent algorithm

An emergent algorithm is an algorithm that exhibits emergent behavior. In essence an emergent algorithm implements a set of simple building block behaviors that when combined exhibit more complex behaviors. One example of this is the implementation of fuzzy motion controllers used to adapt robot movement in response to environmental obstacles. An emergent algorithm has the following characteristics: it achieves predictable global effects it does not require global visibility it does not assume any kind of centralized control it is self-stabilizing Other examples of emergent algorithms and models include cellular automata, artificial neural networks and swarm intelligence systems (ant colony optimization, bees algorithm, etc.).
Read more →
ELVIS Act

The ELVIS Act or Ensuring Likeness Voice and Image Security Act, signed into law by Tennessee Governor Bill Lee on March 21, 2024, marked a significant milestone in the area of regulation of artificial intelligence and public sector policies for artists in the era of artificial intelligence (AI) and AI alignment. It was noted as the first enacted legislation in the United States specifically designed to protect musicians from the unauthorized use of their voices through artificial intelligence technologies and against audio deepfakes and voice cloning. This legislation distinguishes itself by adding penalties for copying a performer's voice. == Origin and advocacy == The inception of the ELVIS Act has been attributed to Gebre Waddell, founder of Sound Credit, who initially conceptualized a framework in 2023 that later evolved into the legislation. Representative Justin J. Pearson acknowledged Waddell's pivotal role during the March 4 House Floor Session on the bill. Leading Tennessee musicians supported the ELVIS Act. Tennessee Governor Bill Lee endorsed it as a Governor's Bill, and it was introduced in the Tennessee Legislature as House Bill 2091 by William Lamberth (R-44) and Senate Bill 2096 by Jack Johnson (R-27). The ELVIS Act is an amendment to a 1984 law that was the result of the Elvis Presley estate litigation for controlling how his likeness could be used after death. == Lobbying from the recording industry == The legislative journey of the ELVIS Act included a broad coalition of music industry stakeholders, including: These organizations, led by the Recording Academy and the RIAA, played roles in drafting the legislation, advocating for passage, and rallying support among the industry and legislators. The act gained momentum through discussions that bridged industry concerns with legislative action. This collaborative process led to a proposal that specifically targets the use of AI to create unauthorized reproductions of artists' voices and images. == Opposition == The ELVIS Act saw industry opposition from the Motion Picture Association, including testimony in the House Banking & Consumer Affairs Subcommittee, including remarks that the law risks "interference with our members’ ability to portray real people and events." TechNet, representing companies such as OpenAI, Google and Amazon, expressed their opposition in the hearing to the bill as drafted, asserting that the language was too broadly written and could have unintended consequences. Other concerns included its potential application to cover bands, but lawmakers assured people that this was not the intention. The bill passed the Tennessee House and Senate with a unanimous, bi-partisan vote including 93 ayes and 0 Noes in the House, and 30 ayes and 0 noes in the Senate. == Passage == By explicitly addressing AI impersonation, the ELVIS Act originated a legal approach to safeguarding personal rights, in the context of digital and technological advancements. It extends protections to an artist's voice and likeness, areas vulnerable to exploitation with the proliferation of AI technologies that occurred in 2023. The legislation received widespread support from the music industry, signaling a significant step forward in the ongoing effort to balance innovation with the protection of individual rights and creative integrity. It was reported as underscoring Tennessee's commitment to its musical heritage and showed the state as a leader in adapting copyright and privacy protections to the modern technological landscape. Artists including Chris Janson and Luke Bryan appeared at the signing ceremony hosted at Robert's Western World to support the new law and commemorate its passing. == Legal precedent == The ELVIS Act was reported as representing a development in the discourse surrounding AI, intellectual property, and personal rights. It was hoped by proponents to set a precedent for future legislative efforts both within and beyond Tennessee, offering a model for how states and potentially the federal government could address similar challenges. As AI technology continues to evolve, the act represents a foundational framework for protecting the authenticity and rights of artists, ensuring contributions remain protected. The act prohibits usage of AI to clone the voice of an artist without consent and can be criminally enforced as a Class A misdemeanor. This legislation's success was hoped by its supporters to inspire similar actions in other states, contributing to a unified approach to copyright and privacy in the digital age. Such a national response would reinforce the importance of safeguarding artists' rights against unauthorized use of their voices and likenesses.
Read more →
Utah Artificial Intelligence Policy Act

The Utah Artificial Intelligence Policy Act (SB-149) was signed into law in Utah in 2024 and amended in 2025. The first state law in the United States specifically regulating generative AI, it went into effect on May 1, 2024. The law requires companies to disclose if their customers interact with AI instead of a human. It also established an Office of Artificial Intelligence Policy. Amendments to the Act went into effect on May 7, 2025. While the 2024 Act requires companies to disclose generative AI use when asked by customers, the amendments introduced stricter requirements for higher-risk interactions. SB 226 mandates disclosure of AI use in high-risk interactions involving health, financial, and biometric data, or when providing consumers with advice on financial, legal, or healthcare matters.
Read more →
Ilya Sutskever

Ilya Sutskever (Hebrew: איליה סוצקבר; born 1986) is a computer scientist who specializes in machine learning. He has made several major contributions to the field of deep learning, including sequence-to-sequence learning, reasoning models, GPT models, and contributions to CLIP, DALL-E, and AlphaGo. With Alex Krizhevsky and Geoffrey Hinton, he co-created AlexNet, a convolutional neural network. One of the most highly cited computer scientists in history, he has won the NeurIPS Test of Time Award for his lasting impact on AI research three times in a row (2022–2024) and received the National Academy of Sciences Award for the Industrial Application of Science in 2026. Sutskever co-founded and was chief scientist at OpenAI, where he oversaw the research breakthroughs that led to large language models and to the launch of ChatGPT. He also led the research that led to reasoning models such as o1. In 2023, he was one of the members of OpenAI's board that ousted Sam Altman as its CEO; Altman was reinstated a week later, and Sutskever stepped down from the board. In June 2024, Sutskever co-founded the company Safe Superintelligence Inc., alongside Daniel Gross and Daniel Levy. Within a year, the company was valued at more than $30 billion. == Early life and education == Sutskever was born in 1986 into a Jewish family in Nizhny Novgorod, Russia (then Gorky, Russian SFSR, Soviet Union). At the age of 5, he immigrated to Israel with his family and grew up in Jerusalem. Sutskever proved to be a good student in school, and in eighth grade started taking classes at the Open University of Israel. At 16, he moved with his family to Canada, where he attended high school for a month before being admitted to the University of Toronto in Ontario as a third-year undergraduate student. At the University of Toronto, Sutskever received a bachelor's degree in mathematics in 2005, a master's degree in computer science in 2007, and a PhD in computer science in 2013. His doctoral advisor was Geoffrey Hinton. In 2012, Sutskever built AlexNet in collaboration with Geoffrey Hinton and Alex Krizhevsky. == Career and research == In 2012, Sutskever spent about two months as a postdoc with Andrew Ng at Stanford University. He then returned to the University of Toronto and joined Hinton's new research company DNNResearch, a spinoff of Hinton's research group. In 2013, Google acquired DNNResearch and hired Sutskever as a research scientist at Google Brain. At Google Brain, Sutskever worked with Oriol Vinyals and Quoc Viet Le to create the sequence-to-sequence learning algorithm, and worked on TensorFlow. He is also one of the AlphaGo paper's many co-authors. At the end of 2015, Sutskever left Google to become cofounder and chief scientist of the newly founded organization OpenAI. In 2022, Sutskever tweeted, "it may be that today's large neural networks are slightly conscious", which triggered debates about AI consciousness. He is considered to have played a key role in the development of ChatGPT, and later in leading the research that led to reasoning models. He is credited with establishing OpenAI’s scaling ethos. In 2023, he announced that he would co-lead OpenAI's new "Superalignment" project, which was trying to solve the alignment of superintelligences within four years. He wrote that even if superintelligence seems far off, it could happen this decade. Sutskever was formerly one of the six board members of the nonprofit entity that controlled OpenAI. In November 2023, the board fired Sam Altman, saying that "he was not consistently candid in his communications with the board". He authored a 52-page memo that relied heavily on information from Mira Murati, accusing Altman of lying, manipulating executives, and fostering internal division. Sutskever submitted the memo to the board after months of tension and dissatisfaction with Altman's leadership style, and ultimately joined the board in voting for Altman's termination. In an all-hands company meeting shortly after the board meeting, Sutskever said that firing Altman was "the board doing its duty", but the next week, he expressed regret at having participated in Altman's ouster. Altman's firing and OpenAI's co-founder Greg Brockman's resignation led three senior researchers to resign from OpenAI. After that, Sutskever stepped down from the OpenAI board and was absent from OpenAI's office. Some sources suggested he was leading the team remotely, while others said he no longer had access to the team's work. In May 2024, Sutskever announced his departure from OpenAI to focus on a new project that was "very personally meaningful" to him. His decision followed a turbulent period at OpenAI marked by leadership crises and internal debates about the direction of AI development and alignment protocols. Jan Leike, the other leader of the superalignment project, announced his departure hours later, citing an erosion of safety and trust in OpenAI's leadership. In June 2024, Sutskever announced Safe Superintelligence Inc., a new company he founded with Daniel Gross and Daniel Levy with offices in Palo Alto and Tel Aviv. In contrast to OpenAI, which releases revenue-generating products, Sutskever said the new company's "first product will be the safe superintelligence, and it will not do anything else up until then". In September 2024, the company announced that it had raised $1 billion from venture capital firms including Andreessen Horowitz, Sequoia Capital, DST Global, and SV Angel. In March 2025, Safe Superintelligence Inc. raised $2 billion more and reportedly reached a $32 billion valuation, notably due to Sutskever's reputation. In June 2025, SSI rejected an offer from Meta Platforms to buy the company. Sutskever became CEO of SSI shortly thereafter, after co-founder and CEO Gross left for Meta. In an October 2024 interview after winning the Nobel Prize in Physics, Geoffrey Hinton expressed support for Sutskever's decision to fire Altman, emphasizing concerns about AI safety. During the Musk v. Altman trial in 2026, Sutskever confirmed he had a $7 billion stake in OpenAI. === Awards and honors === In 2015, Sutskever was named in MIT Technology Review's 35 Innovators Under 35. In 2018, he was the keynote speaker at Nvidia Ntech 2018 and AI Frontiers Conference 2018. In 2022, he was elected a Fellow of the Royal Society (FRS). In 2023 and 2024, included in Time's list of the 100 most influential people in AI In 2022, 2023, and 2024, he won Neural Information Processing Systems’ Test of Time award, which recognizes papers that significantly shaped the AI field over at least ten years. In 2025, he received an honorary doctorate from his alma mater, the University of Toronto In 2026, he received the National Academy of Sciences Award for the Industrial Application of Science, presented for the first time in artificial intelligence.
Read more →
Statistical learning theory

Statistical learning theory is a framework for machine learning drawing from the fields of statistics and functional analysis. Statistical learning theory deals with the statistical inference problem of finding a predictive function based on data. Statistical learning theory has led to successful applications in fields such as computer vision, speech recognition, and bioinformatics. == Introduction == The goals of learning are understanding and prediction. Learning falls into many categories, including supervised learning, unsupervised learning, online learning, and reinforcement learning. From the perspective of statistical learning theory, supervised learning is best understood. Supervised learning involves learning from a training set of data. Every point in the training is an input–output pair, where the input maps to an output. The learning problem consists of inferring the function that maps between the input and the output, such that the learned function can be used to predict the output from future input. Depending on the type of output, supervised learning problems are either problems of regression or problems of classification. If the output takes a continuous range of values, it is a regression problem. Using Ohm's law as an example, a regression could be performed with voltage as input and current as an output. The regression would find the functional relationship between voltage and current to be R {\displaystyle R} , such that V = I R {\displaystyle V=IR} Classification problems are those for which the output will be an element from a discrete set of labels. Classification is very common for machine learning applications. In facial recognition, for instance, a picture of a person's face would be the input, and the output label would be that person's name. The input would be represented by a large multidimensional vector whose elements represent pixels in the picture. After learning a function based on the training set data, that function is validated on a test set of data, data that did not appear in the training set. == Formal description == Take X {\displaystyle X} to be the vector space of all possible inputs, and Y {\displaystyle Y} to be the vector space of all possible outputs. Statistical learning theory takes the perspective that there is some unknown probability distribution over the product space Z = X × Y {\displaystyle Z=X\times Y} , i.e. there exists some unknown p ( z ) = p ( x , y ) {\displaystyle p(z)=p(\mathbf {x} ,y)} . The training set is made up of n {\displaystyle n} samples from this probability distribution, and is notated S = { ( x 1 , y 1 ) , … , ( x n , y n ) } = { z 1 , … , z n } {\displaystyle S=\{(\mathbf {x} _{1},y_{1}),\dots ,(\mathbf {x} _{n},y_{n})\}=\{\mathbf {z} _{1},\dots ,\mathbf {z} _{n}\}} Every x i {\displaystyle \mathbf {x} _{i}} is an input vector from the training data, and y i {\displaystyle y_{i}} is the output that corresponds to it. In this formalism, the inference problem consists of finding a function f : X → Y {\displaystyle f:X\to Y} such that f ( x ) ∼ y {\displaystyle f(\mathbf {x} )\sim y} . Let H {\displaystyle {\mathcal {H}}} be a space of functions f : X → Y {\displaystyle f:X\to Y} called the hypothesis space. The hypothesis space is the space of functions the algorithm will search through. Let V ( f ( x ) , y ) {\displaystyle V(f(\mathbf {x} ),y)} be the loss function, a metric for the difference between the predicted value f ( x ) {\displaystyle f(\mathbf {x} )} and the actual value y {\displaystyle y} . The expected risk is defined to be I [ f ] = ∫ X × Y V ( f ( x ) , y ) p ( x , y ) d x d y {\displaystyle I[f]=\int _{X\times Y}V(f(\mathbf {x} ),y)\,p(\mathbf {x} ,y)\,d\mathbf {x} \,dy} The target function, the best possible function f {\displaystyle f} that can be chosen, is given by the f {\displaystyle f} that satisfies f = argmin h ∈ H ⁡ I [ h ] {\displaystyle f=\mathop {\operatorname {argmin} } _{h\in {\mathcal {H}}}I[h]} Because the probability distribution p ( x , y ) {\displaystyle p(\mathbf {x} ,y)} is unknown, a proxy measure for the expected risk must be used. This measure is based on the training set, a sample from this unknown probability distribution. It is called the empirical risk I S [ f ] = 1 n ∑ i = 1 n V ( f ( x i ) , y i ) {\displaystyle I_{S}[f]={\frac {1}{n}}\sum _{i=1}^{n}V(f(\mathbf {x} _{i}),y_{i})} A learning algorithm that chooses the function f S {\displaystyle f_{S}} that minimizes the empirical risk is called empirical risk minimization. == Loss functions == The choice of loss function is a determining factor on the function f S {\displaystyle f_{S}} that will be chosen by the learning algorithm. The loss function also affects the convergence rate for an algorithm. It is important for the loss function to be convex. Different loss functions are used depending on whether the problem is one of regression or one of classification. === Regression === The most common loss function for regression is the square loss function (also known as the L2-norm). This familiar loss function is used in Ordinary Least Squares regression. The form is: V ( f ( x ) , y ) = ( y − f ( x ) ) 2 {\displaystyle V(f(\mathbf {x} ),y)=(y-f(\mathbf {x} ))^{2}} The absolute value loss (also known as the L1-norm) is also sometimes used: V ( f ( x ) , y ) = | y − f ( x ) | {\displaystyle V(f(\mathbf {x} ),y)=|y-f(\mathbf {x} )|} === Classification === In some sense the 0-1 indicator function is the most natural loss function for classification. It takes the value 0 if the predicted output is the same as the actual output, and it takes the value 1 if the predicted output is different from the actual output. For binary classification with Y = { − 1 , 1 } {\displaystyle Y=\{-1,1\}} , this is: V ( f ( x ) , y ) = θ ( − y f ( x ) ) {\displaystyle V(f(\mathbf {x} ),y)=\theta (-yf(\mathbf {x} ))} where θ {\displaystyle \theta } is the Heaviside step function. == Regularization == In machine learning problems, a major problem that arises is that of overfitting. Because learning is a prediction problem, the goal is not to find a function that most closely fits the (previously observed) data, but to find one that will most accurately predict output from future input. Empirical risk minimization runs this risk of overfitting: finding a function that matches the data exactly but does not predict future output well. Overfitting is symptomatic of unstable solutions; a small perturbation in the training set data would cause a large variation in the learned function. It can be shown that if the stability for the solution can be guaranteed, generalization and consistency are guaranteed as well. Regularization can solve the overfitting problem and give the problem stability. Regularization can be accomplished by restricting the hypothesis space H {\displaystyle {\mathcal {H}}} . A common example would be restricting H {\displaystyle {\mathcal {H}}} to linear functions: this can be seen as a reduction to the standard problem of linear regression. H {\displaystyle {\mathcal {H}}} could also be restricted to polynomial of degree p {\displaystyle p} , exponentials, or bounded functions on L1. Restriction of the hypothesis space avoids overfitting because the form of the potential functions are limited, and so does not allow for the choice of a function that gives empirical risk arbitrarily close to zero. One example of regularization is Tikhonov regularization. This consists of minimizing 1 n ∑ i = 1 n V ( f ( x i ) , y i ) + γ ‖ f ‖ H 2 {\displaystyle {\frac {1}{n}}\sum _{i=1}^{n}V(f(\mathbf {x} _{i}),y_{i})+\gamma \left\|f\right\|_{\mathcal {H}}^{2}} where γ {\displaystyle \gamma } is a fixed and positive parameter, the regularization parameter. Tikhonov regularization ensures existence, uniqueness, and stability of the solution. == Bounding empirical risk == Consider a binary classifier f : X → { 0 , 1 } {\displaystyle f:{\mathcal {X}}\to \{0,1\}} . We can apply Hoeffding's inequality to bound the probability that the empirical risk deviates from the true risk to be a Sub-Gaussian distribution. P ( | R ^ ( f ) − R ( f ) | ≥ ϵ ) ≤ 2 e − 2 n ϵ 2 {\displaystyle \mathbb {P} (|{\hat {R}}(f)-R(f)|\geq \epsilon )\leq 2e^{-2n\epsilon ^{2}}} But generally, when we do empirical risk minimization, we are not given a classifier; we must choose it. Therefore, a more useful result is to bound the probability of the supremum of the difference over the whole class. P ( sup f ∈ F | R ^ ( f ) − R ( f ) | ≥ ϵ ) ≤ 2 S ( F , n ) e − n ϵ 2 / 8 ≈ n d e − n ϵ 2 / 8 {\displaystyle \mathbb {P} {\bigg (}\sup _{f\in {\mathcal {F}}}|{\hat {R}}(f)-R(f)|\geq \epsilon {\bigg )}\leq 2S({\mathcal {F}},n)e^{-n\epsilon ^{2}/8}\approx n^{d}e^{-n\epsilon ^{2}/8}} where S ( F , n ) {\displaystyle S({\mathcal {F}},n)} is the shattering number and n {\displaystyle n} is the number of samples in your dataset. The exponential term comes from Hoeffding but there is an extra cost of taking the supremum over the whole cla
Read more →
DAYDREAMER

DAYDREAMER is a goal-based agent and cognitive architecture developed at the University of California, Los Angeles by Erik T. Mueller and Michael G. Dyer beginning in 1983. The system models the human stream of thought and how it is triggered and directed by emotions, simulating human daydreaming. Taking situational descriptions as input, DAYDREAMER produces English-language daydreams as output and encodes new daydreams, plans, and planning strategies for later reuse. The program comprises five components: a scenario generator based on relaxed planning, a dynamic episodic memory, a collection of personal goals and control goals, an emotion component, and domain knowledge of interpersonal relations and everyday occurrences. The source code was released under a free software license in 2015. == History == Erik Mueller began DAYDREAMER in 1983 while he was a doctoral student in the Artificial Intelligence Laboratory of the Computer Science Department at the University of California, Los Angeles, studying under Michael G. Dyer. Initial development of the project was supported by a grant from the W. M. Keck Foundation with matching funds from the UCLA School of Engineering and Applied Sciences. Additionally, Mueller was supported by an Atlantic Richfield Doctoral Fellowship and Dyer by an IBM Faculty Development Award. The first published descriptions of the program appeared in 1985 at the Ninth International Joint Conference on Artificial Intelligence in Los Angeles and at the Seventh Annual Conference of the Cognitive Science Society in Irvine. Work on the program continued, and a book, Daydreaming in Humans and Machines, was published by Ablex Publishing in 1990. The program was implemented on top of GATE, a knowledge-representation and inference substrate developed by Mueller and Uri Zernik at UCLA, and was originally written in T, a dialect of Scheme. In 2015, Mueller released the DAYDREAMER source code, version 3.5, a Common Lisp rewrite of the original T implementation, on GitHub under the GNU General Public License version 2. The release comprised approximately 12,000 lines of Common Lisp code, along with the GATE knowledge-representation substrate on which DAYDREAMER had originally been built. == Architecture == The program operates in two modes. In daydreaming mode it daydreams continuously until interrupted, while performance mode allows it to demonstrate behavior it has learned through daydreaming. === Emotion and control goals === Emotions and daydreaming form a feedback loop for DAYDREAMER. Emotions activate goals that produce daydreams, and the resulting daydreams modify existing emotions and trigger new ones, which prompt subsequent daydreaming. Recall of a goal success produces a positive emotion whereas recall of a goal failure produces a negative emotion. Emotions activate a set of goals, called control goals, which direct the course of a daydream. The program has four control goals. "Rationalization" generates reasons why an unsatisfactory outcome is in fact acceptable, in order to reduce a negative emotion and maintain self-esteem. "Revenge" is activated by anger when a failure is caused by another and reduces negative emotion through imagined retaliation. "Failure/success reversal" imagines alternative scenarios in which a failure was prevented or a success did not occur as a means of learning planning strategies for future situations. "Preparation" generates hypothetical future scenarios in order to rehearse plans and actions for events that have not yet occurred. === Scenario generator and relaxed planning === The scenario generator produces the sequence of events that make up a daydream. It operates under multiple, often conflicting personal goals rather than pursuing a single goal, applies relaxation rules that permit the generation of non-realistic scenarios, and it draws on episodic memory of past experiences both as subject matter and as a source of planning knowledge. The personal goals that guide the scenario generator include health, food, sex, friendship, love, possessions, self-esteem, social esteem, enjoyment, and achievement. These goals are organized into a goal tree that specifies their relative importance at any given time. Relaxation rules allow the program to set aside its ordinary constraints when generating a scenario. The four constraints that may be relaxed are the behavior of others, the daydreamer's own attributes, physical constraints, and social constraints. The degree of relaxation varies with the active control goal. For example a failure-reversal goal aimed at alternatives uses a low level of relaxation, whereas a revenge goal aimed at a retaliation uses a high level. === Episodic memory and analogy === DAYDREAMER's episodic memory stores its personal and vicarious experiences along with the daydreams it generates. The memory is described as dynamic because it is continually modified during daydreaming such that previously daydreamed episodes become available alongside real ones. As it daydreams, the program indexes daydreams, future plans or actions, and planning strategies into memory. Episodes are organized and retrieved using surface-level similarities, emotions, abstract themes, and Plot Units which are abstract configurations of positive and negative outcomes developed by Wendy Lehnert. A recalled episode is adapted to the current situation through analogy, which requires less effort than generating an equivalent scenario from scratch. == Sample output == In the sample experience from the source code, called LOVERS1, DAYDREAMER begins from an initial situation in which it has a job, is not romantically involved, and is at home. Starting in daydreaming mode, it activates a top-level goal to be in a romantic relationship because it is not currently in one, and a positive motivating emotion of interest becomes associated with that goal. The program then activates a goal to be entertained and pursues seeing a film as a way to achieve it. Facts asserted into memory are converted to English and produced as output, such as "I want to be going out with someone" and "I have to go see a movie". == Reception and influence == DAYDREAMER has been cited in research on computational models of creativity, emotion, and narrative. Linda Wills and Janet Kolodner cite the program as an example of work on opportunism in their study of serendipitous recognition in design. Joseph Bates, A. Bryan Loyall, and W. Scott Reilly of the Carnegie Mellon Oz Project cite DAYDREAMER among prior work in their description of an architecture combining action, emotion, and social behavior. Rafael Pérez y Pérez, Ricardo Sosa, and Christian Lemaitre cite Mueller's DAYDREAMER as one of the few computer models at the time to model daydreaming during the creative process. Jichen Zhu and D. Fox Harrell likewise cite the program in their work on imagining and agency in generative interactive narrative.
Read more →
Computer Power and Human Reason

Computer Power and Human Reason: From Judgment to Calculation is a 1976 nonfiction book by German-American computer scientist Joseph Weizenbaum in which he contends that while artificial intelligence may be possible, we should never allow computers to make important decisions, as they will always lack human qualities such as compassion and wisdom. == Background == Before writing Computer Power and Human Reason, Weizenbaum had garnered significant attention for creating the ELIZA program, an early milestone in conversational computing. His firsthand observation of people attributing human-like qualities to a simple program prompted him to reflect more deeply on society's readiness to entrust moral and ethical considerations to machines. == Reception and legacy == Computer Power and Human Reason sparked scholarly debate on the acceptable scope of AI applications, particularly in fields where human welfare and ethical considerations are paramount. Early academic reviews highlighted that Weizenbaum's stance pushed readers to recognize that even as computers grow more capable, they lack the intrinsic moral compass and empathy required for certain kinds of judgment. The book caused disagreement with, and separation from, other members of the artificial intelligence research community, a status the author later said he'd come to take pride in.
Read more →
Eimear Kenny

Eimear E. Kenny is a researcher in population genetics and translation genomics, and is the Founding Director of the Institute for Genomic Health, and Endowed Chair and Professor of Genomic Health at the Icahn School of Medicine at Mount Sinai. She is known for novel approaches in computational genomics, advancing the study of human genetic variation and its connection to disease risk and diagnosis. Her research has laid the foundation for integrating artificial intelligence (AI) and genomics into precision medicine and routine clinical care. By combining genomics, computer science, and medicine, her work leverages genomic sequencing technologies and machine learning algorithms to uncover insights that improve patient care, accelerate genomic data analysis, and enable the future of AI-driven healthcare. She has led multiple genomics-based clinical trials, applying computational biology and AI in clinical settings to advance genomic medicine and precision healthcare. == Research == A recipient of the Early-Career Award from the American Society of Human Genetics (USA), Kenny, as of 2024, leads a team in genetics, computer science, and medicine, focusing on genetic ancestry, large-scale genomics, clinical trials, and genomic medicine at the Institute for Genomic Health. The lab works to advance understanding of genetic ancestry and its impact on health in order to inform better clinical medicine models. She is recognized for her work to leverage biobanks for translational genomics and her development of new genetic tests an strategies for health care management. In one study, she and her colleagues investigated genetic disorders that might be under-diagnosed due to insufficient data, and found a variant in a collagen gene associated with Steel syndrome. This syndrome caused short stature and bone and joint issues and was thought to be rare. However, the study revealed it is common in individuals with Puerto Rican ancestry. Three of Kenny's genomic medicine clinical trials assessed how to bring new technology, such as digital apps, or information, such as polygenic risk scores, into routine clinical care. In the 2010s, Kenny was instrumental in several large-scale sequencing studies, including the 1000 Genomes Project, the Exome Sequencing Project, the Genome Sequencing Project, and the Trans-Omics for Precision Medicine. In 2012, she led work that discovered the variant responsible for blond hair in Melanesia, work that was featured in the Smithsonian NHGRI Human Genome Exhibit in Washington, D.C. In 2017, her group was one of the first to demonstrate that polygenic risk scores derived in predominantly European populations have reduced accuracy when applied in populations now widely acknowledged as a major challenge in the field of genomic risk prediction. As of 2024, she is Principal Investigator in many NIH-funded international consortium focused on computational genomics and genomic medicine, including Electronic Medical Records and Genomics, Polygenic Risk Methods in Diverse Populations, and the Human Pangenome Reference Consortium. In 2023, Kenny played a key role in a groundbreaking advancement in genomics research by helping to map a diverse human pangenome—a major shift from reliance on a single reference genome. Unlike the earlier genetic map, based on one man of mixed European and African ancestry in Buffalo, this new pangenome project captures far greater human genetic diversity. As reported by The Washington Post, Kenny's work demonstrates how a more inclusive human genome can drive discoveries in rare genetic diseases, improve genomic medicine, and accelerate the future of precision healthcare. Kenny was co-developer and current license holder for Random Forest adMIXture (RFMix), a patented software for inferring continental and sub-continental ancestry at genomic loci. == Education and career == Kenny graduated from Trinity College Dublin with a BA in Biochemistry in 1999 and did a masters in Bioinformatics at Leeds University. She received her PhD in Computational Genomics at Rockefeller University, and did her post-doctoral work in the lab of Dr. Carlos D. Bustamante at Stanford University. === Academic appointments === As of 2024, at Mount Sinai, she serves as the Endowed Chair and Professor of Genomic Health, Professor at the Department of Medicine and Professor at the Department of Genetics and Genomic Sciences. Since 2018 she has served as the Founding Director of the Institute for Genomic Health, and since 2022, she also serves as the Founding Director of the Center for Translational Genomics. She is also the Director of Translational Research, Division for Genomic Medicine. Former appointments include Assistant Professor at the Department of Genetics and Genomic Sciences and Member at The Charles Bronfman Institute of Personalized Medicine, both at Mount Sinai. She was also Bioinformatics Programmer at the California Institute of Technology, and research assistant at the Massachusetts Institute of Technology. == Publications == As of 2024, Kenny is an advisor to Cell Genomics. Google Scholar reports 50,623 citations, an h-index of 66 and an i10-index of 130. The five most-cited articles she contributed to are: Auton, A; Brooks, LD; Durbin, RM; Garrison, EP; Kang, HM; Korbel, JO; Marchini, JL; McCarthy, S; McVean, GA; Abecasis, GR (2015). "A global reference for human genetic variation". Nature. 526 (7571): 68–74. Bibcode:2015Natur.526...68T. doi:10.1038/nature15393. PMC 4750478. PMID 26432245.. Cited by 14847 Abecasis, GR; Auton, A; Brooks, LD; DePristo, MA; Durbin, RM; Handsaker, RE; Kang, HM; Marth, GT; McVean, GA (2012). "An integrated map of genetic variation from 1,092 human genomes". Nature. 491 (7422): 56–65. Bibcode:2012Natur.491...56T. doi:10.1038/nature11632. PMC 3498066. PMID 23128226.. Cited by 8287 Jacob A. Tennessen et al. Evolution and Functional Impact of Rare Coding Variation from Deep Sequencing of Human Exomes.Science337,64–69(2012).DOI:10.1126/science.1219240 Cited by 1886 Taliun, D.; Harris, D.N.; Kessler, M.D.; et al. (2021). "Sequencing of 53,831 diverse genomes from the NHLBI TOPMed Program". Nature. 590 (7845): 290–299. Bibcode:2021Natur.590..290T. doi:10.1038/s41586-021-03205-y. PMC 7875770. PMID 33568819.. Cited by 1369 Vilhjálmsson, BJ; et al. (2015). "Modeling Linkage Disequilibrium Increases Accuracy of Polygenic Risk Scores". Am J Hum Genet. 97 (4): 576–92. doi:10.1016/j.ajhg.2015.09.001. PMC 4596916. PMID 26430803.. Cited by 1327
Read more →
Learning rate

In machine learning and statistics, the learning rate is a tuning parameter in an optimization algorithm that determines the step size at each iteration while moving toward a minimum of a loss function. Since it influences to what extent newly acquired information overrides old information, it metaphorically represents the speed at which a machine learning model "learns". In the adaptive control literature, the learning rate is commonly referred to as gain. In setting a learning rate, there is a trade-off between the rate of convergence and overshooting. While the descent direction is usually determined from the gradient of the loss function, the learning rate determines how big a step is taken in that direction. Too high a learning rate will make the learning jump over minima, but too low a learning rate will either take too long to converge or get stuck in an undesirable local minimum. In order to achieve faster convergence, prevent oscillations and getting stuck in undesirable local minima the learning rate is often varied during training either in accordance to a learning rate schedule or by using an adaptive learning rate. The learning rate and its adjustments may also differ per parameter, in which case it is a diagonal matrix that can be interpreted as an approximation to the inverse of the Hessian matrix in Newton's method. The learning rate is related to the step length determined by inexact line search in quasi-Newton methods and related optimization algorithms. == Learning rate schedule == Initial rate can be left as system default or can be selected using a range of techniques. A learning rate schedule changes the learning rate during learning and is most often changed between epochs/iterations. This is mainly done with two parameters: decay and momentum. There are many different learning rate schedules but the most common are time-based, step-based and exponential. Decay serves to settle the learning in a nice place and avoid oscillations, a situation that may arise when too high a constant learning rate makes the learning jump back and forth over a minimum, and is controlled by a hyperparameter. Momentum is analogous to a ball rolling down a hill; we want the ball to settle at the lowest point of the hill (corresponding to the lowest error). Momentum both speeds up the learning (increasing the learning rate) when the error cost gradient is heading in the same direction for a long time and also avoids local minima by 'rolling over' small bumps. Momentum is controlled by a hyperparameter analogous to a ball's mass which must be chosen manually—too high and the ball will roll over minima which we wish to find, too low and it will not fulfil its purpose. The formula for factoring in the momentum is more complex than for decay but is most often built in with deep learning libraries such as Keras. Time-based learning schedules alter the learning rate depending on the learning rate of the previous time iteration. Factoring in the decay the mathematical formula for the learning rate is: η n + 1 = η 0 1 + d n {\displaystyle \eta _{n+1}={\frac {\eta _{0}}{1+dn}}} where η {\displaystyle \eta } is the learning rate, η 0 {\displaystyle \eta _{0}} is the original learning rate, d {\displaystyle d} is a decay parameter and n {\displaystyle n} is the iteration step. Step-based learning schedules changes the learning rate according to some predefined steps. The decay application formula is here defined as: η n = η 0 d ⌊ 1 + n r ⌋ {\displaystyle \eta _{n}=\eta _{0}d^{\left\lfloor {\frac {1+n}{r}}\right\rfloor }} where η n {\displaystyle \eta _{n}} is the learning rate at iteration n {\displaystyle n} , η 0 {\displaystyle \eta _{0}} is the initial learning rate, d {\displaystyle d} is how much the learning rate should change at each drop (0.5 corresponds to a halving) and r {\displaystyle r} corresponds to the drop rate, or how often the rate should be dropped (10 corresponds to a drop every 10 iterations). The floor function ( ⌊ … ⌋ {\displaystyle \lfloor \dots \rfloor } ) here drops the value of its input to 0 for all values smaller than 1. Exponential learning schedules are similar to step-based, but instead of steps, a decreasing exponential function is used. The mathematical formula for factoring in the decay is: η n = η 0 e − d n {\displaystyle \eta _{n}=\eta _{0}e^{-dn}} where d {\displaystyle d} is a decay parameter. == Adaptive learning rate == The issue with learning rate schedules is that they all depend on hyperparameters that must be manually chosen for each given learning session and may vary greatly depending on the problem at hand or the model used. To combat this, there are many different types of adaptive gradient descent algorithms such as Adagrad, Adadelta, RMSprop, and Adam which are generally built into deep learning libraries such as Keras.
Read more →
TD-Gammon

TD-Gammon is a computer backgammon program developed in the 1990s by Gerald Tesauro at IBM's Thomas J. Watson Research Center. Its name comes from the fact that it is an artificial neural net trained by a form of temporal-difference learning, specifically TD-Lambda. It explored strategies that humans had not pursued and led to advances in the theory of correct backgammon play. In 1993, TD-Gammon (version 2.1) was trained with 1.5 million games of self-play, and achieved a level of play just slightly below that of the top human backgammon players of the time. In 1998, during a 100-game series, it was defeated by the world champion by a mere margin of 8 points. Its unconventional assessment of some opening strategies had been accepted and adopted by expert players. TD-gammon is commonly cited as an early success of reinforcement learning and neural networks, and was cited in, for example, papers for deep Q-learning and AlphaGo. == Algorithm for play and learning == During play, TD-Gammon examines on each turn all possible legal moves and all their possible responses (lookahead search), feeds each resulting board position into its evaluation function, and chooses the move that leads to the board position that got the highest score. In this respect, TD-Gammon is no different than almost any other computer board-game program. TD-Gammon's innovation was in how it learned its evaluation function. TD-Gammon's learning algorithm consists of updating the weights in its neural net after each turn to reduce the difference between its evaluation of previous turns' board positions and its evaluation of the present turn's board position—hence "temporal-difference learning". The score of any board position is a set of four numbers reflecting the program's estimate of the likelihood of each possible game result: White wins normally, Black wins normally, White wins a gammon, Black wins a gammon. For the final board position of the game, the algorithm compares with the actual result of the game rather than its own evaluation of the board position. The core of TD-gammon is a neural network with 3 layers. The input layer has two types of neurons. One type codes for the board position. They are non-negative integers ranging from 0 to 15, indicating the number of White or Black checkers at each board location. There are 99 input neurons for each, totaling 198 neurons. Another type codes for hand-crafted features previously used in Neurogammon. These features encoded standard concepts used by human experts, such as "advanced anchor," "blockade strength," "home board strength" and the probability of a "blot" (single checker) being hit. The hidden layer contains hidden neurons. Later versions had more of these. The output layer contains 4 neurons, representing the network's estimate of the probability ("equity") that the current board would lead to. The 4 neurons code for: White normal win, White gammon win, Black normal win, Black gammon win. Backgammon win is so rare that Tesauro opted to not represent it. After each turn, the learning algorithm updates each weight in the neural net according to the following rule: w t + 1 − w t = α ( Y t + 1 − Y t ) ∑ k = 1 t λ t − k ∇ w Y k {\displaystyle w_{t+1}-w_{t}=\alpha (Y_{t+1}-Y_{t})\sum _{k=1}^{t}\lambda ^{t-k}\nabla _{w}Y_{k}} where: It was found that picking small λ {\displaystyle \lambda } offered performance roughly equally good, and large λ {\displaystyle \lambda } degraded performance. Because of this, after 1992, TD-Gammon was trained with λ = 0 {\displaystyle \lambda =0} , degenerating into standard TD-learning. This saved compute by a factor of 2. == Development history == Version 1.0 used simple 1-ply search: every next move is scored by the neural net, and the highest-scoring move is selected. Versions 2.0 and 2.1 used 2-ply search: Make a 1-ply analysis to remove unlikely moves ("forward pruning"). Make a 2-play minimax analysis for only the likely moves. Pick the best move, probability-weighted by each of the opponent's 21 possible dice rolls (weighting non-doubles twice as much as doubles). Versions 3.0 and 3.1 used 3-ply search, using 21 2 = 441 {\displaystyle 21^{2}=441} possible dice rolls instead of 21. The last version, 3.1, was trained specifically for an exhibition match against Malcolm Davis at the 1998 AAAI Hall of Champions. It lost at -8 points, mainly due to one blunder, where TD-Gammon opted to double and got gammoned at -32 points. == Experiments and stages of training == Unlike previous neural-net backgammon programs such as Neurogammon (also written by Tesauro), where an expert trained the program by supplying the "correct" evaluation of each position, TD-Gammon was at first programmed "knowledge-free". In early experimentation, using only a raw board encoding with no human-designed features, TD-Gammon reached a level of play comparable to Neurogammon: that of an intermediate-level human backgammon player. Even though TD-Gammon discovered insightful features on its own, Tesauro wondered if its play could be improved by using hand-designed features like Neurogammon's. Indeed, the self-training TD-Gammon with expert-designed features soon surpassed all previous computer backgammon programs. It stopped improving after about 1,500,000 games (self-play) using a three-layered neural network, with 198 input units encoding expert-designed features, 80 hidden units, and one output unit representing predicted probability of winning. == Advances in backgammon theory == TD-Gammon's exclusive training through self-play (rather than imitation learning) enabled it to explore strategies that humans previously had not considered or had ruled out erroneously. Its success with unorthodox strategies had a significant impact on the backgammon community. Late 1991, Bill Robertie, Paul Magriel, and Malcolm Davis, were invited to play against TD-Gammon (version 1.0). A total of 51 games were played, with TD-Gammon losing at -0.25 ppg. Robertie found TD-Gammon to be at the level of a competent advanced player, and better than any previous backgammon program. Robertie subsequently wrote about the use of TD-Gammon for backgammon study. For example, on the opening play, the conventional wisdom was that given a roll of 2-1, 4-1, or 5-1, White should move a single checker from point 6 to point 5. Known as "slotting", this technique trades the risk of a hit for the opportunity to develop an aggressive position. TD-Gammon found that the more conservative play of splitting 24-23 was superior. Tournament players began experimenting with TD-Gammon's move, and found success. Within a few years, slotting had disappeared from tournament play, replaced by splitting, though in 2006 it made a reappearance for 2-1. Backgammon expert Kit Woolsey found that TD-Gammon's positional judgement, especially its weighing of risk against safety, was superior to his own or any human's. TD-Gammon's excellent positional play was undercut by occasional poor endgame play. The endgame requires a more analytical approach, sometimes with extensive lookahead. TD-Gammon's limitation to two-ply lookahead put a ceiling on what it could achieve in this part of the game. TD-Gammon's strengths and weaknesses were the opposite of symbolic artificial intelligence programs and most computer software in general: it was good at matters that require an intuitive "feel" but bad at systematic analysis. It is also poor at doubling strategies. This is likely due to the fact that the neural network is trained without the doubling cube, with the doubling added by feeding the neural network's cubeless equity estimates into theoretically-based heuristic formulae. This was particularly the case in the 1998 exhibition match, where it played 100 games against Malcolm Davis. A single doubling blunder lost the match. TD-gammon was never commercialized or released to the public in some other form, but it inspired commercial backgammon programs based on neural networks, such as JellyFish (1994) and Snowie (1998).
Read more →
Cognitive tutor

A cognitive tutor is a particular kind of intelligent tutoring system that utilizes a cognitive model to provide feedback to students as they are working through problems. This feedback will immediately inform students of the correctness, or incorrectness, of their actions in the tutor interface; however, cognitive tutors also have the ability to provide context-sensitive hints and instruction to guide students towards reasonable next steps. == Introduction == The name of Cognitive Tutor now usually refers to a particular type of intelligent tutoring system produced by Carnegie Learning for high school mathematics based on John Anderson's ACT-R theory of human cognition. However, cognitive tutors were originally developed to test ACT-R theory for research purposes since the early 1980s and they are developed also for other areas and subjects such as computer programming and science. Cognitive Tutors can be implemented into classrooms as a part of blended learning that combines textbook and software activities. The Cognitive Tutor programs utilize cognitive model and are based on model tracing and knowledge tracing. Model tracing means that the cognitive tutor checks every action performed by students such as entering a value or clicking a button, while knowledge tracing is used to calculate the required skills students learned by measuring them on a bar chart called Skillometer. Model tracing and knowledge tracing are essentially used to monitor students' learning progress, guide students to correct path to problem solving, and provide feedback. The Institute of Education Sciences published several reports regarding the effectiveness of Carnegie Cognitive Tutor. A 2013 report concluded that Carnegie Learning Curricula and Cognitive Tutor was found to have mixed effects on mathematics achievement for high school students. The report identified 27 studies that investigate the effectiveness of Cognitive Tutor, and the conclusion is based on 6 studies that meet What Works Clearinghouse standards. Among the 6 studies included, 5 of them show intermediate to significant positive effect, while 1 study shows statistically significant negative effect. Another report published by Institute of Education Sciences in 2009 found that Cognitive Tutor Algebra I to have potentially positive effects on math achievement based on only 1 study out of 14 studies that meets What Works Clearinghouse standards. It should be understood that What Works Clearinghouse standards call for relatively large numbers of participants, true random assignments to groups, and for a control group receiving either no treatment or a different treatment. Such experimental conditions are difficult to meet in schools, and thus only a small percentage of studies in education meet the standards of this clearinghouse, even though they may still be of value. == Theoretical foundations == === Four-component architecture === Intelligent tutoring systems (ITS) have a four-component architecture: a domain model, a student model, a tutoring model and an interface component. The domain model contains the rules, concepts, and knowledge related to the domain to be learned. It helps to evaluate students' performance and detect students' errors by setting a standard of domain expertise. The student model, the central component of an ITS, is expected to contain knowledge about the students: their cognitive and affective states, and their progress as they learn. The function of the student model is threefold: to gather data from and about the learner, to represent the learner's knowledge and learning process, and to perform diagnostics of a student's knowledge and select optimal pedagogical strategies. The tutoring model uses the data gained from the domain model and student model to make decisions about tutoring strategies such as whether or not to intervene, or when and how to intervene. Functions of the tutoring model include instruction delivery and content planning. The interface component reflects the decisions made by the tutoring model in different forms such as Socratic dialogs, feedback and hints. Students interact with the tutor through the learning interface, also known as communication. The interface provides domain knowledge elements. === Cognitive model === A cognitive model replicates the domain knowledge and skills comparable to that of a human expert or an advanced student of the domain. A cognitive model enables intelligent tutoring systems to respond to problem-solving situations in a way similar to a human tutor. A tutoring system adopting a cognitive model is called a cognitive tutor. A cognitive model is an expert system that generates a multitude of solutions to the problems presented to students. The cognitive model is used to trace each student's solution through complex alternative solution paths, enabling the tutor to provide step-by-step feedback and advice, and to maintain a targeted model of the student's knowledge based on student performance. === Cognitive Tutors === Cognitive Tutors provide step-by-step guidance as a learner develops a complex problem-solving skill through practice. Typically, cognitive tutors provide such forms of support as: (a) a problem-solving environment that is designed rich and "thinking visible"; (b) step-by-step feedback on student performance; (c) feedback messages specific to errors; (d) context-specific next-step hints at student's request, and (e) individualized problem selection. Cognitive Tutors accomplish two of the principal tasks characteristic of human tutoring: (1) monitors the student's performance and providing context-specific individual instruction, and (2) monitors the student's learning and selects appropriate problem-solving activities. Both cognitive model and two underlying algorithms, model tracing and knowledge tracing, are used to monitor the student's learning. In model tracing, the cognitive tutor uses the cognitive model in complex problems to follow the student's individual path and provide prompt accuracy feedback and context-specific advice. In knowledge tracing, the cognitive tutor uses a Bayesian Knowledge Tracing method of evaluating the student's knowledge and uses this student model to select appropriate problems for each student. === Cognitive architecture === Cognitive tutor development is guided by ACT-R cognitive architecture, which specifies the underlying framework developing the cognitive model or expert component of a cognitive tutor. ACT-R, a member of the ACT family, is the most recent cognitive architecture, devoted primarily to modelling human behavior. ACT-R includes a declarative memory of factual knowledge and a procedural memory of production rules. The architecture functions by matching productions on perceptions and facts, mediated by the real-valued activation levels of objects, and executing them to affect the environment or alter declarative memory. ACT-R has been used to model psychological aspects such as memory, attention, reasoning, problem solving, and language processing. == Application and utilization == The first real world applications of cognitive tutors were in the 1980s and involved a geometry proof tutor used by high school students and a LISP programming tutor used by college students in a mini course in introductory programming course at Carnegie Mellon University. Since then, cognitive tutors have been used in a variety of scenarios, with a few organizations developing their own cognitive tutor programs. These programs have been used with students spanning elementary school through university level, though primarily in the subject areas of Computer Programming, Mathematics, and Science. One of the first organizations to develop a system for use within the school system was the PACT Center at Carnegie Mellon University. Their aim was to "...develop systems that provide individualized assistance to students as they work on challenging real-world problems in complex domains such as computer programming, algebra and geometry". PACT's most successful product was the Cognitive Tutor Algebra course. Originally created in the early 1990s, this course was in use in 75 schools through the U.S. by 1999, and then its spin-off company, Carnegie Learning, now offers tutors to thousands of schools in the U.S. The Carnegie Mellon Cognitive Tutor has been shown to raise students' math test scores in high school and middle-school classrooms, and their Algebra course was designated one of five exemplary curricula for K-12 mathematics educated by the US Department of Education. There were several research projects conducted by the PACT Center to utilize Cognitive tutor for courses in Excel and to develop an intelligent tutoring system for algebra expression writing, called Ms. Lindquist. Further, in 2005, Carnegie Learning released Bridge to Algebra, a product intended for middle schools that was piloted in over 100 schools. Cognitive tutoring software is continuing to be used.
Read more →