The Old Axolotl (Polish: Starość aksolotla) is a 2015 digital-only novel by Polish science-fiction author Jacek Dukaj. The novel was released in Polish on March 10, 2015, and shortly afterward, on March 24 that year, in English (translated by Stanley Bill). It has been described as "an experiment in reading (and creating) the electronic literature of the future". It is Dukaj's first novel to be published in English, though several of his short stories (The Golden Galley, 1996, The Iron General, 2010, The Apocrypha of Lem, 2011) have been translated prior to this. The novel has inspired two Netflix original series: the 2020 Belgian Into the Night, and its 2022 Turkish language spin-off Yakamoz S-245. == Plot == The novel presents a post-apocalyptic, cyberpunk vision of Earth where biological life has been wiped out, inhabited by robots and mechs, many of which are humans whose consciousness has been digitized in the wake of an extinction event. == Significance and analysis == The novel is an example of electronic literature, available only in digital formats, and has no traditional paper version. It was designed from the beginning not only to incorporate more traditional elements such as illustrations, but also hypertext, and 3D-printable models of main robotic characters designed by Alex Jaeger, the art director of Transformers films. The novel composition is layered, with the narrative layer, an encyclopedic/hyperlinked footnote layer, and a multimedia layer, including illustrations and a short promotional video by the Oscar-nominated Platige Image studio. One of the novel's central questions is: "What does it mean to be human?" Other subjects include post humanism and other "staples of cyberpunk and related genres, such as the artificial intelligence". The novel is representative of Dukaj's prose, posing philosophical questions about the future of man and technology. The author explained that: "stories such as The Old Axolotl that model an ‘escape from the body’ are born out of a sense of progress as a process of ‘de-animalising’ human beings through science. This has its origin in the pre-Enlightenment intuition of ‘liberation from nature’. For one of the last shackles of nature is corporeality itself, the limitations of our physicality." The other major element of the novel is Dukaj's attempts to introduce the reader to the new style of electronic literature. The novel was nominated for the 2016 Janusz A. Zajdel Award.
Multi-armed bandit
In probability theory and machine learning, the multi-armed bandit problem (sometimes called the K- or N-armed bandit problem) is named from imagining a gambler at a row of slot machines (sometimes known as "one-armed bandits"), who has to decide which machines to play, how many times to play each machine and in which order to play them, and whether to continue with the current machine or try a different machine. More generally, it is a problem in which a decision maker iteratively selects one of multiple fixed choices (i.e., arms or actions) when the properties of each choice are only partially known at the time of allocation, and may become better understood as time passes. A fundamental aspect of bandit problems is that choosing an arm does not affect the properties of the arm or other arms. Instances of the multi-armed bandit problem include the task of iteratively allocating a fixed, limited set of resources between competing (alternative) choices in a way that minimizes the regret. A notable alternative setup for the multi-armed bandit problem includes the "best arm identification (BAI)" problem where the goal is instead to identify the best choice by the end of a finite number of rounds. The multi-armed bandit problem is a classic reinforcement learning problem that exemplifies the exploration–exploitation tradeoff dilemma. In contrast to general reinforcement learning, the selected actions in bandit problems do not affect the reward distribution of the arms. The multi-armed bandit problem also falls into the broad category of stochastic scheduling. In the problem, each machine provides a random reward from a probability distribution specific to that machine, that is not known a priori. The objective of the gambler is to maximize the sum of rewards earned through a sequence of lever pulls. The crucial tradeoff the gambler faces at each trial is between "exploitation" of the machine that has the highest expected payoff and "exploration" to get more information about the expected payoffs of the other machines. The trade-off between exploration and exploitation is also faced in machine learning. In practice, multi-armed bandits have been used to model problems such as managing research projects in a large organization, like a science foundation or a pharmaceutical company. In early versions of the problem, the gambler begins with no initial knowledge about the machines. Herbert Robbins in 1952, realizing the importance of the problem, constructed convergent population selection strategies in "some aspects of the sequential design of experiments". A theorem, the Gittins index, first published by John C. Gittins, gives an optimal policy for maximizing the expected discounted reward. == Empirical motivation == The multi-armed bandit problem models an agent that simultaneously attempts to acquire new knowledge (called "exploration") and optimize their decisions based on existing knowledge (called "exploitation"). The agent attempts to balance these competing tasks in order to maximize their total value over the period of time considered. There are many practical applications of the bandit model, for example: clinical trials investigating the effects of different experimental treatments while minimizing patient losses, adaptive routing efforts for minimizing delays in a network, financial portfolio design In these practical examples, the problem requires balancing reward maximization based on the knowledge already acquired with attempting new actions to further increase knowledge. This is known as the exploitation vs. exploration tradeoff in machine learning. The model has also been used to control dynamic allocation of resources to different projects, answering the question of which project to work on, given uncertainty about the difficulty and payoff of each possibility. Originally considered by Allied scientists in World War II, it proved so intractable that, according to Peter Whittle, the problem was proposed to be dropped over Germany so that German scientists could also waste their time on it. The version of the problem now commonly analyzed was formulated by Herbert Robbins in 1952. == The multi-armed bandit model == The multi-armed bandit (short: bandit or MAB) can be seen as a set of real distributions B = { R 1 , … , R K } {\displaystyle B=\{R_{1},\dots ,R_{K}\}} , each distribution being associated with the rewards delivered by one of the K ∈ N + {\displaystyle K\in \mathbb {N} ^{+}} levers. Let μ 1 , … , μ K {\displaystyle \mu _{1},\dots ,\mu _{K}} be the mean values associated with these reward distributions. The gambler iteratively plays one lever per round and observes the associated reward. The objective is to maximize the sum of the collected rewards. The horizon H {\displaystyle H} is the number of rounds that remain to be played. The bandit problem is formally equivalent to a one-state Markov decision process. The regret ρ {\displaystyle \rho } after T {\displaystyle T} rounds is defined as the expected difference between the reward sum associated with an optimal strategy and the sum of the collected rewards: ρ = T μ ∗ − ∑ t = 1 T r ^ t {\displaystyle \rho =T\mu ^{}-\sum _{t=1}^{T}{\widehat {r}}_{t}} , where μ ∗ {\displaystyle \mu ^{}} is the maximal reward mean, μ ∗ = max k { μ k } {\displaystyle \mu ^{}=\max _{k}\{\mu _{k}\}} , and r ^ t {\displaystyle {\widehat {r}}_{t}} is the reward in round t {\displaystyle t} . A zero-regret strategy is a strategy whose average regret per round ρ / T {\displaystyle \rho /T} tends to zero with probability 1 when the number of played rounds tends to infinity. Intuitively, zero-regret strategies are guaranteed to converge to a (not necessarily unique) optimal strategy if enough rounds are played. == Variations == A common formulation is the Binary multi-armed bandit or Bernoulli multi-armed bandit, which issues a reward of one with probability p {\displaystyle p} , and otherwise a reward of zero. Another formulation of the multi-armed bandit has each arm representing an independent Markov machine. Each time a particular arm is played, the state of that machine advances to a new one, chosen according to the Markov state evolution probabilities. There is a reward depending on the current state of the machine. In a generalization called the "restless bandit problem", the states of non-played arms can also evolve over time. There has also been discussion of systems where the number of choices (about which arm to play) increases over time. Computer science researchers have studied multi-armed bandits under worst-case assumptions, obtaining algorithms to minimize regret in both finite and infinite (asymptotic) time horizons for both stochastic and non-stochastic arm payoffs. === Best arm identification === An important variation of the classical regret minimization problem in multi-armed bandits is best arm identification (BAI), also known as pure exploration. This problem is crucial in various applications, including clinical trials, adaptive routing, recommendation systems, and A/B testing. In BAI, the objective is to identify the arm having the highest expected reward. An algorithm in this setting is characterized by a sampling rule, a decision rule, and a stopping rule, described as follows: Sampling rule: ( a t ) t ≥ 1 {\displaystyle (a_{t})_{t\geq 1}} is a sequence of actions at each time step Stopping rule: τ {\displaystyle \tau } is a (random) stopping time which suggests when to stop collecting samples Decision rule: a ^ τ {\displaystyle {\hat {a}}_{\tau }} is a guess on the best arm based on the data collected up to time τ {\displaystyle \tau } There are two predominant settings in BAI: Fixed budget setting: Given a time horizon T ≥ 1 {\displaystyle T\geq 1} , the objective is to identify the arm with the highest expected reward a ⋆ ∈ arg max k μ k {\displaystyle a^{\star }\in \arg \max _{k}\mu _{k}} minimizing probability of error δ {\displaystyle \delta } . Fixed confidence setting: Given a confidence level δ ∈ ( 0 , 1 ) {\displaystyle \delta \in (0,1)} , the objective is to identify the arm with the highest expected reward a ⋆ ∈ arg max k μ k {\displaystyle a^{\star }\in \arg \max _{k}\mu _{k}} with the least possible amount of trials and with probability of error P ( a ^ τ ≠ a ⋆ ) ≤ δ {\displaystyle \mathbb {P} ({\hat {a}}_{\tau }\neq a^{\star })\leq \delta } . For example using a decision rule, we could use m 1 {\displaystyle m_{1}} where m {\displaystyle m} is the machine no.1 (you can use a different variable respectively) and 1 {\displaystyle 1} is the amount for each time an attempt is made at pulling the lever, where ∫ ∑ m 1 , m 2 , ( . . . ) = M {\displaystyle \int \sum m_{1},m_{2},(...)=M} , identify M {\displaystyle M} as the sum of each attempts m 1 + m 2 {\displaystyle m_{1}+m_{2}} , (...) as needed, and from there you can get a ratio, sum or mean as quantitative probability and sample your formulation for each slots. You can also do ∫ ∑ k ∝ i N − (
International Journal of Pattern Recognition and Artificial Intelligence
The International Journal of Pattern Recognition and Artificial Intelligence was founded in 1987 and is published by World Scientific. The journal covers developments in artificial intelligence, and its sub-field, pattern recognition. This includes articles on image and language processing, robotics and neural networks. == Abstracting and indexing == The journal is abstracted and indexed in: SciSearch ISI Alerting Services CompuMath Citation Index Current Contents/Engineering, Computing & Technology Inspec io-port.net Compendex Computer Abstracts
Linagora
Linagora is a French open source software editor, founded in June 2000 by Alexandre Zapolsky and Michel-Marie Maudet. Located in France, as well as in Belgium, Canada, Vietnam, the United States and Tunisia, the company employs around 200 people. In 2023, Linagora created the OpenLLM France community, alongside other French Artificial Intelligence companies and organizations. In 2025, the company launched Lucie, an opensource Large Language Model. == History == Linagora was founded on June 28, 2000. Its name is a contraction of the words "Linux" and "Agora". The company was founded by Alexandre Zapolsky and Michel-Marie Maudet. Soon after, the two entrepreneurs were joined by Alexandre Zapolsky's wife and brother, who took on the roles of commercial director and administrative and financial director of the SME. In 2007, the company was selected by the French National Assembly to provide the software for Linux computers, replacing Microsoft Windows. Linagora then claimed the position of the leading French open source software company by revenue. In 2015, French Prime Minister Manuel Valls allocated €10.7 million from the "Investments for the Future" fund for a research program aimed at developing a new generation of open source software platforms based on Linagora's offerings. In September 2016, Linagora launched the social network "La Cerise" for the newspaper L'Humanité. This app offered a service and tool for readers and citizens mobilizing for causes. It aimed to share engagement through petitions, discussions, agendas, and contacts. In October 2016, the company won two public contracts for supporting open source software in forty-two French ministries and other administrative entities. In May 2019, Linagora organized a fundraising event in the presence of the French Secretary of State for Digital Affairs, Cédric O, to celebrate its 19th anniversary. The funds were intended for: Supporting parents of hospitalized Polynesian children in France. Equipping primary school students with digital devices (tablets or PCs). Establishing a digital academy "OpenHackademy" in French Polynesia to train unemployed youth in digital skills and help them find jobs. In December 2022, Linagora acquired a property known as "Maison Rocher" and later "Maison Chocolat," located on the Île Saint-Germain in Issy-les-Moulineaux. Renamed "Villa Good Tech" by Linagora, this award-winning architectural work by Éric Daniel-Lacombe became the company's new headquarters, aiming to provide a space for associative actors and companies to develop technologies that contribute to a better world. In July 2023, Linagora launched OpenLLM France, a community initially comprising around twenty actors focused on generative AI. The goal was to develop a sovereign and open source large language model. This initiative, led by co-founder and CEO Michel-Marie Maudet, had more than four hundred French members by early 2024. and announced its expansion to the European sphere during Fosdem 2024. In February 2024, the CNRS and Linagora signed a framework agreement to strengthen their research collaboration. In January 2025, Linagora released Lucie, an open source and sovereign AI that faced ridicule due to tests on an unfinished, uncensored version designed for scientific and experimental use. The platform divided opinions between those who saw it as a technological achievement and those who criticized it as "French bashing" compared to American and Chinese AIs. == Acquisitions == The company acquired: In July 2007, the SME AliaSource, based in Ramonville-Saint-Agne and led by its founder, Pierre Baudracco. In 2008, the open source web hosting company Netaktiv, a member of the GIE Gitoyen, announced during the 2008 Solutions Linux trade show. In 2012, the Toulouse-based company EBM Websourcing, the publisher of the open-source software Petals Link, and took over its development. In 2016, the digital agency Neoma Interactive, specializing in UX design and digital communication strategy. == Locations == In 2017, the company's headquarters was located in Issy-les-Moulineaux, with branches in Lyon, Toulouse, Marseille, and internationally in Brussels, San Francisco, Montreal, Vietnam, and Tunisia. In 2005, the company attempted to establish a presence in Nantes. In 2024, the headquarters was moved to Issy-les-Moulineaux. == Activity == === Software === Twake Workplace One of Linagora's flagship products is Twake Workplace, which stands out as a 100% open-source solution compared with those of the GAFAMs. Twake Workplace is available as a complete platform or module by module. It includes : Twake Mail, a powerful modern messaging solution based on the JMAP protocol and the James email server from the Apache Foundation, for which Linagora provides technical management; Twake Chat, an instant communications solution for businesses developed using the Matrix protocol and compatible with the French government's chat solution, Tchap; Twake Drive, an easy-to-use collaborative platform for group work using OnlyOffice. ==== OpenPaaS ==== In 2018, the search engine Qwant announced that its email service Qwantmail would be based on the OpenPaaS product. In 2022, Qwant announced the abandonment of its Qwantmail project due to Linagora's collection of personal email addresses and serious security breaches. The site Next (formerly PC INpact) published an article in January 2020 criticizing the "failures and delays" of the Qwantmail project led by Linagora, which led to the CNIL's intervention regarding Qwant and Linagora. ==== LinTO ==== In 2017, Linagora launched its open source voice assistant project named LinTO. This enterprise voice assistant, described as "GAFAM Free," was presented at CES 2018 in Las Vegas. The LinTO voice framework was developed as part of the eponymous research project funded by Bpifrance (Grands Défis du Numérique instrument). === Services === ==== OSSA (Open Source Software Assurance) ==== One of the company's main activities is OSSA. Through OSSA, Linagora provided support for open source software for 42 ministries and other administrative entities in 2012. == Legal issues == === Dispute with BlueMind === In 2012, a legal dispute arose between BlueMind and Linagora. Linagora accused BlueMind of copyright infringement, unfair competition, and breach of a non-compete clause, leading to several legal actions. Linagora sued BlueMind for copyright infringement and unfair competition in the Bordeaux court, which ruled in Linagora's favor for unfair competition and parasitism but rejected the copyright claim. BlueMind was ordered to pay nearly €170,000 to Linagora. Linagora sued former associates Pierre Baudracco and Pierre Carlier in the Paris Commercial Court for breach of a non-compete clause and violation of a warranty of eviction. The court dismissed Linagora's claims and ordered it to pay €20,000 each to Baudracco and Carlier. Linagora appealed, and the Paris Court of Appeal partially overturned the decision, awarding Linagora €480,000. BlueMind sued Linagora for defamation and public insult in the Toulouse Criminal Court. The court ruled against Linagora, but the decision was overturned by the Court of Cassation in January 2024, and the case was remanded for retrial. === Conviction for wrongful termination and harassment === On June 14, 2017, France 3 reported on a decision by the Versailles Court of Appeal, which ruled that Linagora had wrongfully terminated an employee and subjected them to moral harassment. The court ordered Linagora to pay the employee €22,000 for wrongful termination, €11,000 for notice pay, €6,600 for legal severance pay, €3,200 for conservative suspension, and €3,000 for moral harassment.
Harvey (software)
Harvey is a generative artificial intelligence (AI) product developed by the Counsel AI Corporation for the legal industry. The product has been described as a provider of customised large language models (LLMs) for law firms and in-house legal teams. It is named after the lead character of the legal drama Suits, Harvey Specter. == History == Harvey was founded in the summer of 2022 by Winston Weinberg, who was a securities and antitrust litigator at O'Melveny & Myers, and Gabriel Pereyra, who was a research scientist at Google DeepMind and Meta. Pereyra and Weinberg were roommates in Los Angeles. Pereyra was brainstorming startup ideas with his research colleagues. He showed Weinberg OpenAI's GPT-3 text-generating system, and Weinberg realized that it could be used to improve legal workflows. They developed an early chain-of-thought prompt based on GPT-3, focused on California tenant law. They ran the model on 100 legal questions from a public forum and hired three attorneys to evaluate the answers and determine whether they could be sent to clients unchanged. Out of those 100 questions, 86 were approved. After that, Pereyra and Weinberg contacted Sam Altman and Jason Kwon, General Counsel at OpenAI, about their results. Shortly after, on July 4, 2022, they met with OpenAI's C-suite, and OpenAI became their seed investor. OpenAI also gave Pereyra and Weinberg early access to GPT-4. Gordon Moodie, a corporate partner at Wachtell, Lipton, Rosen & Katz, also joined Harvey in July 2023 as the company's chief product officer. In March 2024, Harvey had 82 employees and stated that it intended to double that figure by the end of 2024. The company has reportedly hired a large number of lawyers, including from White & Case, Latham & Watkins, Skadden, Gunderson Dettmer, Katten Muchin Rosenman, and Paul Weiss. Harvey CEO Weinberg explained that many members of the company's sales team were formerly attorneys at 'Big Law', i.e. large US law firms, and that the sales team's experience was useful in convincing attorneys to trial the company's software. The integration of former 'Big Law' attorneys into product and sales teams has been attributed as a major factor in Harvey's success. In February 2026, Harvey announced its first brand partnership with actor Gabriel Macht, who portrayed the character Harvey Specter in Suits, to launch the company's Instagram page. In May 2026, it was announced the company is sponsoring the Golden State Valkyries and the New York Liberty. == Funding == In November 2022, it was reported that Harvey raised US$5 million in funding led by the OpenAI Startup Fund, together with other investors such as Jeff Dean, the head of Google AI, Elad Gil, the founder of Mixer Labs, Sarah Guo, the founder of Conviction, and other angel investors. Harvey raised another $23 million in April 2023 in a funding round led by Sequoia Capital. Harvey announced in December 2023 that it had raised $80 million in a Series B funding round led by Elad Gil and Kleiner Perkins which valued the company at $715 million. Other investors in the round included Sequoia Capital and the OpenAI Startup Fund. In July 2024, Harvey announced that it had raised $100 million in a Series C funding round that valued the company at $1.5 billion. The round was led by venture capital firm GV, and other participants included OpenAI, Kleiner Perkins, Sequoia Capital, Elad Gil, and SV Angel. In February 2025, Harvey announced it had raised $300 million in a Series D funding round that valued the company at $3 billion. Just months later, in June 2025, Harvey closed a $300 million Series E co-led by Kleiner Perkins and Coatue, again with participation from Conviction, Elad Gil, OpenAI, and Sequoia, boosting its valuation to about $5 billion and supporting international growth and expanded legal product offerings. In December 2025, Harvey secured a $160 million Series F round led by Andreessen Horowitz, with continued participation from investors including EQT, WndrCo, Sequoia, Kleiner Perkins, Conviction, and Elad Gil, valuing the legal AI company at roughly $8 billion. In March 2026, Harvey raised $200 million at a valuation of $11 billion, in a round co-led by GIC and Sequoia Capital. == Features == In May 2024, Harvey launched its products on Microsoft Azure and stated that it would offer a Harvey on Azure version of its product going forward. It was also reported that Harvey would begin offering general commercial access to some of its products, such as its case law models, as well as product bundles that included its AI assistant, specialised models, and its Vault feature for running prompts on large document collections. == Applications == Various law firms around the world are customers of Harvey. US law firm Paul Weiss began testing Harvey within the firm in January 2023, and became a client of the company later that year. Gina Lynch, the firm's chief knowledge and innovation officer, explained that the firm was not using hard metrics, such as time saved, to assess productivity gains because the time and effort needed to carefully review the output made efficiency gains difficult to measure. In February 2023, the UK law firm, Allen & Overy (now A&O Shearman), announced that it had been trialing Harvey since November 2022 within its Markets Innovation Group. This was reported to be the first known use of a generative AI product within the UK magic circle law firms. According to Allen & Overy, during the trial, 3,500 lawyers had used Harvey for around 40,000 queries in the course of their day to day work. The firm's press release stated that "Whilst the output needs careful review by an A&O lawyer, Harvey can help generate insights, recommendations and predictions based on large volumes of data". David Wakeling, head of the Markets Innovation Group, also cautioned that "You must validate everything coming out of the system. You have to check everything". The Irish law firm, A&L Goodbody, announced in February 2024 that it would be working with Harvey to enhance its services in relation to document analysis, due diligence, litigation, and regulatory compliance. In June 2024, UK law firm Ashurst announced that it would partner with Harvey and roll out its services to its branches worldwide. In September 2024, PwC announced that it would be adopting Harvey to empower its lawyers in Singapore. Singapore law firm WongPartnership also announced that month that it had become the first Southeast Asian law firm to test Harvey's generative AI solutions.
SwissCovid
SwissCovid is a COVID-19 contact tracing app used for digital contact tracing in Switzerland. Use of the app is voluntary and based on a decentralized approach using Bluetooth Low Energy and Decentralized Privacy-Preserving Proximity Tracing (dp3t). == Development == The app was developed in collaboration with the FOPH by Federal Office for Information Technology, Systems and Communications FOITT, École polytechnique fédérale de Lausanne (EPFL) and the Swiss Federal Institute of Technology in Zurich (ETH) as well as other experts. == Non-interoperability with applications in European countries == There is an agreement between EU countries to make applications compatible. However, there is no legal basis for the SwissCovid application to be part of this portal even though technically speaking it is ready, according to Sang-Ill Kim, head of the digital transformation department of the Federal Office of Public Health. == Criticism == === Not full open source and dependence on Google and Apple === In June 2020, researchers Serge Vaudenay and Martin Vuagnoux published a critical analysis of the application, noting that it relies heavily on Google and Apple's exposure notification system, which is integrated into their respective Android and iOS operating systems. Since Google and Apple have not released the full source code of this system, this would call into question the truly open source nature of the application. The researchers note that the dp3t collective, which includes the developers of the application, has asked Google and Apple to release their code. Moreover, they criticize the official description of the application and its functionalities, as well as the adequacy of the legal basis for its effective operation. === Cyber attacks === Professor Serge Vaudenay and Martin Vuagnoux identify also various security vulnerabilities in the application. The system would thus allow a third party to trace the movements of a phone using the application by means of Bluetooth sensors scattered along its path, for example in a building. Another possible attack would be to copy identifiers from the phones of people who may be ill (for example, in a hospital), and to reproduce those identifiers in order to receive notification of exposure to COVID-19 and illegitimately benefit from quarantine (thus entitling them to paid leave, a postponed examination, or other benefits). The system would also allow a third party to use a phone using the application by means of Bluetooth sensors scattered along the way. Paul-Olivier Dehaye of Personaldata.io and professor Joel Reardon of the University of Calgary published in June 2020 several examples of AEM (Associated Encrypted Metadata) replay and manipulation attacks via software development kits (SDKs) found in benign third-party mobile applications downloaded by the general public and having the phone's Bluetooth access permissions and in September 2020 a paper indicating that "Bluetooth-based proximity tracing apps are fundamentally insecure with respect to an attacker leveraging a malevolent app or SDK". === Costs === According to a publication by the federal administration, "the costs of developing the software for the mobile phone application, the GR back-end and the code management system as well as the costs for access management for the cantonal doctors' services are estimated at a one-off amount of 1.65 million francs. However, the Zurich-based company Ubique, responsible for the development of the application, was finally awarded the mandate to develop the application for an amount of 1.8 million francs. Through the Botnar Foundation based in Basel, École polytechnique fédérale de Lausanne received 3.5 million Swiss francs for the development of the application
Richard S. Sutton
Richard Stuart Sutton (born 1957 or 1958) is a Canadian computer scientist. He is a professor of computing science at the University of Alberta, fellow & Chief Scientific Advisor at the Alberta Machine Intelligence Institute, and a research scientist at Keen Technologies. Sutton is considered one of the founders of modern computational reinforcement learning. In particular, he contributed to temporal difference learning and policy gradient methods. He received the 2024 Turing Award with Andrew Barto. == Education and early life == Richard Sutton was born in either 1957 or 1958 in Toledo, Ohio, and grew up in Oak Brook, Illinois, a suburb of Chicago, United States. Sutton received his Bachelor of Arts (BA) degree in psychology from Stanford University in 1978 before taking a Master of Science (1980) and PhD (1984) in computer science from the University of Massachusetts Amherst supervised by Andrew Barto. His doctoral dissertation introduced actor-critic architectures and temporal credit assignment. He was influenced by Harry Klopf's work in the 1970s, which proposed that supervised learning is insufficient for AI or explaining intelligent behavior, and trial-and-error learning, driven by "hedonic aspects of behavior", is necessary. This focused his interest to reinforcement learning. == Career and research == Sutton held a postdoctoral research position at the University of Massachusetts Amherst in 1984. He worked at GTE Laboratories in Waltham, Massachusetts as principal member of technical staff from 1985 to 1994, then returned to the University of Massachusetts Amherst as a senior research scientist. He joined AT&T Labs Shannon Laboratory in Florham Park, New Jersey as principal technical staff member from 1998 to 2002. He has been a professor of computing science at the University of Alberta since 2003, where he helped establish the Reinforcement Learning and Artificial Intelligence Laboratory. In 2017 he became a distinguished research scientist with Google DeepMind and helped launch DeepMind Alberta in Edmonton, a research office operated in close collaboration with the University of Alberta. 1984: Postdoctoral researcher, University of Massachusetts Amherst (Amherst, Massachusetts) 1985–1994: Principal member of technical staff, Computer and Intelligent Systems Laboratory, GTE Laboratories (Waltham, Massachusetts) 1995–1998: Senior research scientist, University of Massachusetts Amherst (Amherst, Massachusetts) 1998–2002: Principal technical staff member, Artificial Intelligence Department, AT&T Labs Shannon Laboratory (Florham Park, New Jersey) 2003–present: Professor of computing science, University of Alberta (Edmonton, Alberta) 2017–2023: Distinguished research scientist, DeepMind Alberta, Google DeepMind (Edmonton, Alberta) 2024–Present: Research scientist, Keen Technologies === Reinforcement learning === Sutton joined Andrew Barto in the early 1980s at UMass, trying to explore the behavior of neurons in the human brain as the basis for human intelligence, a concept that had been advanced by computer scientist A. Harry Klopf. Sutton and Barto used mathematics toward furthering the concept and using it as the basis for artificial intelligence. This concept became known as reinforcement learning and went on to becoming a key part of artificial intelligence techniques. Barto and Sutton used Markov decision processes (MDP) as the mathematical foundation to explain how agents (algorithmic entities) made decisions when in a stochastic or random environment, receiving rewards at the end of every action. Traditional MDP theory assumed the agents knew all information about the MDPs in their attempt toward maximizing their cumulative rewards. Barto and Sutton's reinforcement learning techniques allowed for both the environment and the rewards to be unknown, and thus allowed for these category of algorithms to be applied to a wide array of problems. Sutton returned to Canada in the 2000s and continued working on the topic which continued to develop in academic circles until one of its first major real world applications saw Google's AlphaGo program built on this concept defeating the then prevailing human champion. Barto and Sutton have widely been credited and accepted as pioneers of modern reinforcement learning, with the technique itself being foundational to the AI boom. In a 2019 essay, Sutton proposed the "bitter lesson", which criticized the field of AI research for failing to learn that "building in how we think we think does not work in the long run", arguing that "70 years of AI research [had shown] that general methods that leverage computation are ultimately the most effective, and by a large margin", beating efforts building on human knowledge about specific fields like computer vision, speech recognition, chess or Go. Sutton argues that large language models aren’t capable of learning on-the-job, and so new model architectures are required to enable continual learning. Sutton further argues that a special training phase will be unnecessary — the agent will learn on-the-fly, rendering large language models obsolete. In 2023, Sutton and John Carmack announced a partnership for the development of artificial general intelligence (AGI). === Awards and honors === Sutton has been a Fellow of the Association for the Advancement of Artificial Intelligence (AAAI) since 2001; his nomination read: "For significant contributions to many topics in machine learning, including reinforcement learning, temporal difference techniques, and neural networks." In 2003, he received the President's Award from the International Neural Network Society and in 2013, the Outstanding Achievement in Research award from the University of Massachusetts Amherst. He received the 2024 Turing Award from the Association for Computing Machinery together with Andrew Barto; the citation of the award read: "For developing the conceptual and algorithmic foundations of reinforcement learning." In 2016, Sutton was elected Fellow of the Royal Society of Canada. In 2021, he was elected Fellow of the Royal Society (FRS) of London. === Research === Sutton introduced temporal-difference methods for prediction and control, establishing convergence properties and practical algorithms. He proposed integrated learning and planning through the Dyna architecture. He co-developed the options framework for temporal abstraction in reinforcement learning. He co-authored the first modern policy gradient formulation with function approximation. Sutton's essay The Bitter Lesson argued that general methods that scale with computation dominate domain-specific approaches in the long run. His former doctoral students include David Silver and Doina Precup. === Selected publications === His publications include: == Personal life == Sutton became a Canadian citizen in 2015, and his renunciation of US citizenship was reported in 2017.