DAYDREAMER is a goal-based agent and cognitive architecture developed at the University of California, Los Angeles by Erik T. Mueller and Michael G. Dyer beginning in 1983. The system models the human stream of thought and how it is triggered and directed by emotions, simulating human daydreaming. Taking situational descriptions as input, DAYDREAMER produces English-language daydreams as output and encodes new daydreams, plans, and planning strategies for later reuse. The program comprises five components: a scenario generator based on relaxed planning, a dynamic episodic memory, a collection of personal goals and control goals, an emotion component, and domain knowledge of interpersonal relations and everyday occurrences. The source code was released under a free software license in 2015. == History == Erik Mueller began DAYDREAMER in 1983 while he was a doctoral student in the Artificial Intelligence Laboratory of the Computer Science Department at the University of California, Los Angeles, studying under Michael G. Dyer. Initial development of the project was supported by a grant from the W. M. Keck Foundation with matching funds from the UCLA School of Engineering and Applied Sciences. Additionally, Mueller was supported by an Atlantic Richfield Doctoral Fellowship and Dyer by an IBM Faculty Development Award. The first published descriptions of the program appeared in 1985 at the Ninth International Joint Conference on Artificial Intelligence in Los Angeles and at the Seventh Annual Conference of the Cognitive Science Society in Irvine. Work on the program continued, and a book, Daydreaming in Humans and Machines, was published by Ablex Publishing in 1990. The program was implemented on top of GATE, a knowledge-representation and inference substrate developed by Mueller and Uri Zernik at UCLA, and was originally written in T, a dialect of Scheme. In 2015, Mueller released the DAYDREAMER source code, version 3.5, a Common Lisp rewrite of the original T implementation, on GitHub under the GNU General Public License version 2. The release comprised approximately 12,000 lines of Common Lisp code, along with the GATE knowledge-representation substrate on which DAYDREAMER had originally been built. == Architecture == The program operates in two modes. In daydreaming mode it daydreams continuously until interrupted, while performance mode allows it to demonstrate behavior it has learned through daydreaming. === Emotion and control goals === Emotions and daydreaming form a feedback loop for DAYDREAMER. Emotions activate goals that produce daydreams, and the resulting daydreams modify existing emotions and trigger new ones, which prompt subsequent daydreaming. Recall of a goal success produces a positive emotion whereas recall of a goal failure produces a negative emotion. Emotions activate a set of goals, called control goals, which direct the course of a daydream. The program has four control goals. "Rationalization" generates reasons why an unsatisfactory outcome is in fact acceptable, in order to reduce a negative emotion and maintain self-esteem. "Revenge" is activated by anger when a failure is caused by another and reduces negative emotion through imagined retaliation. "Failure/success reversal" imagines alternative scenarios in which a failure was prevented or a success did not occur as a means of learning planning strategies for future situations. "Preparation" generates hypothetical future scenarios in order to rehearse plans and actions for events that have not yet occurred. === Scenario generator and relaxed planning === The scenario generator produces the sequence of events that make up a daydream. It operates under multiple, often conflicting personal goals rather than pursuing a single goal, applies relaxation rules that permit the generation of non-realistic scenarios, and it draws on episodic memory of past experiences both as subject matter and as a source of planning knowledge. The personal goals that guide the scenario generator include health, food, sex, friendship, love, possessions, self-esteem, social esteem, enjoyment, and achievement. These goals are organized into a goal tree that specifies their relative importance at any given time. Relaxation rules allow the program to set aside its ordinary constraints when generating a scenario. The four constraints that may be relaxed are the behavior of others, the daydreamer's own attributes, physical constraints, and social constraints. The degree of relaxation varies with the active control goal. For example a failure-reversal goal aimed at alternatives uses a low level of relaxation, whereas a revenge goal aimed at a retaliation uses a high level. === Episodic memory and analogy === DAYDREAMER's episodic memory stores its personal and vicarious experiences along with the daydreams it generates. The memory is described as dynamic because it is continually modified during daydreaming such that previously daydreamed episodes become available alongside real ones. As it daydreams, the program indexes daydreams, future plans or actions, and planning strategies into memory. Episodes are organized and retrieved using surface-level similarities, emotions, abstract themes, and Plot Units which are abstract configurations of positive and negative outcomes developed by Wendy Lehnert. A recalled episode is adapted to the current situation through analogy, which requires less effort than generating an equivalent scenario from scratch. == Sample output == In the sample experience from the source code, called LOVERS1, DAYDREAMER begins from an initial situation in which it has a job, is not romantically involved, and is at home. Starting in daydreaming mode, it activates a top-level goal to be in a romantic relationship because it is not currently in one, and a positive motivating emotion of interest becomes associated with that goal. The program then activates a goal to be entertained and pursues seeing a film as a way to achieve it. Facts asserted into memory are converted to English and produced as output, such as "I want to be going out with someone" and "I have to go see a movie". == Reception and influence == DAYDREAMER has been cited in research on computational models of creativity, emotion, and narrative. Linda Wills and Janet Kolodner cite the program as an example of work on opportunism in their study of serendipitous recognition in design. Joseph Bates, A. Bryan Loyall, and W. Scott Reilly of the Carnegie Mellon Oz Project cite DAYDREAMER among prior work in their description of an architecture combining action, emotion, and social behavior. Rafael Pérez y Pérez, Ricardo Sosa, and Christian Lemaitre cite Mueller's DAYDREAMER as one of the few computer models at the time to model daydreaming during the creative process. Jichen Zhu and D. Fox Harrell likewise cite the program in their work on imagining and agency in generative interactive narrative.
View model
A view model or viewpoints framework in systems engineering, software engineering, and enterprise engineering is a framework which defines a coherent set of views to be used in the construction of a system architecture, software architecture, or enterprise architecture. A view is a representation of the whole system from the perspective of a related set of concerns. Since the early 1990s there have been a number of efforts to prescribe approaches for describing and analyzing system architectures. A result of these efforts have been to define a set of views (or viewpoints). They are sometimes referred to as architecture frameworks or enterprise architecture frameworks, but are usually called "view models". Usually a view is a work product that presents specific architecture data for a given system. However, the same term is sometimes used to refer to a view definition, including the particular viewpoint and the corresponding guidance that defines each concrete view. The term view model is related to view definitions. == Overview == The purpose of views and viewpoints is to enable humans to comprehend very complex systems, to organize the elements of the problem and the solution around domains of expertise and to separate concerns. In the engineering of physically intensive systems, viewpoints often correspond to capabilities and responsibilities within the engineering organization. Most complex system specifications are so extensive that no single individual can fully comprehend all aspects of the specifications. Furthermore, we all have different interests in a given system and different reasons for examining the system's specifications. A business executive will ask different questions of a system make-up than would a system implementer. The concept of viewpoints framework, therefore, is to provide separate viewpoints into the specification of a given complex system in order to facilitate communication with the stakeholders. Each viewpoint satisfies an audience with interest in a particular set of aspects of the system. Each viewpoint may use a specific viewpoint language that optimizes the vocabulary and presentation for the audience of that viewpoint. Viewpoint modeling has become an effective approach for dealing with the inherent complexity of large distributed systems. Architecture description practices, as described in IEEE Std 1471-2000, utilize multiple views to address several areas of concerns, each one focusing on a specific aspect of the system. Examples of architecture frameworks using multiple views include Kruchten's "4+1" view model, the Zachman Framework, TOGAF, DoDAF, and RM-ODP. == History == In the 1970s, methods began to appear in software engineering for modeling with multiple views. Douglas T. Ross and K.E. Schoman in 1977 introduce the constructs context, viewpoint, and vantage point to organize the modeling process in systems requirements definition. According to Ross and Schoman, a viewpoint "makes clear what aspects are considered relevant to achieving ... the overall purpose [of the model]" and determines How do we look at [a subject being modelled]? As examples of viewpoints, the paper offers: Technical, Operational and Economic viewpoints. In 1992, Anthony Finkelstein and others published a very important paper on viewpoints. In that work: "A viewpoint can be thought of as a combination of the idea of an “actor”, “knowledge source”, “role” or “agent” in the development process and the idea of a “view” or “perspective” which an actor maintains." An important idea in this paper was to distinguish "a representation style, the scheme and notation by which the viewpoint expresses what it can see" and "a specification, the statements expressed in the viewpoint's style describing particular domains". Subsequent work, such as IEEE 1471, preserved this distinction by utilizing two separate terms: viewpoint and view, respectively. Since the early 1990s there have been a number of efforts to codify approaches for describing and analyzing system architectures. These are often termed architecture frameworks or sometimes viewpoint sets. Many of these have been funded by the United States Department of Defense, but some have sprung from international or national efforts in ISO or the IEEE. Among these, the IEEE Recommended Practice for Architectural Description of Software-Intensive Systems (IEEE Std 1471-2000) established useful definitions of view, viewpoint, stakeholder and concern and guidelines for documenting a system architecture through the use of multiple views by applying viewpoints to address stakeholder concerns. The advantage of multiple views is that hidden requirements and stakeholder disagreements can be discovered more readily. However, studies show that in practice, the added complexity of reconciling multiple views can undermine this advantage. IEEE 1471 (now ISO/IEC/IEEE 42010:2011, Systems and software engineering — Architecture description) prescribes the contents of architecture descriptions and describes their creation and use under a number of scenarios, including precedented and unprecedented design, evolutionary design, and capture of design of existing systems. In all of these scenarios the overall process is the same: identify stakeholders, elicit concerns, identify a set of viewpoints to be used, and then apply these viewpoint specifications to develop the set of views relevant to the system of interest. Rather than define a particular set of viewpoints, the standard provides uniform mechanisms and requirements for architects and organizations to define their own viewpoints. In 1996 the ISO Reference Model for Open Distributed Processing (RM-ODP) was published to provide a useful framework for describing the architecture and design of large-scale distributed systems. == View model topics == === View === A view of a system is a representation of the system from the perspective of a viewpoint. This viewpoint on a system involves a perspective focusing on specific concerns regarding the system, which suppresses details to provide a simplified model having only those elements related to the concerns of the viewpoint. For example, a security viewpoint focuses on security concerns and a security viewpoint model contains those elements that are related to security from a more general model of a system. A view allows a user to examine a portion of a particular interest area. For example, an Information View may present all functions, organizations, technology, etc. that use a particular piece of information, while the Organizational View may present all functions, technology, and information of concern to a particular organization. In the Zachman Framework views comprise a group of work products whose development requires a particular analytical and technical expertise because they focus on either the “what,” “how,” “who,” “where,” “when,” or “why” of the enterprise. For example, Functional View work products answer the question “how is the mission carried out?” They are most easily developed by experts in functional decomposition using process and activity modeling. They show the enterprise from the point of view of functions. They also may show organizational and information components, but only as they relate to functions. === Viewpoints === In systems engineering, a viewpoint is a partitioning or restriction of concerns in a system. Adoption of a viewpoint is usable so that issues in those aspects can be addressed separately. A good selection of viewpoints also partitions the design of the system into specific areas of expertise. Viewpoints provide the conventions, rules, and languages for constructing, presenting and analysing views. In ISO/IEC 42010:2007 (IEEE-Std-1471-2000) a viewpoint is a specification for an individual view. A view is a representation of a whole system from the perspective of a viewpoint. A view may consist of one or more architectural models. Each such architectural model is developed using the methods established by its associated architectural system, as well as for the system as a whole. === Modeling perspectives === Modeling perspectives is a set of different ways to represent pre-selected aspects of a system. Each perspective has a different focus, conceptualization, dedication and visualization of what the model is representing. In information systems, the traditional way to divide modeling perspectives is to distinguish the structural, functional and behavioral/processual perspectives. This together with rule, object, communication and actor and role perspectives is one way of classifying modeling approaches === Viewpoint model === In any given viewpoint, it is possible to make a model of the system that contains only the objects that are visible from that viewpoint, but also captures all of the objects, relationships and constraints that are present in the system and relevant to that viewpoint. Such a model is said to be a viewpoint model, or a view of the
Freddy II
Freddy (1969–1971) and Freddy II (1973–1976) were experimental robots built in the Department of Machine Intelligence and Perception (later Department of Artificial Intelligence, now part of the School of Informatics at the University of Edinburgh). == Technology == Technical innovations involving Freddy were at the forefront of the 70s robotics field. Freddy was one of the earliest robots to integrate vision, manipulation and intelligent systems as well as having versatility in the system and ease in retraining and reprogramming for new tasks. The idea of moving the table instead of the arm simplified the construction. Freddy also used a method of recognising the parts visually by using graph matching on the detected features. The system used an innovative collection of high level procedures for programming the arm movements which could be reused for each new task. == Lighthill controversy == In the mid 1970s there was controversy about the utility of pursuing a general purpose robotics programme in both the USA and the UK. A BBC TV programme in 1973, referred to as the "Lighthill Debate", pitched James Lighthill, who had written a critical report for the science and engineering research funding agencies in the UK, against Donald Michie from the University of Edinburgh and John McCarthy from Stanford University. The Edinburgh Freddy II and Stanford/SRI Shakey robots were used to illustrate the state-of-the-art at the time in intelligent robotics systems. == Freddy I and II == Freddy Mark I (1969–1971) was an experimental prototype, with 3 degrees-of-freedom created by a rotating platform driven by a pair of independent wheels. The other main components were a video camera and bump sensors connected to a computer. The computer moved the platform so that the camera could see and then recognise the objects. Freddy II (1973–1976) was a 5 degrees of freedom manipulator with a large vertical 'hand' that could move up and down, rotate about the vertical axis and rotate objects held in its gripper around one horizontal axis. Two remaining translational degrees of freedom were generated by a work surface that moved beneath the gripper. The gripper was a two finger pinch gripper. A video camera was added as well as later a light stripe generator. The Freddy and Freddy II projects were initiated and overseen by Donald Michie. The mechanical hardware and analogue electronics were designed and built by Stephen Salter (who also pioneered renewable energy from waves (see Salter's Duck)), and the digital electronics and computer interfacing were designed by Harry Barrow and Gregan Crawford. The software was developed by a team led by Rod Burstall, Robin Popplestone and Harry Barrow which used the POP-2 programming language, one of the world's first functional programming languages. The computing hardware was an Elliot 4130 computer with 384KB (128K 24-bit words) RAM and a hard disk linked to a small Honeywell H316 computer with 16KB of RAM which directly performed sensing and control. Freddy was a versatile system which could be trained and reprogrammed to perform a new task in a day or two. The tasks included putting rings on pegs and assembling simple model toys consisting of wooden blocks of different shapes, a boat with a mast and a car with axles and wheels. Information about part locations was obtained using the video camera, and then matched to previously stored models of the parts. It was soon realised in the Freddy project that the 'move here, do this, move there' style of robot behavior programming (actuator or joint level programming) is tedious and also did not allow for the robot to cope with variations in part position, part shape and sensor noise. Consequently, the RAPT robot programming language was developed by Pat Ambler and Robin Popplestone, in which robot behavior was specified at the object level. This meant that robot goals were specified in terms of desired position relationships between the robot, objects and the scene, leaving the details of how to achieve the goals to the underlying software system. Although developed in the 1970s RAPT is still considerably more advanced than most commercial robot programming languages. The team of people who contributed to the project were leaders in the field at the time and included Pat Ambler, Harry Barrow, Ilona Bellos, Chris Brown, Rod Burstall, Gregan Crawford, Jim Howe, Donald Michie, Robin Popplestone, Stephen Salter, Austin Tate and Ken Turner. Also of interest in the project was the use of a structured-light 3D scanner to obtain the 3D shape and position of the parts being manipulated. The Freddy II robot is currently on display at the Royal Museum in Edinburgh, Scotland, with a segment of the assembly video shown in a continuous loop.
REEM
REEM is a prototype humanoid robot built by PAL Robotics in Spain. It is a 1.70 m high humanoid robot with 22 degrees of freedom, with a mobile base with wheels, allowing it to move at 4 km/hour. The upper part of the robot consists of a torso with a touch screen, two motorized arms, which give it a high degree of expression, and a head, which is also motorized. REEM-A and REEM-B are the first and second prototypes of humanoid robots created by PAL Robotics. REEM-B can recognize, grasp and lift objects and walk by itself, avoiding obstacles through simultaneous localization and mapping. The robot accepts voice commands and can recognize faces. == Specifications ==
Vivid knowledge
Vivid knowledge refers to a specific kind of knowledge representation. The idea of a vivid knowledge base is to get an interpretation mostly straightforward out of it – it implies the interpretation. Thus, any query to such a knowledge base can be reduced to a database-like query. == Propositional knowledge base == A propositional knowledge base KB is vivid iff KB is a complete and consistent set of literals (over some vocabulary). Such a knowledge base has the property that it as exactly one interpretation, i.e. the interpretation is unique. A check for entailment of a sentence can simply be broken down into its literals and those can be answered by a simple database-like check of KB. == First-order knowledge base == A first-order knowledge base KB is vivid iff for some finite set of positive function-free ground literals KB+, KB = KB+ ∪ Negations ∪ DomainClosure ∪ UniqueNames, whereby Negations ≔ { ¬p | p is atomic and KB ⊭ p }, DomainClosure ≔ { (ci ≠ cj) | ci, cj are distinct constants }, UniqueNames ≔ { ∀x: (x = c1) ∨ (x = c2) ∨ ..., where the ci are all the constants in KB+ }. All interpretations of a vivid first-order knowledge base are isomorphic.
Oversampled binary image sensor
An oversampled binary image sensor is an image sensor with non-linear response capabilities reminiscent of traditional photographic film. Each pixel in the sensor has a binary response, giving only a one-bit quantized measurement of the local light intensity. The response function of the image sensor is non-linear and similar to a logarithmic function, which makes the sensor suitable for high dynamic range imaging. == Working principle == Before the advent of digital image sensors, photography, for the most part of its history, used film to record light information. At the heart of every photographic film are a large number of light-sensitive grains of silver-halide crystals. During exposure, each micron-sized grain has a binary fate: Either it is struck by some incident photons and becomes "exposed", or it is missed by the photon bombardment and remains "unexposed". In the subsequent film development process, exposed grains, due to their altered chemical properties, are converted to silver metal, contributing to opaque spots on the film; unexposed grains are washed away in a chemical bath, leaving behind the transparent regions on the film. Thus, in essence, photographic film is a binary imaging medium, using local densities of opaque silver grains to encode the original light intensity information. Thanks to the small size and large number of these grains, one hardly notices this quantized nature of film when viewing it at a distance, observing only a continuous gray tone. The oversampled binary image sensor is reminiscent of photographic film. Each pixel in the sensor has a binary response, giving only a one-bit quantized measurement of the local light intensity. At the start of the exposure period, all pixels are set to 0. A pixel is then set to 1 if the number of photons reaching it during the exposure is at least equal to a given threshold q. One way to build such binary sensors is to modify standard memory chip technology, where each memory bit cell is designed to be sensitive to visible light. With current CMOS technology, the level of integration of such systems can exceed 109~1010 (i.e., 1 giga to 10 giga) pixels per chip. In this case, the corresponding pixel sizes (around 50~nm ) are far below the diffraction limit of light, and thus the image sensor is oversampling the optical resolution of the light field. Intuitively, one can exploit this spatial redundancy to compensate for the information loss due to one-bit quantizations, as is classic in oversampling delta-sigma converters. Building a binary sensor that emulates the photographic film process was first envisioned by Fossum, who coined the name digital film sensor (now referred to as a quanta image sensor). The original motivation was mainly out of technical necessity. The miniaturization of camera systems calls for the continuous shrinking of pixel sizes. At a certain point, however, the limited full-well capacity (i.e., the maximum photon-electrons a pixel can hold) of small pixels becomes a bottleneck, yielding very low signal-to-noise ratios (SNRs) and poor dynamic ranges. In contrast, a binary sensor whose pixels need to detect only a few photon-electrons around a small threshold q has much less requirement for full-well capacities, allowing pixel sizes to shrink further. == Imaging model == === Lens === Consider a simplified camera model shown in Fig.1. The λ 0 ( x ) {\displaystyle \lambda _{0}(x)} is the incoming light intensity field. By assuming that light intensities remain constant within a short exposure period, the field can be modeled as only a function of the spatial variable x {\displaystyle x} . After passing through the optical system, the original light field λ 0 ( x ) {\displaystyle \lambda _{0}(x)} gets filtered by the lens, which acts like a linear system with a given impulse response. Due to imperfections (e.g., aberrations) in the lens, the impulse response, a.k.a. the point spread function (PSF) of the optical system, cannot be a Dirac delta, thus, imposing a limit on the resolution of the observable light field. However, a more fundamental physical limit is due to light diffraction. As a result, even if the lens is ideal, the PSF is still unavoidably a small blurry spot. In optics, such diffraction-limited spot is often called the Airy disk, whose radius R a {\displaystyle R_{a}} can be computed as R a = 1.22 w f , {\displaystyle R_{a}=1.22\,wf,} where w {\displaystyle w} is the wavelength of the light and f {\displaystyle f} is the F-number of the optical system. Due to the lowpass (smoothing) nature of the PSF, the resulting λ ( x ) {\displaystyle \lambda (x)} has a finite spatial-resolution, i.e., it has a finite number of degrees of freedom per unit space. === Sensor === Fig.2 illustrates the binary sensor model. The s m {\displaystyle s_{m}} denote the exposure values accumulated by the sensor pixels. Depending on the local values of s m {\displaystyle s_{m}} , each pixel (depicted as "buckets" in the figure) collects a different number of photons hitting on its surface. y m {\displaystyle y_{m}} is the number of photons impinging on the surface of the m {\displaystyle m} th pixel during an exposure period. The relation between s m {\displaystyle s_{m}} and the photon count y m {\displaystyle y_{m}} is stochastic. More specifically, y m {\displaystyle y_{m}} can be modeled as realizations of a Poisson random variable, whose intensity parameter is equal to s m {\displaystyle s_{m}} , As a photosensitive device, each pixel in the image sensor converts photons to electrical signals, whose amplitude is proportional to the number of photons impinging on that pixel. In a conventional sensor design, the analog electrical signals are then quantized by an A/D converter into 8 to 14 bits (usually the more bits the better). But in the binary sensor, the quantizer is 1 bit. In Fig.2, b m {\displaystyle b_{m}} is the quantized output of the m {\displaystyle m} th pixel. Since the photon counts y m {\displaystyle y_{m}} are drawn from random variables, so are the binary sensor output b m {\displaystyle b_{m}} . === Spatial and temporal oversampling === If it is allowed to have temporal oversampling, i.e., taking multiple consecutive and independent frames without changing the total exposure time τ {\displaystyle \tau } , the performance of the binary sensor is equivalent to the sensor with same number of spatial oversampling under certain condition. It means that people can make trade off between spatial oversampling and temporal oversampling. This is quite important, since technology usually gives limitation on the size of the pixels and the exposure time. == Advantages over traditional sensors == Due to the limited full-well capacity of conventional image pixel, the pixel will saturate when the light intensity is too strong. This is the reason that the dynamic range of the pixel is low. For the oversampled binary image sensor, the dynamic range is not defined for a single pixel, but a group of pixels, which makes the dynamic range high. == Reconstruction == One of the most important challenges with the use of an oversampled binary image sensor is the reconstruction of the light intensity λ ( x ) {\displaystyle \lambda (x)} from the binary measurement b m {\displaystyle b_{m}} . Maximum likelihood estimation can be used for solving this problem. Fig. 4 shows the results of reconstructing the light intensity from 4096 binary images taken by single photon avalanche diodes (SPADs) camera. A better reconstruction quality with fewer temporal measurements and faster, hardware friendly implementation, can be achieved by more sophisticated algorithms.
Illia Polosukhin
Illia Polosukhin is a Ukrainian-born computer scientist and entrepreneur known for his work on the transformer architecture in machine learning and for co-founding the NEAR blockchain. == Early life and education == Polosukhin studied at the Kharkiv Polytechnic Institute, later relocating to San Diego and then moving to Silicon Valley. == Career == === Google and transformer research === Polosukhin worked at Google and was part of the team associated with research on self-attention that culminated in the 2017 paper Attention Is All You Need, widely credited with introducing the transformer architecture used in modern large language models. === NEAR Protocol === After his work in machine learning, Polosukhin became a co-founder of NEAR Protocol and later associated with the NEAR Foundation ecosystem. In 2023, Polosukhin publicly argued that increasingly capable A.I. systems should be more transparent and user-controlled, and expressed skepticism that conventional regulation alone would solve problems created by closed, corporate models, warning about risks such as regulatory capture. He has promoted “user-owned AI” concepts that combine open approaches with decentralized infrastructure aligned with the blockchain technology. In 2024, Polosukhin downplayed scenarios of A.I. independently causing human extinction, arguing that conflicts are driven by people and that misuse of AI would reflect human intent and incentives. Later this year, Polosukhin said the NEAR Foundation would reduce its workforce by about 40%. == Publications == Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Lukasz Kaiser, Illia Polosukhin; et al. (2017). "Attention Is All You Need". arXiv.{{cite journal}}: CS1 maint: multiple names: authors list (link)