Encyclopaedistics or encyclopaedics as a discipline, is the academic scholarship of encyclopedias as sources of encyclopedic knowledge and cultural objects as well; in this sense, this discipline is also known as "encyclopaedia studies" and can be termed as "theoretical encyclopaediography" by analogy with theoretical lexicography. Encyclopaedistics as a practical activity (profession or business) also called "encyclopaedic practice" or "encyclopedism" is the process of assembling encyclopaedias available to the public for sale or for free (encyclopaedia publishing or practical encyclopediography). In this sense, it is the art or craft of writing, compiling, and editing the paper or online encyclopedias. As a practical activity, encyclopaedistics originated in the Middle Ages in connection with the development of compendiums based on alphabetical structuring (e.g. first edition of Polyanthea by Dominicus Nanus Mirabellius). Encyclopaedistics is often defined as "the art and science of selecting and disseminating the information most significant to mankind". == Field of study == Encyclopaedistics is a specialized aspect of information science and communication science. At the same time, encyclopaedistics is also considered as one of scholarly disciplines which are seen as auxiliary for historical research (auxiliary sciences of history) . Third, encyclopaedics is a domain of philosophy (Romanticism). This term associated with German philosophers of the 18th century, such as Novalis, Friedrich Schlegel, who sought to create a "Scientific Bible" - both real and ideal book as the quintessence of human education (enlightenment). In any case, the most popular topics in encyclopaedia studies refferd the history of organization of encyclopaedic knowledge, encyclopaedic knowledge determination and selection, glossary composition, current state of development of encyclopaedic activity, features of making encyclopaedias and encyclopaedic articles, usage, role and significance of encyclopaedias, typology of encyclopaedic literature, encyclopaedists and encyclopaedic schools, opposition of classical encyclopaedias and Wikipedia as well as paper encyclopaedias and online encyclopaedias, case experience in building encyclopedias etc. In general, scholarly studies contribute to appearance of successful well-crafted encyclopaedias with high-quality articles. == Contemporary encyclopaedic practice == Today, academic institutions, universities, and publishing companies worldwide are engaged in encyclopaedic activity building national, multinational (universal), regional and subject-specific encyclopaedias, or doing studies related encyclopaedias. The development of national encyclopaedias is one of the prerogatives of the European Parliament in the policy of protection of accurate and verified information and in the fight against mis- and disinformation as well as in the policy of protecting, promoting and projecting Europe's values and interests in the world.
CHAOS (chess)
CHAOS (Chess Heuristics and Other Stuff) is a chess playing program that was developed by programmers working at the RCA Systems Programming division in the late 1960s. It played competitively in computer chess competitions in the 1970s and 1980s. It differed from other programs of that era in its look-ahead philosophy, choosing to use chess knowledge to evaluate fewer positions and continuations as opposed to simple evaluations that relied on deep look-ahead to avoid bad moves. == Introduction == CHAOS was originally developed by Ira Ruben, Fred Swartz, Victor Berman, Joe Winograd and William Toikka while working at RCA in Cinnaminson, NJ. Its name is an acronym for 'Chess Heuristics and Other Stuff.' Program development moved to the Computing Center of the University of Michigan when Swartz changed jobs, and Mike Alexander joined the development group. Swartz, Alexander and Berman were continuously group members from that point onward in CHAOS' evolution, as others of the original authors left and new members contributed episodically. Chess Senior Master Jack O'Keefe contributed to CHAOS' development from about 1980 onwards. CHAOS was written in Fortran, except for low-level board representation manipulations written in assembly language or C. Due to this portability, it ran on RCA, Univac and IBM-compatible mainframes in its lifetime. CHAOS heralds from the mainframe computing era when only machines of that capacity were able to play at a high level. Consequently, development and testing could only take place at off-peak times for production use of the machine. In a competition, CHAOS had to run on a dedicated mainframe with a telephone link to the match venue. In its later years, CHAOS ran on computers on the machine assembly floor of Amdahl Corporation on MTS. == Background == === Chess and artificial intelligence === Mathematicians Claude Shannon and Alan Turing, working separately, were the first to view playing chess as a challenge to machines. Working for AT&T / Bell Labs with its access to telephone switching equipment, Shannon built a relay-based machine that learned how to work its way through a two-dimensional, 5x5 cell maze in 1949. Shannon viewed this as an analogue of the way that organisms learn things about their natural environment. There is a random element to searching it, a memory element to benefit from the search outcome, and a reward element that reinforces learning when the global outcome is favorable to the organism. Soon afterward, Shannon wrote a mathematical analysis of the game of chess, published in 1950. Like with the maze, he broke down game play into the necessary elements for reinforcement learning. Associated with each board configuration a move will be made from, there is a numerical score. To decide what move to make, a player wants to maximize their own position's score after the move and to minimize their opponent's score (a minimax view). Since there are about 32 possible moves at each of the early stages of the game, and about 40 moves and responses in each game, then there are about 32 80 {\displaystyle 32^{80}} or about 10 120 {\displaystyle 10^{120}} possible games - an impossibly large set to evaluate completely. Therefore, there must be a way to limit the number of moves to look ahead for to find the best one. Reducing the game to these few key elements provided a way to think about human intelligence in general. Shannon became part of a wider group using computing machines to mimic aspects of human intelligence that grew into the general idea of artificial intelligence. (Other members of this group were John McCarthy, Herbert Simon, Allen Newell, Alan Kotok, Alex Bernstein and Richard Greenblatt.) The paradigm that evolved was that there was a quantification of the position on the board into a score, an evaluation method to find favorable outcomes (minimax, later alpha-beta pruning), and a strategy to manage the combinatorial explosion of the look-ahead possibilities. By the early 1960s, there were computer programs that played chess at a rudimentary level. They used very simple evaluation functions for each position and tried to search as far forward as was practical given the time constraints and available compute power. Naturally, programmers optimized their code to use the available computing resources. This led to a major philosophical divide among chess programs: those that tried to evaluate as many positions as possible, and those that tried to evaluate the most promising move sequences as deeply as possible. CHAOS was firmly in the camp believing only the most promising moves should be evaluated in depth. Said Swartz, "The 'brute force people' ... look at every (possible move) no matter what garbage it is. Most moves are just terrible, terrible moves, and most computing time is being spent on pure garbage." The program spent more time evaluating each board position in the expectation that it would find the most promising lines of play to explore in depth. In 1983, the then-fastest chess program (Belle) evaluated 110,000 positions per second, and typical programs 1000–50,000 per second, whereas CHAOS evaluated about 50-100 per second. === Machine learning and strategies to manage search === From about 1949 onward, Arthur Samuel began work for IBM on machine learning, culminating in a checkers-playing program in 1952 and publications on the topic. Concurrently, Christopher Strachey created Checkers, a program to play the board game of checkers in 1951, but it had no capacity to learn from its play. Checkers was chosen by both authors because it was simpler than chess yet contained the basic characteristics of an intellectual activity, and, in Samuel's view, was a test-bed in which heuristic procedures and learning processes could be evaluated quickly. Checker playing programs introduced the notion of the game tree and evaluating play to various depths to choose the best move. The complexity of chess, however, promoted it to the status of an analogue for human intelligence, and it attracted computer scientists' attention, who referred to it as research into artificial intelligence (AI). Like checkers, it required a numerical assessment of each arrangement of chess pieces on a board. It also required looking ahead to future moves to decide how to play the present position. Due to the enormous number of possible moves, there had to be a way to confine the look-ahead search to the most promising lines of play. From these factors, the notion of minimax score evaluation developed and, later, alpha-beta tree pruning to abandon looking at positions worse than any that have already been examined. === Chess search strategies === The AI community viewed artificial intelligence as comprising two parts: a way to symbolically quantify the knowledge in hand (a chess board position), and a set of heuristics to limit look-ahead to the consequences of a move. The early chess playing programs attempted to look forward as far as possible, perhaps to 3 moves ahead by each player, and to choose the best outcome. This led to the horizon effect, whereby a key move 4 or more moves ahead would be unexamined and therefore missed. Consequently, the programs were quite weak and heuristics to manage the search became important in their development. CHAOS used a selective search strategy with iterative widening. As chess programs evolved, they incorporated books of opening lines of play from historic sources. Nowadays, book moves are catalogued in machine-readable form, but originally programmers had to type them in. CHAOS had an extensive book for its time of around 10,000 moves that O'Keefe helped to develop. A problem with play from an opening book is the behavior of the program when the play leaves the book: the positional advantage may be so subtle that the evaluation scheme may be unable to understand it, leading to very wide and shallow searches to establish a line of play. The horizon effect again plagues move selection after leaving the book. CHAOS mitigated these problems by only using book lines that it could understand, and by relying on cached analyses of continuations out of the book made while the opponent's clock was running. == Game Play History == CHAOS played in twelve ACM computer chess tournaments and four World Computer Chess Championships (WCCC). Its debut was the ACM computer chess tournament in 1973, taking 2nd place. In 1974, it again won 2nd place in the WCCC, defeating the tournament favorite Chess 4.0 but losing to Kaissa. CHAOS was close to winning the 1980 WCCC, but lost to Belle in a playoff. The 1985 ACM computer chess tournament was CHAOS' last competition. One of CHAOS' notable victories was over Chess 4.0 at the 1974 WCCC tournament. Chess 4.0 was unbeaten by any other program up until then. Playing as white, CHAOS made a knight sacrifice (16 Nd4-e6!!) that traded material for open lines of attack and eventually won the game. CHAOS’ authors thought the move was due to a
ActivityPub
ActivityPub is a protocol and open standard for decentralized social networking. It provides a client-to-server (C2S) API for creating and modifying content, as well as a federated server-to-server (S2S) protocol for delivering notifications and content to other servers. ActivityPub is the defining standard of the Fediverse, a decentralised social network of various social interaction models, and content types, which consists of independently managed instances of software such as Mastodon, Pixelfed and PeerTube, among others. ActivityPub is considered to be an update to the ActivityPump protocol used in pump.io, and the official W3C repository for ActivityPub is identified as a fork of ActivityPump. The creation of a new standard for decentralized social networking was prompted by the complexity of OStatus, the most commonly used protocol at the time. OStatus was built using a multitude of technologies (such as Atom, Salmon, WebSub and WebFinger), a product of the infrastructure used in GNU social (the originator and largest user of the OStatus protocol), which made it difficult to implement the protocol into new software. OStatus was also only designed to work with microblogging services, with little flexibility to the types of data that it could hold. The standard was first published by the World Wide Web Consortium (W3C) as a W3C Recommendation in January 2018 by the Social Web Working Group (SocialWG), a working group chartered to build the protocols and vocabularies needed to create a standard for social functionality. Shortly after, further development was moved to the Social Web Community Group (SocialCG), the successor to the SocialWG. == Design == ActivityPub uses the ActivityStreams 2.0 format for building its content, which itself uses JSON-LD. The three main data types used in ActivityPub are Objects, Activities and Actors. Objects are the most common data type, and can be images, videos, or more abstract items such as locations or events. Activities are actions that create and modify objects, for example a Create activity creates an object. Actors are representative of an individual, a group, an application or a service, and are the owners of objects. Every actor type contains an inbox and outbox stream, which sends and receives activities for a user. In order to publish data (for example liking an article), a user creates an activity that declares that they liked an Article object and publishes it to their outbox, where it is then delivered by the ActivityPub server via a POST request to the inboxes listed in the activity's to, bto, cc and bcc fields. The receiving servers then account for the newly received activity and update the article by adding the like action to it. === Example data === An example actor object that represents a user account: An example activity that likes an article object: An example article object: == Project status == The SocialCG previously organized a yearly free conference called ActivityPub Conf about the future of ActivityPub. Triages are held regularly to review issues pertaining to the ActivityPub and ActivityStreams 2.0 specifications as part of the SocialCG. In 2023, Germany's Sovereign Tech Fund donated €152,000 to socialweb.coop with the goal of building a new suite for testing various ActivityPub implementations and their compliance with the specification. === Adoption === The initial wave of adoption for ActivityPub (circa 2016–2018) came from software that was already using OStatus as their federation protocol, such as Mastodon, GNU social and Pleroma. Following the acquisition of Twitter by Elon Musk in 2022, many groups of users that were critical of the acquisition migrated to Mastodon, bringing new attention to the ActivityPub protocol with it. Various major social media platforms and corporations have since pledged to implement ActivityPub support, including Tumblr, Flipboard and Meta Platforms' Threads. Threads introduced crossposting to ActivityPub in 2024 for users outside of the European Economic Area, however full 2-way compatibility remains incomplete as of 2025. == Criticism == === Accidental denial-of-service attacks === Poorly optimized ActivityPub implementations can cause unintentional distributed denial-of-service (DDOS) attacks on other websites and servers, due to the decentralized nature of the network. An example would be Mastodon's implementation of OpenGraph link previews, wherein every instance that receives a post that contains a link with OpenGraph metadata will download the associated data, such as a thumbnail, in a very short timeframe, which can slow down or crash servers as a result of the sudden burst of requests. === Account migration === ActivityPub has been criticized for not natively supporting moving accounts from one server to another, forcing implementations to build their own solutions. While there has been work on building a standardized system for migrating accounts using the Move activity via the Fediverse Enhancement Proposal organization, the current proposal only allows for basic follower migration, with all other data remaining linked to the original account. === Missing content and data === ActivityPub implementations have been criticized for missing replies and parts of reply threads from remote posts, and presenting outdated statistics (e.g. likes and reposts) about remote posts. However, this isn't a problem with the ActivityPub protocol itself, but with implementations not refreshing their content for updated data when needed. == Software using ActivityPub == === Future implementations === Flarum, an internet forum software Forgejo, a Git forge and development platform === Uncertain future implementations === GitLab, a Git forge and development platform which had previously had an open issue discussing the topic, but was later closed due to the development team moving focus to other areas. Tumblr, a microblogging platform. Despite previous statements from Automattic CEO Matt Mullenweg, ActivityPub integration has been delayed indefinitely. The integration would have been implemented with its WordPress migration, as the first-party plugin for interoperability would have been used for federation. Flickr, an image and video hosting site.
NationBuilder
NationBuilder is a Los Angeles-based technology start-up that develops content management and customer relationship management (CRM) software. Although the company initially targeted political campaigns and nonprofit organizations, it later expanded its marketing efforts to include other people and organizations trying to build an online following, such as artists, musicians and restaurants. The software uses voter data such as names, addresses and other information, such as previous voting records in the case of political campaigns, to allow users to centralize, build and manage campaigns by integrating various communication tools like websites, newsletters, text messaging and social media channels under one platform. Among other features, the software enables users to quickly create websites, build databases through registrations, send targeted newsletters, analyse data from multiple sources and leverage micro-donations. The software's appeal towards political campaigns comes from the combination of a number of previously separate campaigning services, channels and data sources into a single platform that was presented as a facile solution for non-technical users and which enabled political campaigners to quickly deploy campaigns by convincing numerous people to donate. == History == NationBuilder was founded in 2009 in Los Angeles by Jim Gilliam and launched in 2011. In 2012 Joe Green joined NationBuilder as co-founder and president. He left that role 11 months later in February 2013. Gilliam was previously a movie-maker who co-founded Brave New Films with Robert Greenwald and had sought funding for his films through crowd-sourcing. Green, who studied organizing at Harvard and was Mark Zuckerberg's roommate, is also the co-founder of the Causes Facebook app; he left NationBuilder in 2013. Since its founding, the company has helped campaigns raise $1.2 billion. In 2012, NationBuilder announced that 1,000 subscribers have used its software to amass 2.5 million supporters and raise $12 million in campaign donations. In 2015 it has helped raise $264 million, recruit over one million volunteers and coordinate some 129,000 events. By 2016, the company said its software was used by about 40 percent of all contested elections at the state and national level in the U.S., which included 3,000 political campaigns. Using such software is easier in the U.S. than Europe, where comprehensive data protection and privacy laws are in effect since 2018. The Scottish National Party was the first political party to use NationBuilder, harvesting vast amounts of data pertaining to voter activity via websites such as Facebook and Twitter. This revelation prompted outrage over privacy concerns. Guy Herbert of the No2ID campaign called the use of such data harvesting tools by the SNP "utterly hypocritical". == Funding == Investors in NationBuilder include Chris Hughes - the Facebook co-founder, Sean Parker - first president of Facebook and co-founder of Napster and Causes, Dan Senor - the former Republican foreign-policy adviser and Ben Horowitz, co-founder of Andreessen Horowitz. In 2012, it has raised $6.3 million in funding from a number of investors. == Notable implementations == The software is reported to have played a role in some public elections in Europe, the US and New Zealand, as well as non-profit initiatives, and political parties in Australia. Notable users include Bernie Sanders, Mitch McConnell, Andrew Yang, Theresa May, Amnesty International, the NAACP and Donald Trump. === France === La République En Marche used NationBuilder in their campaign for the 2017 National Assembly. === New Zealand === NationBuilder's services are used by New Zealand political parties, including in the campaigns of both the National and Labour parties in the 2017 general election. === United Kingdom === Despite stricter data protection and privacy laws in the UK and EU, NationBuilder was used to significant impact in a number of UK elections, most notably in the 2016 campaign for withdrawal of the United Kingdom from the European Union. The company later made a public announcement that both sides in that campaign had used its software. === United States === NationBuilder was used in the Donald Trump presidential campaign to advance his election efforts and eventually win the 2016 presidential race. Jill Stein of the Green Party, Republican Rick Santorum, and independent supporters of various candidates all used NationBuilder during their 2016 runs for president. During the 2018 US election cycle, political entities paid more than $1 million for the use of NationBuilder. Among the entities paying the most were Donald J. Trump for President, Prosperity Action and the Republican Party of Tennessee.
Closest point method
The closest point method (CPM) is an embedding method for solving partial differential equations on surfaces. The closest point method uses standard numerical approaches such as finite differences, finite element or spectral methods in order to solve the embedding partial differential equation (PDE) which is equal to the original PDE on the surface. The solution is computed in a band surrounding the surface in order to be computationally efficient. In order to extend the data off the surface, the closest point method uses a closest point representation. This representation extends function values to be constant along directions normal to the surface. == Definitions == Closest Point function: Given a surface S , c p ( x ) {\displaystyle {\mathcal {S}},cp(\mathbf {x} )} refers to a (possibly non-unique) point belonging to S {\displaystyle {\mathcal {S}}} , which is closest to x {\displaystyle \mathbf {x} } [SE]. Closest point extension: Let S {\displaystyle {\mathcal {S}}} , be a smooth surface in R d {\displaystyle \mathbb {R} ^{d}} . The closest point extension of a function u : S → R {\displaystyle u:{\mathcal {S}}\rightarrow \mathbb {R} } , to a neighborhood Ω {\displaystyle \Omega } of S {\displaystyle {\mathcal {S}}} , is the function v : Ω → R {\displaystyle v:\Omega \rightarrow \mathbb {R} } , defined by v ( x ) = u ( c p ( x ) ) {\displaystyle v(\mathbf {x} )=u(cp(\mathbf {x} ))} . == Closest point method == Initialization consists of these steps [EW]: If it is not already given, a closest point representation of the surface is constructed. A computational domain is chosen. Typically this is a band around the surface. Replace surface gradients by standard gradients in R 3 {\displaystyle \mathbb {R} ^{3}} . Solution is initialized by extending the initial surface data on to the computational domain using the closest point function. After initialization, alternate between the following two steps: Using the closest point function, extend the solution off the surface to the computational domain. Compute the solution to the embedding PDE on a Cartesian mesh in the computational domain for one time step. == Banding == The surface PDE is extended into R 3 {\displaystyle \mathbb {R} ^{3}} however it is only necessary to solve this new PDE near the surface. Hence, we solve the PDE in a band surrounding the surface for efficient computational purposes. Ω c x : ‖ x − c p ( x ) ‖ 2 ≤ λ {\displaystyle \Omega _{c}{x:\|x-cp(x)\|_{2}\leq \lambda }} where λ {\displaystyle \lambda } is the bandwidth. == Example: Heat equation on a circle == Using initial profile u S ( θ , t ) = sin ( θ ) {\displaystyle u_{S}(\theta ,t)=\sin(\theta )} leads to the solution u S ( θ , t ) = exp ( − t ) sin ( θ ) {\displaystyle u_{S}(\theta ,t)=\exp(-t)\sin(\theta )} for the heat equation. Forward Euler time-stepping is used with relation Δ t = 0.1 Δ x 2 {\displaystyle \Delta t=0.1\Delta x^{2}} and degree-four interpolation polynomials for the interpolations. Second-order centered differences are used for the spatial discretization. The CPM results in the expected second order error in the solution u {\displaystyle u} . == Applications == The closest point method can be applied to various PDEs on surfaces. Reaction–diffusion problems on point clouds [RD], eigenvalue problems [EV], and level set equations [LS] are a few examples.
Mountain car problem
Mountain Car, a standard testing domain in Reinforcement learning, is a problem in which an under-powered car must drive up a steep hill. Since gravity is stronger than the car's engine, even at full throttle, the car cannot simply accelerate up the steep slope. The car is situated in a valley and must learn to leverage potential energy by driving up the opposite hill before the car is able to make it to the goal at the top of the rightmost hill. The domain has been used as a test bed in various reinforcement learning papers. == Introduction == The mountain car problem, although fairly simple, is commonly applied because it requires a reinforcement learning agent to learn on two continuous variables: position and velocity. For any given state (position and velocity) of the car, the agent is given the possibility of driving left, driving right, or not using the engine at all. In the standard version of the problem, the agent receives a negative reward at every time step when the goal is not reached; the agent has no information about the goal until an initial success. == History == The mountain car problem appeared first in Andrew Moore's PhD thesis (1990). It was later more strictly defined in Singh and Sutton's reinforcement learning paper with eligibility traces. The problem became more widely studied when Sutton and Barto added it to their book Reinforcement Learning: An Introduction (1998). Throughout the years many versions of the problem have been used, such as those which modify the reward function, termination condition, and the start state. == Techniques used to solve mountain car == Q-learning and similar techniques for mapping discrete states to discrete actions need to be extended to be able to deal with the continuous state space of the problem. Approaches often fall into one of two categories, state space discretization or function approximation. === Discretization === In this approach, two continuous state variables are pushed into discrete states by bucketing each continuous variable into multiple discrete states. This approach works with properly tuned parameters but a disadvantage is information gathered from one state is not used to evaluate another state. Tile coding can be used to improve discretization and involves continuous variables mapping into sets of buckets offset from one another. Each step of training has a wider impact on the value function approximation because when the offset grids are summed, the information is diffused. === Function approximation === Function approximation is another way to solve the mountain car. By choosing a set of basis functions beforehand, or by generating them as the car drives, the agent can approximate the value function at each state. Unlike the step-wise version of the value function created with discretization, function approximation can more cleanly estimate the true smooth function of the mountain car domain. === Eligibility traces === One aspect of the problem involves the delay of actual reward. The agent is not able to learn about the goal until a successful completion. Given a naive approach for each trial the car can only backup the reward of the goal slightly. This is a problem for naive discretization because each discrete state will only be backed up once, taking a larger number of episodes to learn the problem. This problem can be alleviated via the mechanism of eligibility traces, which will automatically backup the reward given to states before, dramatically increasing the speed of learning. Eligibility traces can be viewed as a bridge from temporal difference learning methods to Monte Carlo methods. == Technical details == The mountain car problem has undergone many iterations. This section focuses on the standard well-defined version from Sutton (2008). === State variables === Two-dimensional continuous state space. V e l o c i t y = ( − 0.07 , 0.07 ) {\displaystyle Velocity=(-0.07,0.07)} P o s i t i o n = ( − 1.2 , 0.6 ) {\displaystyle Position=(-1.2,0.6)} === Actions === One-dimensional discrete action space. m o t o r = ( l e f t , n e u t r a l , r i g h t ) {\displaystyle motor=(left,neutral,right)} === Reward === For every time step: r e w a r d = − 1 {\displaystyle reward=-1} === Update function === For every time step: A c t i o n = [ − 1 , 0 , 1 ] {\displaystyle Action=[-1,0,1]} V e l o c i t y = V e l o c i t y + ( A c t i o n ) ∗ 0.001 + cos ( 3 ∗ P o s i t i o n ) ∗ ( − 0.0025 ) {\displaystyle Velocity=Velocity+(Action)0.001+\cos(3Position)(-0.0025)} P o s i t i o n = P o s i t i o n + V e l o c i t y {\displaystyle Position=Position+Velocity} === Starting condition === Optionally, many implementations include randomness in both parameters to show better generalized learning. P o s i t i o n = − 0.5 {\displaystyle Position=-0.5} V e l o c i t y = 0.0 {\displaystyle Velocity=0.0} === Termination condition === End the simulation when: P o s i t i o n ≥ 0.6 {\displaystyle Position\geq 0.6} == Variations == There are many versions of the mountain car which deviate in different ways from the standard model. Variables that vary include but are not limited to changing the constants (gravity and steepness) of the problem so specific tuning for specific policies become irrelevant and altering the reward function to affect the agent's ability to learn in a different manner. An example is changing the reward to be equal to the distance from the goal, or changing the reward to zero everywhere and one at the goal. Additionally, a 3D mountain car can be used, with a 4D continuous state space.
Digital image
A digital image is an image composed of picture elements, also known as pixels, each with finite, discrete quantities of numeric representation for its intensity or gray level that is an output from its two-dimensional functions fed as input by its spatial coordinates denoted with x, y on the x-axis and y-axis, respectively. An image can be vector or raster type. By itself, the term "digital image" usually refers to raster images or bitmapped images (as opposed to vector images). == Raster == Raster images have a finite set of digital values, called picture elements or pixels. The digital image contains a fixed number of rows and columns of pixels. Pixels are the smallest individual element in an image, holding quantized values that represent the brightness of a given color at any specific point. Typically, the pixels are stored in computer memory as a raster image or raster map, a two-dimensional array of small integers. These values are often transmitted or stored in a compressed form. Raster images can be created by a variety of input devices and techniques, such as digital cameras, scanners, coordinate-measuring machines, seismographic profiling, airborne radar, and more. They can also be synthesized from arbitrary non-image data, such as mathematical functions or three-dimensional geometric models; the latter being a major sub-area of computer graphics. The field of digital image processing is the study of algorithms for their transformation. === Raster file formats === Most users come into contact with raster images through digital cameras, which use any of several image file formats. Some digital cameras give access to almost all the data captured by the camera, using a raw image format. The Universal Photographic Imaging Guidelines (UPDIG) suggests these formats be used when possible since raw files produce the best quality images. These file formats allow the photographer and the processing agent the greatest level of control and accuracy for output. Their use is inhibited by the prevalence of proprietary information (trade secrets) for some camera makers, but there have been initiatives such as OpenRAW to influence manufacturers to release these records publicly. An alternative may be Digital Negative (DNG), a proprietary Adobe product described as "the public, archival format for digital camera raw data". Although this format is not yet universally accepted, support for the product is growing, and increasingly professional archivists and conservationists, working for respectable organizations, variously suggest or recommend DNG for archival purposes. == Vector == Vector images resulted from mathematical geometry (vector). In mathematical terms, a vector consists of both a magnitude, or length, and a direction. Often, both raster and vector elements will be combined in one image; for example, in the case of a billboard with text (vector) and photographs (raster). Example of vector file types are EPS, PDF, and AI. == Image viewing == Image viewer software displayed on images. Web browsers can display standard internet images formats including JPEG, GIF and PNG. Some can show SVG format which is a standard W3C format. In the past, when the Internet was still slow, it was common to provide "preview" images that would load and appear on the website before being replaced by the main image (to give a preliminary impression). Now Internet is fast enough and this preview image is seldom used. Some scientific images can be very large (for instance, the 46 gigapixel size image of the Milky Way, about 194 GB in size). Such images are difficult to download and are usually browsed online through more complex web interfaces. Some viewers offer a slideshow utility to display a sequence of images. == History == Early digital fax machines such as the Bartlane cable picture transmission system preceded digital cameras and computers by decades. The first picture to be scanned, stored, and recreated in digital pixels was displayed on the Standards Eastern Automatic Computer (SEAC) at NIST. The advancement of digital imagery continued in the early 1960s, alongside development of the space program and in medical research. Projects at the Jet Propulsion Laboratory, MIT, Bell Labs and the University of Maryland, among others, used digital images to advance satellite imagery, wirephoto standards conversion, medical imaging, videophone technology, character recognition, and photo enhancement. Rapid advances in digital imaging began with the introduction of MOS integrated circuits in the 1960s and microprocessors in the early 1970s, alongside progress in related computer memory storage, display technologies, and data compression algorithms. The invention of computerized axial tomography (CAT scanning), using x-rays to produce a digital image of a "slice" through a three-dimensional object, was of great importance to medical diagnostics. As well as origination of digital images, digitization of analog images allowed the enhancement and restoration of archaeological artifacts and began to be used in fields as diverse as nuclear medicine, astronomy, law enforcement, defence and industry. Advances in microprocessor technology paved the way for the development and marketing of charge-coupled devices (CCDs) for use in a wide range of image capture devices and gradually displaced the use of analog film and tape in photography and videography towards the end of the 20th century. The computing power necessary to process digital image capture also allowed computer-generated digital images to achieve a level of refinement close to photorealism. === Digital image sensors === The first semiconductor image sensor was the CCD, developed by Willard S. Boyle and George E. Smith at Bell Labs in 1969. While researching MOS technology, they realized that an electric charge was the analogy of the magnetic bubble and that it could be stored on a tiny MOS capacitor. As it was fairly straightforward to fabricate a series of MOS capacitors in a row, they connected a suitable voltage to them so that the charge could be stepped along from one to the next. The CCD is a semiconductor circuit that was later used in the first digital video cameras for television broadcasting. Early CCD sensors suffered from shutter lag. This was largely resolved with the invention of the pinned photodiode (PPD). It was invented by Nobukazu Teranishi, Hiromitsu Shiraki and Yasuo Ishihara at NEC in 1980. It was a photodetector structure with low lag, low noise, high quantum efficiency and low dark current. In 1987, the PPD began to be incorporated into most CCD devices, becoming a fixture in consumer electronic video cameras and then digital still cameras. Since then, the PPD has been used in nearly all CCD sensors and then CMOS sensors. The NMOS active-pixel sensor (APS) was invented by Olympus in Japan during the mid-1980s. This was enabled by advances in MOS semiconductor device fabrication, with MOSFET scaling reaching smaller micron and then sub-micron levels. The NMOS APS was fabricated by Tsutomu Nakamura's team at Olympus in 1985. The CMOS active-pixel sensor (CMOS sensor) was later developed by Eric Fossum's team at the NASA Jet Propulsion Laboratory in 1993. By 2007, sales of CMOS sensors had surpassed CCD sensors. === Digital image compression === An important development in digital image compression technology was the discrete cosine transform (DCT), a lossy compression technique first proposed by Nasir Ahmed in 1972. DCT compression is used in JPEG, which was introduced by the Joint Photographic Experts Group in 1992. JPEG compresses images down to much smaller file sizes, and has become the most widely used image file format on the Internet. == Mosaic == In digital imaging, a mosaic is a combination of non-overlapping images, arranged in some tessellation. Gigapixel images are an example of such digital image mosaics. Satellite imagery are often mosaicked to cover Earth regions. Interactive viewing is provided by virtual-reality photography.