AI Content Provenance

AI Content Provenance — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  Automatic scorer

  An automatic scorer is the computerized scoring system to keep track of scoring in ten-pin bowling. It was introduced en masse in bowling alleys in the 1970s and combined with mechanical pinsetters to detect overturned pins. By eliminating the need for manual score-keeping, these systems have introduced new bowlers into the game who otherwise would not participate because they had to count the score themselves, as many do not understand the mathematical formula involved in bowler scoring. At first, people were skeptical about whether a computer could keep an accurate score. In the twenty-first century, automatic scorers are used in most bowling centers around the world. The three manufacturers of these specialty computers have been Brunswick Bowling, AMF Bowling (later QubicaAMF), and RCA. == History == Automatic equipment is considered a cornerstone of the modern bowling center. The traditional bowling center of the early 20th century was advanced in automation when the pinsetter person ("pin boy"), who set back up by hand the bowled down pins, was replaced by a machine that automatically replaced the pins in their proper play positions. This machine came out in the 1950s. A detection system was developed from the pinsetter mechanism in the 1960s that could tell which pins had been knocked down, and that information could be transferred to a digital computer. Automatic electronic scoring was first conceived by Robert Reynolds, who was described by a newspaper story at the time as "a West Coast electronics calculator expert." He worked with the technical staff of Brunswick Bowling to develop it. The goal was realized in the late 1960s when a specialized computer was designed for the purpose of automatic scorekeeping for bowling. The field test for the automatic scorer took place at Village Lanes bowling center, Chicago in 1967. The scoring machine received approval for official use by the American Bowling Congress in August of that year. They were first used in national official league gaming on October 10, 1967. In November, Brunswick announced that they were accepting orders for the new digital computer, which cost around $3,000 per bowling lane. Bowling centers that installed these new automatic scoring devices in the 1970s charged a ten cents extra per line of scoring for the convenience. == Description == Each Automatic Scorer computer unit kept score for four lanes. It had two bowler identification panels serving two lanes each. The bowler pushed it into his named position when his turn came up so the computer knew who was bowling and score accordingly. After the bowler rolled the bowling ball down the lane and knocked down pins, the pinsetter detected which pins were down and relayed this information back to the computer for scoring. The result was then printed on a scoresheet and projected overhead onto a large screen for all to see. The Automatic Scorer digital computer was mathematically accurate, however the detection system at the pinsetter mechanism sometimes reported the wrong number of pins knocked down. The computer could be corrected manually for any errors in the system; similarly, human errors, such as neglecting to move the bowler identification mechanism, could be corrected for by manual action. The scorer could take into account bowlers' handicaps and could adjust for late-arriving bowlers. The automatic scorer is directly connected to the foul detection unit. As a result, foul line violations are automatically scored. Brunswick had put ten years of research and development into the Automatic Scorer, and by 1972 there were over 500 of these computers installed in bowling centers around the world. AMF Bowling, competitor to Brunswick, entered into the automatic scorer computer field during the 1970s and their systems were installed into their brand of bowling centers. By 1974, RCA was also making these computers for automatic scoring. == Reception and further developments == The purposes of the computerized scoring were to avoid errors by human scorers and to prevent cheating. It had the side benefit of speeding up the progress of the game and introducing new bowlers to the game. Score-keeping for bowling is based on a formula that many new to bowling were not familiar with and thought difficult to learn. These casual bowlers unfamiliar with the formula thought the scores given by the computers were confusing. Some bowlers were not comfortable with automatic scorers when they were introduced in the 1970s, so kept score using the traditional method on paper score sheets. The introduction of this device increased the popularity of the sport. Automatic scorers came to be considered a normal part of modern bowling installations worldwide, with owners and managers saying that bowlers expect such equipment to be present in bowling establishments and that business increased following their introduction. Brunswick introduced a color television style automatic scorer in 1983. Bowling center owners could use these style automatic scorers for advertising, management, videos, and live television. By the 2010s, these types of electronic visual displays could show bowler avatars and social media connections to publicize the bowlers' scores. Some are capable of being extended entertainment systems of games for children and adults. Some scoring systems support variations on traditional bowling, such as different kinds of bingo games where certain pins have to be knocked down at certain times or practice regimes where certain spares have to be accomplished. By this point, QubicaAMF Worldwide, an outgrowth of AMF, was one of the leading providers of bowling scoring equipment.

  Read more →
• 

  Minimum information standard

  Minimum information standards are sets of guidelines and formats for reporting data derived by specific high-throughput methods. Their purpose is to ensure the data generated by these methods can be easily verified, analysed and interpreted by the wider scientific community. Ultimately, they facilitate the transfer of data from journal articles (unstructured data) into databases (structured data) in a form that enables data to be mined across multiple data sets. Minimal information standards are available for a vast variety of experiment types including microarray (MIAME), RNAseq (MINSEQE), metabolomics (MSI) and proteomics (MIAPE). Minimum information standards typically have two parts. Firstly, there is a set of reporting requirements – typically presented as a table or a checklist. Secondly, there is a data format. Information about an experiment needs to be converted into the appropriate data format for it to be submitted to the relevant database. In the case of MIAME, the data format is provided in spreadsheet format (MAGE-TAB). Some of the communities that maintain minimum information standards also provide tools to help experimental researchers to annotate their data. == MI Standards == The individual minimum information standards are brought by the communities of cross-disciplinary specialists focused on the problematic of the specific method used in experimental biology. The standards then provide specifications what information about the experiments (metadata) is crucial and important to be reported together with the resultant data to make it comprehensive. The need for this standardization is largely driven by the development of high-throughput experimental methods that provide tremendous amounts of data. The development of minimum information standards of different methods is since 2008 being harmonized by "Minimum Information about a Biomedical or Biological Investigation" (MIBBI) project. === MIAPPE, Minimum Information About a Plant Phenotyping Experiment === MIAPPE is an open, community driven project to harmonize data from plant phenotyping experiments. MIAPPE comprises both a conceptual checklist of metadata required to adequately describe a plant phenotyping experiment. === MIQE, Minimum Information for Publication of Quantitative Real-Time PCR Experiments === Published in 2009 these guidelines for the basis of requirements by many journals when submitting QPCR data, sadly they are not adhered to enough. === MIAME, gene expression microarray === Minimum Information About a Microarray Experiment (MIAME) describes the Minimum Information About a Microarray Experiment that is needed to enable the interpretation of the results of the experiment unambiguously and potentially to reproduce the experiment and is aimed at facilitating the dissemination of data from microarray experiments. It was published by the FGED Society in 2001 and was the first published minimum information standard for high-throughput experiments in the life sciences. MIAME contains a number of extensions to cover specific biological domains, including MIAME-env, MIAME-nut and MIAME-tox, covering environmental genomics, nutritional genomics and toxogenomics, respectively. === MINI: Minimum Information about a Neuroscience Investigation === ==== MINI: Electrophysiology ==== Electrophysiology is a technology used to study the electrical properties of biological cells and tissues. Electrophysiology typically involves the measurements of voltage change or electric current flow on a wide variety of scales from single ion channel proteins to whole tissues. This document is a single module, as part of the Minimum Information about a Neuroscience investigation (MINI) family of reporting guideline documents, produced by community consultation and continually available for public comment. A MINI module represents the minimum information that should be reported about a dataset to facilitate computational access and analysis to allow a reader to interpret and critically evaluate the processes performed and the conclusions reached, and to support their experimental corroboration. In practice a MINI module comprises a checklist of information that should be provided (for example about the protocols employed) when a data set is described for publication. The full specification of the MINI module can be found here. === MIARE, RNAi experiment === Minimum Information About an RNAi Experiment (MIARE) is a data reporting guideline which describes the minimum information that should be reported about an RNAi experiment to enable the unambiguous interpretation and reproduction of the results. === MIACA, cell based assay === Advances in genomics and functional genomics have enabled large-scale analyses of gene and protein function by means of high-throughput cell biological analyses. Thereby, cells in culture can be perturbed in vitro and the induced effects recorded and analyzed. Perturbations can be triggered in several ways, for instance with molecules (siRNAs, expression constructs, small chemical compounds, ligands for receptors, etc.), through environmental stresses (such as temperature shift, serum starvation, oxygen deprivation, etc.), or combinations thereof. The cellular responses to such perturbations are analyzed in order to identify molecular events in the biological processes addressed and understand biological principles. We propose the Minimum Information About a Cellular Assay (MIACA) for reporting a cellular assay, and CA-OM, the modular cellular assay object model, to facilitate exchange of data and accompanying information, and to compare and integrate data that originate from different, albeit complementary approaches, and to elucidate higher order principles. Documents describing MIACA are available and provide further information as well as the checklist of terms that should be reported. === MIAPE, proteomic experiments === The Minimum Information About a Proteomic Experiment documents describe information which should be given along with a proteomic experiment. The parent document describes the processes and principles underpinning the development of a series of domain specific documents which now cover all aspects of a MS-based proteomics workflow. === MIMIx, molecular interactions === This document has been developed and maintained by the Molecular Interaction worktrack of the HUPO-PSI (www.psidev.info) and describes the Minimum Information about a Molecular Interaction experiment. === MIAPAR, protein affinity reagents === The Minimum Information About a Protein Affinity Reagent has been developed and maintained by the Molecular Interaction worktrack of the HUPO-PSI (www.psidev.info)in conjunction with the HUPO Antibody Initiative and a European consortium of binder producers and seeks to encourage users to improve their description of binding reagents, such as antibodies, used in the process of protein identification. === MIABE, bioactive entities === The Minimum Information About a Bioactive Entity was produced by representatives from both large pharma and academia who are looking to improve the description of usually small molecules which bind to, and potentially modulate the activity of, specific targets in a living organism. This document encompasses drug-like molecules as well as herbicides, pesticides and food additives. It is primarily maintained through the EMBL-EBI Industry program (www.ebi.ac.uk/industry). === MIGS/MIMS, genome/metagenome sequences === This specification is being developed by the Genomic Standards Consortium === MIFlowCyt, flow cytometry === === Minimum Information about a Flow Cytometry Experiment === The Minimum Information about a Flow Cytometry Experiment (MIFlowCyt) is a standard related to flow cytometry which establishes criteria to record information on experimental overview, samples, instrumentation and data analysis. It promotes consistent annotation of clinical, biological and technical issues surrounding a flow cytometry experiment. === MINDR, dual gene expression reporters === Requires (1) reporting absolute values of reporter readouts, (2) list of positive and negative controls, and (3) sequences of all reporter constructs. === MISFISHIE, In Situ Hybridization and Immunohistochemistry Experiments === === MIAPA, Phylogenetic Analysis === Criteria for Minimum Information About a Phylogenetic Analysis were described in 2006. === MIRAGE, Glycomics === The MIRAGE project is supported and coordinated by the Beilstein-Institut to establish guidelines for data handling and processing in glycomics research [1] === MIAO, ORF === === MIAMET, METabolomics experiment === === MIAFGE, Functional Genomics Experiment === === MIRIAM, Minimum Information Required in the Annotation of Models === The Minimal Information Required In the Annotation of Models (MIRIAM), is a set of rules for the curation and annotation of quantitative models of biological systems. === MIASE, Minimum Information About a Simulation Experiment =

  Read more →
• 

  Mental mapping

  In behavioral geography, a mental map is a person's point-of-view perception of their area of interaction. Although this kind of subject matter would seem most likely to be studied by fields in the social sciences, this particular subject is most often studied by modern-day geographers. Researchers have also applied mental mapping to understand and define cognitive regions. They study it to determine subjective qualities from the public such as personal preference and practical uses of geography like driving directions. Mass media also have a virtually direct effect on a person's mental map of the geographical world. The perceived geographical dimensions of a foreign nation (relative to one's own nation) may often be heavily influenced by the amount of time and relative news coverage that the news media may spend covering news events from that foreign region. For instance, a person might perceive a small island to be nearly the size of a continent, merely based on the amount of news coverage that they are exposed to on a regular basis. In psychology, the term names the information maintained in the mind of an organism by means of which it may plan activities, select routes over previously traveled territories, etc. The rapid traversal of a familiar maze depends on this kind of mental map if scents or other markers laid down by the subject are eliminated before the maze is re-run. == Background == Mental maps are an outcome of the field of behavioral geography. The imagined maps are considered one of the first studies that intersected geographical settings with human action. The most prominent contribution and study of mental maps was in the writings of Kevin Lynch. In The Image of the City, Lynch used simple sketches of maps created from memory of an urban area to reveal five elements of the city; nodes, edges, districts, paths and landmarks. Lynch claimed that “Most often our perception of the city is not sustained, but rather partial, fragmentary, mixed with other concerns. Nearly every sense is in operation, and the image is the composite of them all.” (Lynch, 1960, p 2.) The creation of a mental map relies on memory as opposed to being copied from a preexisting map or image. In The Image of the City, Lynch asks a participant to create a map as follows: “Make it just as if you were making a rapid description of the city to a stranger, covering all the main features. We don’t expect an accurate drawing- just a rough sketch.” (Lynch 1960, p 141) In the field of human geography mental maps have led to an emphasizing of social factors and the use of social methods versus quantitative or positivist methods. Mental maps have often led to revelations regarding social conditions of a particular space or area. Haken and Portugali (2003) developed an information view, which argued that the face of the city is its information . Bin Jiang (2012) argued that the image of the city (or mental map) arises out of the scaling of city artifacts and locations. He addressed that why the image of city can be formed , and he even suggested ways of computing the image of the city, or more precisely the kind of collective image of the city, using increasingly available geographic information such as Flickr and Twitter . Using mental maps, we will be able to predict individual decision making and spatial selection, as well as evaluate their routing and navigation. A cognitive maps utility as a mnemonic and metaphorical device is precisely one of its other benefits as a shaper of the world and local attitudes. The first major field of study within the domain of memory maps is geography, spatial cognition and neurophysiology. This aims to understand how routes are drawn by subject from their set of subjects out into space which lead to memorization and internal representations. Overall these representations take the form of drawings, positioning in a graph, or oral/textual narratives, but are reflected as behavior is space that can be recorded as tracking items. == Research applications == Mental maps have been used in a collection of spatial research. Many studies have been performed that focus on the quality of an environment in terms of feelings such as fear, desire and stress. A study by Matei et al. in 2001 used mental maps to reveal the role of media in shaping urban space in Los Angeles. The study used Geographic Information Systems (GIS) to process 215 mental maps taken from seven neighborhoods across the city. The results showed that people's fear perceptions in Los Angeles are not associated with high crime rates but are instead associated with a concentration of certain ethnicities in a given area. The mental maps recorded in the study draw attention to these areas of concentrated ethnicities as parts of the urban space to avoid or stay away from. Mental maps have also been used to describe the urban experience of children. In a 2008 study by Olga den Besten mental maps were used to map out the fears and dislikes of children in Berlin and Paris. The study looked into the absence of children in today's cities and the urban environment from a child's perspective of safety, stress and fear. Peter Gould and Rodney White have performed prominent analyses in the book “Mental Maps.” This book is an investigation into people's spatial desires. The book asks of its participants: “Suppose you were suddenly given the chance to choose where you would like to live- an entirely free choice that you could make quite independently of the usual constraints of income or job availability. Where would you choose to go?” (Gould, 1974, p 15) Gould and White use their findings to create a surface of desire for various areas of the world. The surface of desire is meant to show people's environmental preferences and regional biases. In an experiment done by Edward C. Tolman, the development of a mental map was seen in rats. A rat was placed in a cross shaped maze and allowed to explore it. After this initial exploration, the rat was placed at one arm of the cross and food was placed at the next arm to the immediate right. The rat was conditioned to this layout and learned to turn right at the intersection in order to get to the food. When placed at different arms of the cross maze however, the rat still went in the correct direction to obtain the food because of the initial mental map it had created of the maze. Rather than just deciding to turn right at the intersection no matter what, the rat was able to determine the correct way to the food no matter where in the maze it was placed. The idea of mental maps is also used in strategic analysis. David Brewster, an Australian strategic analyst, has applied the concept to strategic conceptions of South Asia and Southeast Asia. He argues that popular mental maps of where regions begin and end can have a significant impact on the strategic behaviour of states. A collection of essays, documenting current geographical and historical research in mental maps is published by the Journal of Cultural Geography in 2018.

  Read more →
• 

  TasteDive

  TasteDive (formerly named TasteKid) is an entertainment recommendation engine for films, TV shows, music, video games, books, people, places, and brands. It also has elements of a social media site; it allows users to connect with "tastebuds", people with like minded interests. == History == TasteDive was founded in 2008 as TasteKid by brothers Andrei Oghina and Felix Oghina. In 2019, it was acquired by Qloo headquartered in NYC. "Qloo has built for developers and enterprises what TasteDive has built for individuals". == Description == When a user types in the title of a film or TV show, the site's algorithm provides a list of similar content. It provides recommendations for TV shows to watch based on films liked by the user, and vice versa. It also provides recommendations for music, video games, and books, and includes film and TV trailers and music videos. An account is free and is not required to receive recommendations, but recommendations are more accurate for those with an account. The more a user explores the site, the more the site learns about the user's preferences and the better the results become. The site also has a social media aspect where one can see activity and gain recommendations from other users, how many others in the community like or dislike any recommendation, and how popular their tastes are within the TasteDive community. The main competitors of TasteDive are Taste App, Trakt.tv and Tastoid.

  Read more →
• 

  World model (artificial intelligence)

  A world model in artificial intelligence is a machine learning system that builds an internal representation of an environment. The model predicts how that environment changes over time in response to actions. Researchers design world models to help agents plan, reason, and act without constant real-world trial and error. World models differ from systems that merely classify or generate outputs. They simulate dynamics such as physics, object interactions, and causality. Early ideas date to the 1990s. Modern versions power robots, autonomous driving, and interactive video generation. == History == Jürgen Schmidhuber introduced the term world model in machine learning in 1990. He proposed recurrent neural networks that predict future states from observations and use those predictions to train agents. David Ha and Schmidhuber revived the concept in a 2018 paper. Their agents learned to drive virtual cars and play video games inside self-generated simulations. Yann LeCun advanced the idea in a 2022 position paper titled "A Path Towards Autonomous Machine Intelligence". He argued that intelligence requires predictive models of the world rather than pure pattern matching. LeCun proposed the joint embedding predictive architecture (JEPA) as a practical foundation. LeCun and collaborators developed several JEPA variants. V-JEPA 2 reached state-of-the-art performance on video understanding and physical reasoning at the time. It supports zero-shot robot control in unfamiliar environments. Introduced in March 2026, LeWorldModel trains stably end-to-end from raw pixels and uses two loss terms and avoids hand-crafted heuristics. LeCun founded Advanced Machine Intelligence Labs in 2026 to further develop world models. Google DeepMind introduced Genie in 2024. The model learned interactive environments from unlabeled internet videos. Genie 2 followed in late 2024 and added three-dimensional generation. The Genie series set benchmarks for general-purpose simulation. Genie 3 was introduced in August 2025. It produces photorealistic, real-time interactive worlds from text prompts which are displayed at 24 frames per second and explored in real time with text or image prompts. The model supports persistent three-dimensional worlds and real-time interaction. Waymo adopted Genie 3 in February 2026 and used it to create a specialized world model for autonomous driving simulation, called the Waymo World Model. It produces synchronized camera and lidar outputs and creates edge cases that real robotaxis rarely encounter. The edge cases were reported to be unusual by PCMag. General Intuition announced a $133.7 million seed round. World Labs raised $1 billion. AMI raised $1.03 billion. In April 2026, Alibaba announced Happy Oyster, its world model designed for real-time and “flowy” world model. It includes a directing mode for world building based on text and image prompts and a wandering mode for exploring the resulting world. It can generate 3-minute in-world video clips. Also in April, World Labs, co-founded by Li Fei Fei, unveiled Spark 2.0, an open-source 3D Gaussian splatting rendering engine that targets smartphone-class devices. In June 2026, Nvidia released Cosmos 3, a family of open-weight models. It combines previously independent physical reasoning, world simulation, and action generation. Cosmos 3 integrates can process and generate text, image, video, audio, and action sequences. The model employs a Mixture-of-Transformers" (MoT) approach. An autoregressive (AR) transformer handles reasoning and next-token prediction, while a diffusion transformer (DT) does multimodal generation. Encoders (ViT for vision, VAE for visual/audio, and domain-specific for actions) and generate a shared representation space using 3D multi-dimensional rotary position embedding (mRoPE) for spatial and temporal information. The family includes Cosmos3-Nano (16B parameters) for workstations; Cosmos3-Super (64B parameters) for research. == Architecture == World models process raw sensory data such as video frames or lidar scans. They compress this input into compact latent representations. The system then predicts future representations rather than pixel-by-pixel reconstructions. Many modern world models use joint embedding predictive architecture (JEPA). An encoder turns observations into embeddings. A predictor estimates one or a suite of embeddings from the current one and an action. In some cases a critic chooses one embedding as the best result. A regularizer keeps embeddings well-behaved. The model trains by minimizing prediction error in embedding space. This approach avoids the high cost of generating every detail. Some architectures add explicit components. A fast reactive path handles immediate responses. A slower deliberative path performs longer-horizon planning. Video prediction accuracy or robot success rates are key metrics, but do not always predict real-world performance. Generative world models such as Genie 3 combine these with a simulator. They accept text prompts or layouts and output consistent video, lidar, or three-dimensional scenes. World models often train with self-supervised learning. They use large unlabeled datasets of video or robot interactions. Self-supervised learning can speed learning. Reinforcement learning can fine-tune a model for specific tasks. == Applications == World models support robot learning. Agents train inside simulations and transfer skills to the physical world. This reduces the need for dangerous or expensive real-world trials. Autonomous vehicles use world models to test rare events. Waymo's system simulates tornadoes or unusual pedestrian behavior. Companies train planners without putting vehicles on public roads. Interactive entertainment benefits from world models. Genie 3 lets users generate playable environments from simple descriptions. Game studios prototype levels faster. Scientific simulation gains from these models. Researchers model physical systems or biological processes at scale. Planners in logistics or urban design test strategies inside accurate digital twins. == Comparison with large language models == Both world models and large language models (LLMs) use inferencing on their inputs to make predictions. LLMs operate on textual inputs. They predict the next token in text sequences. They excel at language-oriented tasks such as translation or summarization. However, they lack understanding of physics. World models operate on sensor inputs such as pixels. They predict state changes in that data in latent space. This design supports planning and causal reasoning. LLMs generate fluent text but often fail at consistent physical predictions. Their architecture employs transformers with refinements such as mixture of experts. World models divide an inferencing task into work performed by encoders, predictors, simulators, and other pieces. They typically handle multimodal inputs such as video, lidar, radar, and audio, guided by textual prompting. LLMs power chatbots and code assistants. World models drive embodied agents that act in dynamic environments, such as autonomous driving. The two may be combined in hybrid systems. For example, a LLM handles instructions, while a world model manages low-level control. World model proponents such as LeCun claim that because LLMs are trained only on text, they have no ability to predict anything beyond text, such as real-world events. == Benchmarks == World model benchmarks test physical understanding, long-term consistency, planning, and generalization from sensor data. Meta introduced three benchmarks for V-JEPA 2. IntPhys 2 measures a model's ability to detect physics violations. It presents pairs of videos that diverge when one breaks physical rules. Humans score near 100% accuracy. V-JEPA 2 achieves little better than random chance on many conditions. Minimal Video Pairs (MVPBench) tests physical understanding through multiple-choice questions based on short video clips. It probes object interactions and causality. Something-Something tests action recognition. Epic-Kitchens-100 tests human action anticipation. DeepMind benchmark: Interactive evaluation measures consistency over minutes of interaction, memory of off-screen objects, and response to user actions or text prompts. Waymo benchmark: Output generation quality: Metrics include realism, controllability (via text prompts), and usefulness for training planners in simulated worlds. However, pixel reconstruction error rate with episodic rewards often fails. Other: Epic-Kitchens-100 (often measured with Recall@5) Ego4D 50 Salads, Breakfast, etc. Potential benchmarks: Zero-shot transfer to robots Long-horizon planning Implausible prediction rate

  Read more →
• 

  Resilience (mathematics)

  In mathematical modeling, resilience refers to the ability of a dynamical system to recover from perturbations and return to its original stable steady state. It is a measure of the stability and robustness of a system in the face of changes or disturbances. If a system is not resilient enough, it is more susceptible to perturbations and can more easily undergo a critical transition. A common analogy used to explain the concept of resilience of an equilibrium is one of a ball in a valley. A resilient steady state corresponds to a ball in a deep valley, so any push or perturbation will very quickly lead the ball to return to the resting point where it started. On the other hand, a less resilient steady state corresponds to a ball in a shallow valley, so the ball will take a much longer time to return to the equilibrium after a perturbation. The concept of resilience is particularly useful in systems that exhibit tipping points, whose study has a long history that can be traced back to catastrophe theory. While this theory was initially overhyped and fell out of favor, its mathematical foundation remains strong and is now recognized as relevant to many different systems. == History == In 1973, Canadian ecologist C. S. Holling proposed a definition of resilience in the context of ecological systems. According to Holling, resilience is "a measure of the persistence of systems and of their ability to absorb change and disturbance and still maintain the same relationships between populations or state variables". Holling distinguished two types of resilience: engineering resilience and ecological resilience. Engineering resilience refers to the ability of a system to return to its original state after a disturbance, such as a bridge that can be repaired after an earthquake. Ecological resilience, on the other hand, refers to the ability of a system to maintain its identity and function despite a disturbance, such as a forest that can regenerate after a wildfire while maintaining its biodiversity and ecosystem services. With time, the once well-defined and unambiguous concept of resilience has experienced a gradual erosion of its clarity, becoming more vague and closer to an umbrella term than a specific concrete measure. == Definition == Mathematically, resilience can be approximated by the inverse of the return time to an equilibrium given by resilience ≡ − Re ( λ 1 ( A ) ) {\displaystyle {\text{resilience}}\equiv -{\text{Re}}(\lambda _{1}({\textbf {A}}))} where λ 1 {\textstyle \lambda _{1}} is the maximum eigenvalue of matrix A {\textstyle {\textbf {A}}} . The largest this value is, the faster a system returns to the original stable steady state, or in other words, the faster the perturbations decay. == Applications and examples == In ecology, resilience might refer to the ability of the ecosystem to recover from disturbances such as fires, droughts, or the introduction of invasive species. A resilient ecosystem would be one that is able to adapt to these changes and continue functioning, while a less resilient ecosystem might experience irreversible damage or collapse. The exact definition of resilience has remained vague for practical matters, which has led to a slow and proper application of its insights for management of ecosystems. In epidemiology, resilience may refer to the ability of a healthy community to recover from the introduction of infected individuals. That is, a resilient system is more likely to remain at the disease-free equilibrium after the invasion of a new infection. Some stable systems exhibit critical slowing down where, as they approach a basic reproduction number of 1, their resilience decreases, hence taking a longer time to return to the disease-free steady state. Resilience is an important concept in the study of complex systems, where there are many interacting components that can affect each other in unpredictable ways. Mathematical models can be used to explore the resilience of such systems and to identify strategies for improving their resilience in the face of environmental or other changes. For example, when modelling networks it is often important to be able to quantify network resilience, or network robustness, to the loss of nodes. Scale-free networks are particularly resilient since most of their nodes have few links. This means that if some nodes are randomly removed, it is more likely that the nodes with fewer connections are taken out, thus preserving the key properties of the network.

  Read more →
• 

  Safe and Secure Innovation for Frontier Artificial Intelligence Models Act

  The Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, or SB 1047, was a failed 2024 California bill intended to "mitigate the risk of catastrophic harms from AI models so advanced that they are not yet known to exist". Specifically, the bill would have applied to models which cost more than $100 million to train and were trained using a quantity of computing power greater than 1026 integer or floating-point operations. SB 1047 would have applied to all AI companies doing business in California—the location of the company would not matter. The bill would have created protections for whistleblowers and required developers to perform risk assessments of their models prior to release, with guidance from the Government Operations Agency. It would also have established CalCompute, a University of California public cloud computing cluster for startups, researchers and community groups. == Background == The rapid increase in capabilities of AI systems in the 2020s, including the release of ChatGPT in November 2022, caused some researchers and members of the public to become concerned about the existential risks associated with increasingly powerful AI systems. Hundreds of tech executives and AI researchers, including two of the so-called "Godfathers of AI", Geoffrey Hinton and Yoshua Bengio, signed a statement in May 2023 calling for the mitigation of the "risk of extinction from AI" to be a global priority alongside "pandemics and nuclear war". However, the plausibility of these risks is still widely debated. Strong regulation of AI has been criticized for purportedly causing regulatory capture by large AI companies like OpenAI, a phenomenon in which regulation advances the interest of larger companies at the expense of smaller competition and the public in general, although OpenAI ended up opposing the bill. Other advocates of AI regulation aim to prevent bias and privacy violations, rather than existential risks. For example, some experts who view existential concerns as overblown and unrealistic view them as a distraction from near-term harms of AI like discriminatory automated decision making. In the face of existential concerns, technology companies have made voluntary commitments to conduct safety testing, for example at the AI Safety Summit and AI Seoul Summit. In 2023, not long before the bill was proposed, Governor Newsom of California and President Biden issued executive orders on artificial intelligence. State Senator Wiener said SB 1047 draws heavily on the Biden executive order, and is motivated by the absence of unified federal legislation on AI safety. Historically, California has passed regulation on several tech issues itself, including consumer privacy and net neutrality, in the absence of action by Congress. == History == === Proposal and voting === The bill was authored by State Senator Scott Wiener. Wiener first proposed AI legislation for California through an intent bill called SB 294, the Safety in Artificial Intelligence Act, in September 2023. On February 7, 2024, Wiener introduced SB 1047. On May 21, SB 1047 passed the Senate 32–1. The bill was significantly amended by Wiener on August 15, 2024, in response to industry advice. Amendments included adding clarifications, and removing the creation of a "Frontier Model Division" and the penalty of perjury. On August 28, the bill passed the State Assembly 48–16. Then, due to the amendments, the bill was once again voted on by the Senate, passing 30–9. === Veto by governor === On September 29, Governor Gavin Newsom vetoed the bill. The deadline for California lawmakers to overrule Newsom's veto was November 30, 2024. Newsom cited concerns over the bill's regulatory framework targeting only large AI models based on their computational size, while not taking into account whether the models are deployed in high-risk environments. Newsom emphasized that this approach could create a false sense of security, overlooking smaller models that might present equally significant risks. He acknowledged the need for AI safety protocols but stressed the importance of adaptability in regulation as AI technology continues to evolve rapidly. Governor Newsom also committed to working with technology experts, federal partners, and research institutions, including the Carnegie Endowment for International Peace, led by former California Supreme Court Justice Mariano-Florentino Cuéllar; and Stanford University's Human-Centered AI (HAI) Institute, led by Dr. Fei-Fei Li. He announced plans to collaborate with these entities to advance responsible AI development, aiming to protect the public while fostering innovation. == Provisions == SB 1047 would have covered AI models with training compute over 1026 integer or floating-point operations and a cost of over $100 million. If a covered model is fine-tuned using more than $10 million, the resulting model would also have been covered. The bill would have defined critical harms with respect to four categories: Creation or use of a chemical, biological, radiological, or nuclear weapon Cyberattacks on critical infrastructure causing mass casualties or at least $500 million of damage Autonomous crimes causing mass casualties or at least $500 million of damage Other harms of comparable severity Developers would have needed to create a "safety and security protocol" before training covered models. Before deployment, they would have submitted a statement of compliance, confirming they took reasonable care to take measures to prevent covered models that pose an unreasonable risk of critical harms. The statement would have included risk assessments and descriptions of their compliance process. These rules would have applied to both covered models and their derivatives, including post-training modifications, with annual third-party audits required starting in 2026. Safeguards to reduce risk included the ability to shut down the model, which has been variously described as a "kill switch" and "circuit breaker". Whistleblowing provisions would have protected employees who report safety problems and incidents. Additionally, SB 1047 would have created a public cloud computing cluster called CalCompute, associated with the University of California, to support startups, researchers, and community groups that lack large-scale computing resources. === Compliance and supervision === SB 1047 would have required developers, beginning January 1, 2026, to annually retain a third-party auditor to perform an independent audit of compliance with the requirements of the bill, as provided. The Government Operations Agency would have reviewed the results of safety tests and incidents, and issue guidance, standards, and best practices. The bill would have created a Board of Frontier Models to supervise the application of the bill by the Government Operations Agency. It is would be composed of 9 members. == Reception == === Subjects of debate === Proponents of the bill described its provisions as simple and narrowly focused, with Sen. Scott Weiner describing it as a "light-touch, basic safety bill". This was disputed by critics of the bill, who described the bill's language as vague and criticized it as consolidating power in the largest AI companies at the expense of smaller ones. Proponents, in turn, argued that the bill only applies to models trained using more than 1026 FLOPS and with over $100 million, or fine-tuned with more than $10 million, and that the threshold could be increased if needed. The penalty of perjury was also a subject of debate, and was eventually removed through an amendment. The scope of the "kill switch" requirement was also reduced, following concerns from open-source developers. The use of the term "reasonable assurance" in the bill was also controversial, and it was eventually amended to "reasonable care". Critics then argued that "reasonable care" imposed an excessive burden by requiring confidence that models could not be used to cause catastrophic harm; proponents claimed that the standard did not require certainty and that it already applied to AI developers under existing law. === Support and opposition === Individual supporters of the bill included Turing Award recipients Yoshua Bengio and Geoffrey Hinton, Elon Musk, Bill de Blasio, Kevin Esvelt, Dan Hendrycks, Vitalik Buterin, OpenAI whistleblowers Daniel Kokotajlo and William Saunders, Lawrence Lessig, Sneha Revanur, Stuart Russell, Jan Leike, actors Mark Ruffalo, Sean Astin, and Rosie Perez, Scott Aaronson, and Max Tegmark. Over 120 Hollywood celebrities, including Mark Hamill, Jane Fonda, and J. J. Abrams, also signed a statement in support of the bill. Max Tegmark likened the bill's focus on holding companies responsible for the harms caused by their models to the FDA requiring clinical trials before a company can release a drug to the market. Organizations sponsoring the bill included the Center for AI Safety, Economic Security California and Encode. The la

  Read more →
• 

  Hubert Dreyfus's views on artificial intelligence

  Hubert Dreyfus was a critic of artificial intelligence research. In a series of papers and books, including Alchemy and AI (1965), What Computers Can't Do (1972; 1979; 1992) and Mind over Machine (1986), he presented a skeptical and cautious assessment of AI's progress and a critique of the philosophical foundations of the field. Dreyfus' objections are discussed in most introductions to the philosophy of artificial intelligence, including Russell & Norvig (2021), a standard AI textbook, and in Fearn (2007), a survey of contemporary philosophy. Dreyfus argued that human intelligence and expertise depend primarily on yet-to-be understood informal and unconscious processes rather than symbolic manipulation and that these essentially human skills cannot be fully captured in formal rules. His critique was based on the insights of modern continental philosophers such as Merleau-Ponty and Heidegger, and was directed at the first wave of AI research which tried to reduce intelligence to high level formal symbols. When Dreyfus' ideas were first introduced in the mid-1960s, they were met in the AI community with ridicule and outright hostility. By the 1980s, however, some of his perspectives were rediscovered by researchers working in robotics and the new field of connectionism—approaches that were called "sub-symbolic" at the time because they eschewed early AI research's emphasis on high level symbols. In the 21st century, "sub-symbolic" artificial neural networks and other statistics-based approaches to machine learning were highly successful. Historian and AI researcher Daniel Crevier wrote: "time has proven the accuracy and perceptiveness of some of Dreyfus's comments." Dreyfus said in 2007, "I figure I won and it's over—they've given up." == Dreyfus' critique == === The grandiose promises of artificial intelligence === In Alchemy and AI (1965) and What Computers Can't Do (1972), Dreyfus summarized the history of artificial intelligence and ridiculed the unbridled optimism that permeated the field. For example, Herbert A. Simon, following the success of his program General Problem Solver (1957), predicted that by 1967: A computer would be world champion in chess. A computer would discover and prove an important new mathematical theorem. Most theories in psychology will take the form of computer programs. The press dutifully reported these predictions of the imminent arrival of machine intelligence. Dreyfus felt that this optimism was unwarranted and, in 1965, argued forcefully that predictions like these would not come true. He would eventually be proven right. Pamela McCorduck explains Dreyfus' position: A great misunderstanding accounts for public confusion about thinking machines, a misunderstanding perpetrated by the unrealistic claims researchers in AI have been making, claims that thinking machines are already here, or at any rate, just around the corner. These predictions were based on the success of the cognitive revolution, which promoted an "information processing" model of the mind. It was articulated by Newell and Simon in their physical symbol systems hypothesis, and later expanded into a philosophical position known as computationalism by philosophers such as Jerry Fodor and Hilary Putnam. In AI, the approach is now called symbolic AI or "GOFAI". Dreyfus argued that "symbolic AI" was the latest version of the ancient program of rationalism in philosophy. Rationalism had come under heavy criticism in the 20th century from philosophers like Martin Heidegger and Edmund Husserl. The mind, according to modern continental philosophy, is not "rationalist" and is nothing like a digital computer. Cognitivism led early AI researchers to believe that they had successfully simulated the essential process of human thought, thus it seemed a short step to producing fully intelligent machines. Dreyfus' last paper detailed the ongoing history of the "first step fallacy", where AI researchers tend to wildly extrapolate initial success as promising, perhaps even guaranteeing, wild future successes. === Dreyfus' four assumptions of artificial intelligence research === In Alchemy and AI and What Computers Can't Do, Dreyfus identified four philosophical assumptions, at least one of which he deems necessary for AI to succeed. "In each case," Dreyfus writes, "the assumption is taken by workers in AI as an axiom, guaranteeing results, whereas it is, in fact, one hypothesis among others, to be tested by the success of such work." Dreyfus argues that AI would be impossible without accepting at least one of these four assumptions: The biological assumption The brain processes information in discrete operations by way of some biological equivalent of on/off switches. In the early days of research into neurology, scientists found that neurons fire in all-or-nothing pulses. Several researchers, such as Walter Pitts and Warren McCulloch, speculated with great confidence that neurons functioned similarly to the way Boolean logic gates operate, and so could be imitated by electronic circuitry at the level of the neuron. When digital computers became widely used in the early 50s, this argument was extended to suggest that the brain was a vast physical symbol system, manipulating the binary symbols of zero and one. Dreyfus was able to refute the biological assumption by citing research in neurology that suggested that the action and timing of neuron firing had analog components. But Daniel Crevier observes that "few still held that belief in the early 1970s, and nobody argued against Dreyfus" about the biological assumption. The psychological assumption The mind can be viewed as a device operating on bits of information according to formal rules. He refuted this assumption by showing that much of what we know about the world consists of complex attitudes or tendencies that make us lean towards one interpretation over another. He argued that, even when we use explicit symbols, we are using them against an unconscious and informal background including commonsense knowledge and that without this background our symbols cease to mean anything. This background, in Dreyfus' view, was not implemented in individual brains as explicit individual symbols with explicit individual meanings. The epistemological assumption All knowledge can be formalized. This concerns the philosophical issue of epistemology, or the study of knowledge. Even if we agree that the psychological assumption is false, AI researchers could still argue (as AI founder John McCarthy has) that it is possible for a symbol processing machine to represent all knowledge, regardless of whether human beings represent knowledge the same way. Dreyfus argued that there is no justification for this assumption, since so much of human knowledge is not symbolic or even expressible using formal constructs. The ontological assumption The world consists of independent facts that can be represented by independent symbols AI researchers (and futurists and science fiction writers) often assume that there is no limit to formal, scientific knowledge, because they assume that any phenomenon in the universe can be described by symbols or scientific theories. This assumes that everything that exists can be understood as objects, properties of objects, classes of objects, relations of objects, and so on: precisely those things that can be described by logic, language and mathematics. The study of being or existence is called ontology, and so Dreyfus calls this the ontological assumption. If this is false, then it raises doubts about what we can ultimately know and what intelligent machines will ultimately be able to help us to do. === Knowing-how vs. knowing-that: the primacy of intuition === In Mind Over Machine (1986), written (with his brother) during the heyday of expert systems, Dreyfus analyzed the difference between human expertise and the programs that claimed to capture it. This expanded on ideas from What Computers Can't Do, where he had made a similar argument criticizing the "cognitive simulation" school of AI research practiced by Allen Newell and Herbert A. Simon in the 1960s. Dreyfus argued that human problem solving and expertise depend on our background sense of the context, of what is important and interesting given the situation, rather than on the process of searching through combinations of possibilities to find what we need. Dreyfus would describe it in 1986 as the difference between "knowing-that" and "knowing-how", based on Heidegger's distinction of present-at-hand and ready-to-hand. Knowing-that is our conscious, step-by-step problem solving abilities. We use these skills when we encounter a difficult problem that requires us to stop, step back and search through ideas one at time. At moments like this, the ideas become very precise and simple: they become context free symbols, which we manipulate using logic and language. These are the skills that Newell and Simon had demonstrated with both psy

  Read more →
• 

  Neural network Gaussian process

  A Neural Network Gaussian Process (NNGP) is a Gaussian process (GP) obtained as the limit of a certain type of sequence of neural networks. Specifically, a wide variety of network architectures converges to a GP in the infinitely wide limit, in the sense of distribution. The concept constitutes an intensional definition, i.e., a NNGP is just a GP, but distinguished by how it is obtained. == Motivation == Bayesian networks are a modeling tool for assigning probabilities to events, and thereby characterizing the uncertainty in a model's predictions. Deep learning and artificial neural networks are approaches used in machine learning to build computational models which learn from training examples. Bayesian neural networks merge these fields. They are a type of neural network whose parameters and predictions are both probabilistic. While standard neural networks often assign high confidence even to incorrect predictions, Bayesian neural networks can more accurately evaluate how likely their predictions are to be correct. Computation in artificial neural networks is usually organized into sequential layers of artificial neurons. The number of neurons in a layer is called the layer width. When we consider a sequence of Bayesian neural networks with increasingly wide layers (see figure), they converge in distribution to a NNGP. This large width limit is of practical interest, since the networks often improve as layers get wider. And the process may give a closed form way to evaluate networks. NNGPs also appears in several other contexts: It describes the distribution over predictions made by wide non-Bayesian artificial neural networks after random initialization of their parameters, but before training; it appears as a term in neural tangent kernel prediction equations; it is used in deep information propagation to characterize whether hyperparameters and architectures will be trainable. It is related to other large width limits of neural networks. === Scope === The first correspondence result had been established in the 1995 PhD thesis of Radford M. Neal, then supervised by Geoffrey Hinton at University of Toronto. Neal cites David J. C. MacKay as inspiration, who worked in Bayesian learning. Today the correspondence is proven for: Single hidden layer Bayesian neural networks; deep fully connected networks as the number of units per layer is taken to infinity; convolutional neural networks as the number of channels is taken to infinity; transformer networks as the number of attention heads is taken to infinity; recurrent networks as the number of units is taken to infinity. In fact, this NNGP correspondence holds for almost any architecture: Generally, if an architecture can be expressed solely via matrix multiplication and coordinatewise nonlinearities (i.e., a tensor program), then it has an infinite-width GP. This in particular includes all feedforward or recurrent neural networks composed of multilayer perceptron, recurrent neural networks (e.g., LSTMs, GRUs), (nD or graph) convolution, pooling, skip connection, attention, batch normalization, and/or layer normalization. === Illustration === Every setting of a neural network's parameters θ {\displaystyle \theta } corresponds to a specific function computed by the neural network. A prior distribution p ( θ ) {\displaystyle p(\theta )} over neural network parameters therefore corresponds to a prior distribution over functions computed by the network. As neural networks are made infinitely wide, this distribution over functions converges to a Gaussian process for many architectures. The notation used in this section is the same as the notation used below to derive the correspondence between NNGPs and fully connected networks, and more details can be found there. The figure to the right plots the one-dimensional outputs z L ( ⋅ ; θ ) {\displaystyle z^{L}(\cdot ;\theta )} of a neural network for two inputs x {\displaystyle x} and x ∗ {\displaystyle x^{}} against each other. The black dots show the function computed by the neural network on these inputs for random draws of the parameters from p ( θ ) {\displaystyle p(\theta )} . The red lines are iso-probability contours for the joint distribution over network outputs z L ( x ; θ ) {\displaystyle z^{L}(x;\theta )} and z L ( x ∗ ; θ ) {\displaystyle z^{L}(x^{};\theta )} induced by p ( θ ) {\displaystyle p(\theta )} . This is the distribution in function space corresponding to the distribution p ( θ ) {\displaystyle p(\theta )} in parameter space, and the black dots are samples from this distribution. For infinitely wide neural networks, since the distribution over functions computed by the neural network is a Gaussian process, the joint distribution over network outputs is a multivariate Gaussian for any finite set of network inputs. == Discussion == === Infinitely wide fully connected network === This section expands on the correspondence between infinitely wide neural networks and Gaussian processes for the specific case of a fully connected architecture. It provides a proof sketch outlining why the correspondence holds, and introduces the specific functional form of the NNGP for fully connected networks. The proof sketch closely follows the approach by Novak and coauthors. ==== Network architecture specification ==== Consider a fully connected artificial neural network with inputs x {\displaystyle x} , parameters θ {\displaystyle \theta } consisting of weights W l {\displaystyle W^{l}} and biases b l {\displaystyle b^{l}} for each layer l {\displaystyle l} in the network, pre-activations (pre-nonlinearity) z l {\displaystyle z^{l}} , activations (post-nonlinearity) y l {\displaystyle y^{l}} , pointwise nonlinearity ϕ ( ⋅ ) {\displaystyle \phi (\cdot )} , and layer widths n l {\displaystyle n^{l}} . For simplicity, the width n L + 1 {\displaystyle n^{L+1}} of the readout vector z L {\displaystyle z^{L}} is taken to be 1. The parameters of this network have a prior distribution p ( θ ) {\displaystyle p(\theta )} , which consists of an isotropic Gaussian for each weight and bias, with the variance of the weights scaled inversely with layer width. This network is illustrated in the figure to the right, and described by the following set of equations: x ≡ input y l ( x ) = { x l = 0 ϕ ( z l − 1 ( x ) ) l > 0 z i l ( x ) = ∑ j W i j l y j l ( x ) + b i l W i j l ∼ N ( 0 , σ w 2 n l ) b i l ∼ N ( 0 , σ b 2 ) ϕ ( ⋅ ) ≡ nonlinearity y l ( x ) , z l − 1 ( x ) ∈ R n l × 1 n L + 1 = 1 θ = { W 0 , b 0 , … , W L , b L } {\displaystyle {\begin{aligned}x&\equiv {\text{input}}\\y^{l}(x)&=\left\{{\begin{array}{lcl}x&&l=0\\\phi \left(z^{l-1}(x)\right)&&l>0\end{array}}\right.\\z_{i}^{l}(x)&=\sum _{j}W_{ij}^{l}y_{j}^{l}(x)+b_{i}^{l}\\W_{ij}^{l}&\sim {\mathcal {N}}\left(0,{\frac {\sigma _{w}^{2}}{n^{l}}}\right)\\b_{i}^{l}&\sim {\mathcal {N}}\left(0,\sigma _{b}^{2}\right)\\\phi (\cdot )&\equiv {\text{nonlinearity}}\\y^{l}(x),z^{l-1}(x)&\in \mathbb {R} ^{n^{l}\times 1}\\n^{L+1}&=1\\\theta &=\left\{W^{0},b^{0},\dots ,W^{L},b^{L}\right\}\end{aligned}}} ==== ==== z l | y l {\displaystyle z^{l}|y^{l}} is a Gaussian process We first observe that the pre-activations z l {\displaystyle z^{l}} are described by a Gaussian process conditioned on the preceding activations y l {\displaystyle y^{l}} . This result holds even at finite width. Each pre-activation z i l {\displaystyle z_{i}^{l}} is a weighted sum of Gaussian random variables, corresponding to the weights W i j l {\displaystyle W_{ij}^{l}} and biases b i l {\displaystyle b_{i}^{l}} , where the coefficients for each of those Gaussian variables are the preceding activations y j l {\displaystyle y_{j}^{l}} . Because they are a weighted sum of zero-mean Gaussians, the z i l {\displaystyle z_{i}^{l}} are themselves zero-mean Gaussians (conditioned on the coefficients y j l {\displaystyle y_{j}^{l}} ). Since the z l {\displaystyle z^{l}} are jointly Gaussian for any set of y l {\displaystyle y^{l}} , they are described by a Gaussian process conditioned on the preceding activations y l {\displaystyle y^{l}} . The covariance or kernel of this Gaussian process depends on the weight and bias variances σ w 2 {\displaystyle \sigma _{w}^{2}} and σ b 2 {\displaystyle \sigma _{b}^{2}} , as well as the second moment matrix K l {\displaystyle K^{l}} of the preceding activations y l {\displaystyle y^{l}} , z i l ∣ y l ∼ G P ( 0 , σ w 2 K l + σ b 2 ) K l ( x , x ′ ) = 1 n l ∑ i y i l ( x ) y i l ( x ′ ) {\displaystyle {\begin{aligned}z_{i}^{l}\mid y^{l}&\sim {\mathcal {GP}}\left(0,\sigma _{w}^{2}K^{l}+\sigma _{b}^{2}\right)\\K^{l}(x,x')&={\frac {1}{n^{l}}}\sum _{i}y_{i}^{l}(x)y_{i}^{l}(x')\end{aligned}}} The effect of the weight scale σ w 2 {\displaystyle \sigma _{w}^{2}} is to rescale the contribution to the covariance matrix from K l {\displaystyle K^{l}} , while the bias is shared for all inputs, and so σ b 2 {\displaystyle \sigma _{b}^{2}} makes the z i l {\displaystyle z_{i}^{l}} for different datapoints more similar and

  Read more →
• 

  ITU-WHO Focus Group on Artificial Intelligence for Health

  The ITU-WHO Focus Group on Artificial Intelligence for Health (AI for Health) was an inter-agency collaboration from 2018 between the World Health Organization and the ITU, which in 2019 created a benchmarking framework to assess the accuracy of AI in health. The organization convened an international network of experts and stakeholders from fields like research, practice, regulation, ethics, public health, etc, that developed guideline documentation and code. The documents have addressed ethics, assessment/evaluation, handling, and regulation of AI for health solutions, covering specific use cases including AI in ophthalmology, histopathology, dentistry, malaria detection, radiology, symptom checker applications, etc. FG-AI4H has established an ad hoc group concerned with digital technologies for health emergencies, including COVID-19. All documentation is public. The idea for the Focus Group came out of the Health Track of the 2018 AI for Good Global Summit. Administratively, FG-AI4H was created by ITU-T Study Group 16. Under ITU-T's framework, participation in Focus Groups is open to anyone from an ITU Member State. The secretariat is provided by the Telecommunication Standardization Bureau (under Director Chaesub Lee). It was first created at the July 2018 meeting with a lifetime of two years, at the July 2020 meeting, this was extended for another two years, where the focus group also submitted its deliverables to its parent body. It was also presented at the NeurIPS 2020 health workshop. In July 2023 "the work was grandfathered in the Global Initiative on AI for Health (GI-AI4H)". == AI for Health Framework == The outline of the benchmarking framework was published in a 2019 commentary in The Lancet. The output of the Focus Group AI for Health were structured in the AI for Health Framework. Depending on their primary domain being health or ICT, the individual components of the AI for Health Framework were ratified by the corresponding United Nations Specialized Agency, as WHO Guidelines and ITU Recommendations respectively. Standards drawn up by FG-AI4H were titled as: AI4H ethics considerations AI4H regulatory [best practices | considerations] AI4H requirements specification AI software life cycle specification Data specification AI training best practices specification AI4H evaluation considerations AI4H scale-up and adoption AI4H applications and platforms Use cases of the ITU-WHO Focus Group on AI for Health

  Read more →
• 

  POSC Caesar

  POSC Caesar Association (PCA) is an international, open and not-for-profit, member organization that promotes the development of open specifications to be used as standards for enabling the interoperability of data, software and related matters. PCA is the initiator of ISO 15926 "Integration of life-cycle data for process plants including oil and gas production facilities" and is committed to its maintenance and enhancement. Nils Sandsmark has been the General Manager of POSC Caesar Association since 1999 and Thore Langeland, Norwegian Oil Industry Association (Norwegian: Oljeindustriens Landsforening, OLF), is the chairman of the board. == History == === Caesar Offshore === The first predecessor of POSC Caesar Association, the Caesar Offshore program, started in 1993. The original focus was on standardizing technical data definitions for capital intensive projects at the handover from the EPC contractor to the owner/operators of onshore and offshore oil and gas production facilities. The program was sponsored by The Research Council of Norway, two EPC contractors (Aker Maritime and Kværner), three owners/operators (Norsk Hydro, Saga Petroleum and Statoil) and DNV as service provider and project owner. === POSC Caesar project === During the period 1994–96, Caesar Offshore Program was defined as a project of Petrotechnical Open Software Corporation (POSC) (now Energistics), and changed its name to the POSC Caesar Project. In 1995 the project was joined by BP, Brown and Root and Elf Aquitaine and in 1997 by Intergraph, IBM, Oracle, Lloyd's, Shell, ABB and UMOE Technologies. During that time, POSC Caesar also became a member of European Process Industries STEP Technical Liaison Executive (EPISTLE) where it collaborates with PISTEP (UK), and USPI-NL (The Netherlands) on the development of ISO 10303, also known as "Standard for the Exchange of Product model data (STEP)". === POSC Caesar Association === In 1997, POSC Caesar Association was founded as an independent, global, non-profit, member organization. POSC Caesar Association serves an international membership and collaborates with other international organizations. It has its main office in Norway. Albeit the name of POSC Caesar Association still hints to its past as a project within the Petrotechnical Open Software Corporation (POSC) (now Energistics), from 1997 onwards, the organization has been independent. Energistics and POSC Caesar Association do collaborate, and are formally member in each other's organization. == Membership == POSC Caesar Association has with its current 36 members from around the world and has established an international footprint (with a strong membership in Norway) that includes a variety of backgrounds, from academia and solution providers to engineering contractors and owners/operators. The members are (subdivided by organization type): Associations: Energistics (USA) and The Norwegian Oil Industry Association (OLF, Norway); Universities and Research Institutes: International Research Institute of Stavanger (IRIS, Norway), Norwegian University of Science and Technology (NTNU, Norway), Korea Advanced Institute of Science and Technology (KAIST, Korea), SINTEF (Norway), University of Bergen (Norway), University of Oslo (Norway), University of Stavanger (Norway), University of Tromsø (Norway) and Western Norway Research Institute (Norway); Oil and Gas Companies: BP (UK), Petronas (Malaysia) and Statoil (Norway); Engineering contractors and consultants: Akvaplan-niva (Norway), Aker Solutions (Norway), Asset Life Cycle Information Management (ALCIM, Malaysia), CAESAR systems (USA), Bechtel (USA), Det Norske Veritas (DNV, Norway), Information Logic (USA) and iXIT Engineering Technology (Germany), Phusion IM Ltd (UK); Solution providers: Aveva (UK), Bentley Systems (USA), Jotne EPM Technology (Norway), Epsis (Norway), Eurostep (Sweden), International Business Machines Corporation (IBM, USA), Siemens - Comos Industry Solutions (before Innotec) (Germany), Intergraph (USA), Invenia (Norway), Keel Solution (Denmark), Noumenon (UK), NRX (Canada), Octaga (Norway) and Tektonisk (Norway). In general, the organization holds three membership meetings a year; one in January / February in North-America (typically USA), one in April / May in Europe (typically Norway) and one in October in Asia (typically Malaysia). == Activities and services == === Initiator and custodian of ISO 15926 === In consultation with the other EPISTLE members and the International Organization for Standardization (ISO), it was decided in 2003 (some say already in 1997) that for modeling-technical reasons it was better to discontinue the development of ISO 10303 and to initiate the development of ISO 15926 "Integration of life-cycle data for process plants including oil and gas production facilities." Over the years, the scope of the standard has increased from the initial capital-intensive projects in the upstream oil and gas industry, to include also relevant terminology for downstream oil and gas industry applications and to deal with real-time data related to the actual oil and gas production. ISO 15926 has also over the years evolved from a dictionary (a list of terms with definitions), over a taxonomy (added hierarchy) to an ontology (a formal representation of a set of concepts within a domain and the relationships between those concepts). ISO 15926 is therefore sometimes nicknamed the "Oil and Gas Ontology", for some considered to be an essential prerequisite together with Semantic Web technologies to get to better interoperability, an optimal use of all available data across boundaries and an increase in efficiency. This is what some call the next generation of Integrated Operations. === Reference data services === Placeholders: Flow scheme of WIP - RDS - ISO and role of SIGs RDS Standards in database pilot (ISO) === Special interest groups === Placeholders: Overview of SIGs Drilling and Completion Reservoir and Production Operations and Maintenance == Projects == There are a number of projects (co-)organized by POSC Caesar Association working on the extension of the ISO 15926 standard in different application areas. === Capital intensive projects application domain === The following projects are running at the moment (August 2009): The ADI Project of FIATECH, to build the tools (which will then be made available in the public domain) The IDS Project of POSC Caesar Association, to define product models required for data sheets A joint collaboration project between FIATECH POSC Caesar Association is the ADI-IDS project is the ISO 15926 WIP === Upstream oil and gas industry application domain === The following projects are currently running (August 2009): The Integrated Operations in the High North (IOHN) project is working on extending ISO 15926 to handle real-time data transmission and (pre-)processing to enable the next generation of Integrated Operations. The Environment Web project to include environmental reporting terms and definitions as used in EPIM's EnvironmentWeb in ISO 15926. Finalised projects include: The Integrated Information Platform (IIP) project working on establishing a real-time information pipeline based on open standards. It worked among others on: Daily Drilling Report (DDR) to including all terms and definitions in ISO 15926. This standard became mandatory on February 1, 2008 for reporting on the Norwegian Continental Shelf by the Norwegian Petroleum Directorate (NPD) and Safety Authority Norway (PSA). NPD says that the quality of the reports has improved considerably since. Daily Production Report (DPR) to including all terms and definitions in ISO 15926. This standard was tested successfully on the Valhall (BP-operated) and Åsgard (StatoilHydro-operated) fields offshore Norway. The terminology and XML schemata developed have also been included in Energistics’ PRODML standard. == Conferences and events == === Semantic Days === === Sogndal academic network meeting === == Collaborations == POSC Caesar is collaborating with a number of standardization bodies, including: Mimosa: collaboration on open information standards for Operations and Maintenance mainly for the downstream oil and gas industry; FIATECH: collaboration on open information standards for life cycle data of capital projects; Energistics: collaboration on information standards for the upstream oil and gas industry, including WITSML and PRODML; OASIS: collaboration on e-business standards; ISO TC184/SC4: the host of the ISO 15926 standard.

  Read more →
• 

  Regulation of artificial intelligence in the United States

  The United States federal government and state governments have developed some regulation of artificial intelligence, including executive orders, federal laws, and state laws. Federal agencies have also developed some sector-specific regulations related to AI. At the federal level, the Biden administration released an October 2023 executive order about AI safety and security, Executive Order 14110, with directives related to AI development and deployment. President Trump revoked that executive order in January 2025 and issued Executive Order 14179. In December 2025, President Trump signed Executive Order 14365, an executive order directing federal agencies to develop a unified national approach to AI policy, evaluate state AI laws for potential conflicts, challenge them through legal action, and condition certain federal funding on state compliance, while exempting state laws related to child safety, data center infrastructure, and state government procurement. In 2025, Congress passed legislation targeting AI-generated deepfakes, the TAKE IT DOWN Act. Several U.S. states have enacted laws related to artificial intelligence. Some are already in effect, including in California. Other states have AI-related legislation coming into effect in 2026 and 2027. In 2025 and 2026, the Trump administration mentioned the patchwork nature of state legislation as a motivation for its push for unified national legislation regulating AI. The administration has criticized state lawmakers, threatened to sue states, and issued letters to discourage them from regulating AI companies and products; some states have continued to propose and enact related laws. Discussions about regulating AI have included topics such as the timeliness of regulating AI, the nature of the federal regulatory framework to govern and promote AI, including what agency should lead, the regulatory and governing powers of that agency, and how to update regulations in the face of rapidly changing technology, as well as the roles of state governments and courts. == Federal government == === Obama administration (2009–2017) === As early as 2016, the Obama administration had begun to focus on the risks and regulations for artificial intelligence. In an October 2016 report titled Preparing For the Future of Artificial Intelligence, the National Science and Technology Council set a precedent to allow researchers to continue to develop new AI technologies with few restrictions. The report stated that "the approach to regulation of AI-enabled products to protect public safety should be informed by assessment of the aspects of risk". The first National Artificial Intelligence Research And Development Strategic Plan was published in October 2016. === First Trump administration (2017–2021) === On August 13, 2018, Section 1051 of the Fiscal Year 2019 John S. McCain National Defense Authorization Act (P.L. 115-232) established the National Security Commission on Artificial Intelligence "to consider the methods and means necessary to advance the development of artificial intelligence, machine learning, and associated technologies to comprehensively address the national security and defense needs of the United States." Steering on regulating security-related AI is provided by the National Security Commission on Artificial Intelligence. The Artificial Intelligence Initiative Act (S.1558) is a proposed bill that would establish a federal initiative designed to accelerate research and development on AI for, inter alia, the economic and national security of the United States. On January 7, 2019, following an Executive Order on Maintaining American Leadership in Artificial Intelligence, the White House's Office of Science and Technology Policy released a draft Guidance for Regulation of Artificial Intelligence Applications, which includes ten principles for United States agencies when deciding whether and how to regulate AI. In response, the National Institute of Standards and Technology released a position paper, and the Defense Innovation Board issued recommendations on the ethical use of AI. A year later, the administration called for comments on regulation in another draft of its Guidance for Regulation of Artificial Intelligence Applications. Other specific agencies working on the regulation of AI included the Food and Drug Administration, which created pathways to regulate the incorporation of AI in medical imaging. The National Science and Technology Council also published an updated National Artificial Intelligence Research and Development Strategic Plan in 2019, which received public scrutiny and recommendations to further improve it towards enabling Trustworthy AI. === Biden administration (2021–2025) === In March 2021, the National Security Commission on Artificial Intelligence released their final report. In the report, they stated, "Advances in AI, including the mastery of more general AI capabilities along one or more dimensions, will likely provide new capabilities and applications. Some of these advances could lead to inflection points or leaps in capabilities. Such advances may also introduce new concerns and risks and the need for new policies, recommendations, and technical advances to assure that systems are aligned with goals and values, including safety, robustness and trustworthiness." In June 2022, Senators Rob Portman and Gary Peters introduced the Global Catastrophic Risk Management Act. The bipartisan bill "would also help counter the risk of artificial intelligence... from being abused in ways that may pose a catastrophic risk". On October 4, 2022, President Joe Biden unveiled a new AI Bill of Rights, which outlines five protections Americans should have in the AI age: 1. Safe and Effective Systems, 2. Algorithmic Discrimination Protection, 3.Data Privacy, 4. Notice and Explanation, and 5. Human Alternatives, Consideration, and Fallback. The bill was formally published in October 2022 by the Office of Science and Technology Policy (OSTP), a U.S. government office that advises the President on science and technology policy matters. In July 2023, the Biden administration secured voluntary commitments from seven companies – Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI – to manage the risks associated with AI. The companies committed to ensure AI products undergo both internal and external security testing before public release; to share information on the management of AI risks with the industry, governments, civil society, and academia; to prioritize cybersecurity and protect proprietary AI system components; to develop mechanisms to inform users when content is AI-generated, such as watermarking; to publicly report on their AI systems' capabilities, limitations, and areas of use; to prioritize research on societal risks posed by AI, including bias, discrimination, and privacy concerns; and to develop AI systems to address societal challenges, ranging from cancer prevention to climate change mitigation. In September 2023, eight additional companies – Adobe, Cohere, IBM, Nvidia, Palantir, Salesforce, Scale AI, and Stability AI – subscribed to these voluntary commitments. In January 2023, the National Institute of Standards and Technology (NIST) released the Artificial Intelligence Risk Management Framework (AI RMF 1.0), providing voluntary guidance for organizations to identify, assess, and manage risks associated with AI systems. The Biden administration, in October 2023 signaled that they would release an executive order leveraging the federal government's purchasing power to shape AI regulations, hinting at a proactive governmental stance in regulating AI technologies. On October 30, 2023, President Biden released Executive Order 14110 on Safe, Secure, and Trustworthy Artificial Intelligence. The Executive Order includes directives on standards for critical infrastructure, AI-enhanced cybersecurity, and federally funded biological synthesis projects. The Executive Order provides the authority to various agencies and departments of the US government, including the Energy and Defense departments, to apply existing consumer protection laws to AI development. The Executive Order builds on the Administration's earlier agreements with AI companies to instate new initiatives to "red-team" or stress-test AI dual-use foundation models, especially those that have the potential to pose security risks, with data and results shared with the federal government. The Executive Order also recognizes AI's social challenges, and calls for companies building AI dual-use foundation models to be wary of these societal problems. For example, the Executive Order states that AI should not "worsen job quality", and should not "cause labor-force disruptions". Additionally, Biden's Executive Order mandates that AI must "advance equity and civil rights", and cannot disadvantage marginalized groups. It also called for foundation models to include "watermarks" to help the publi

  Read more →
• 

  Scrolling

  In computer displays, filmmaking, television production, video games and other kinetic displays, scrolling is sliding text, images or video across a monitor or display, vertically or horizontally. "Scrolling," as such, does not change the layout of the text or pictures but moves (pans or tilts) the user's view across what is apparently a larger image that is not wholly seen. A common television and movie special effect is to scroll credits, while leaving the background stationary. Scrolling may take place completely without user intervention (as in film credits) or, on an interactive device, be triggered by touchscreen or a keypress and continue without further intervention until a further user action, or be entirely controlled by input devices. Scrolling may take place in discrete increments (perhaps one or a few lines of text at a time), or continuously (smooth scrolling). Frame rate is the speed at which an entire image is redisplayed. It is related to scrolling in that changes to text and image position can only happen as often as the image can be redisplayed. When frame rate is a limiting factor, one smooth scrolling technique is to blur images during movement that would otherwise appear to "jump". == Computing == === Implementation === Scrolling is often carried out on a computer by the CPU (software scrolling) or by a graphics processor. Some systems feature hardware scrolling, where an image may be offset as it is displayed, without any frame buffer manipulation (see also hardware windowing). This was especially common in 8 and 16bit video game consoles. === UI paradigms === In a WIMP-style graphical user interface (GUI), user-controlled scrolling is carried out by manipulating a scrollbar with a mouse, or using keyboard shortcuts, often the arrow keys. Scrolling is often supported by text user interfaces and command line interfaces. Older computer terminals changed the entire contents of the display one screenful ("page") at a time; this paging mode requires fewer resources than scrolling. Scrolling displays often also support page mode. Typically certain keys or key combinations page up or down; on PC-compatible keyboards the page up and page down keys or the space bar are used; earlier computers often used control key combinations. Some computer mice have a scroll wheel, which scrolls the display, often vertically, when rolled; others have scroll balls or tilt wheels which allow both vertical and horizontal scrolling. Some software supports other ways of scrolling. Adobe Reader has a mode identified by a small hand icon ("hand tool") on the document, which can then be dragged by clicking on it and moving the mouse as if sliding a large sheet of paper. When this feature is implemented on a touchscreen it is called kinetic scrolling. Touch-screens often use inertial scrolling, in which the scrolling motion of an object continues in a decaying fashion after release of the touch, simulating the appearance of an object with inertia. An early implementation of such behavior was in the "Star7" PDA of Sun Microsystems ca. 1991–1992. Scrolling can be controlled in other software-dependent ways by a PC mouse. Some scroll wheels can be pressed down, functioning like a button. Depending on the software, this allows both horizontal and vertical scrolling by dragging in the direction desired; when the mouse is moved to the original position, scrolling stops. A few scroll wheels can also be tilted, scrolling horizontally in one direction until released. On touchscreen devices, scrolling is a multi-touch gesture, done by swiping a finger on the screen vertically in the direction opposite to where the user wants to scroll to. If any content is too wide to fit on a display, horizontal scrolling is required to view all of it. In applications such as graphics and spreadsheets there is often more content than can fit either the width or the height of the screen at a comfortable scale, and scrolling in both directions is necessary. === Infinite scrolling === In contrast to material divided into discrete pages, the web design approach of infinite scrolling dynamically adds new material to the user display, leading to a continuous, apparently bottomless or endless scrolling experience. === Text === In languages written horizontally, such as most Western languages, text documents longer than will fit on the screen are often displayed wrapped and sized to fit the screen width, and scrolled vertically to bring desired content into view. It is possible to display lines too long to fit the display without wrapping, scrolling horizontally to view each entire line. However, this requires inconvenient constant line-by-line scrolling, while vertical scrolling is only needed after reading a full screenful. Software such as word processors and web browsers normally uses word-wrapping to display as many words in a single line as will fit the width of the screen or window or, for text organised in columns, each column. === Demos === Scrolling texts, also referred to as scrolltexts or scrollers, played an important part in the birth of the computer demo culture. The software crackers often used their deep knowledge of computer platforms to transform the information that accompanied their releases into crack intros. The sole role of these intros was to scroll the text on the screen in an impressive way. == Film and television == Scrolling is commonly used to display the credits at the end of films and television programs. Scrolling is often used in the form of a news ticker towards the bottom of the picture for content such as television news, scrolling sideways across the screen, delivering short-form content. In the dynamic layout of kinetic typography, scrolling typography can scroll across the flat screen, or can appear to recede or advance. An iconic example is the Star Wars opening crawl inspired by the Flash Gordon serials. == Video games == In computer and video games, scrolling of a playing field allows the player to control an object in a large contiguous area. Early examples of this method include Taito's 1974 vertical-scrolling racing video game Speed Race, Sega's 1976 forward-scrolling racing games Moto-Cross (Fonz) and Road Race, and Super Bug. Previously the flip-screen method was used to indicate moving backgrounds. The Namco Galaxian arcade system board introduced with Galaxian in 1979 pioneered a sprite system that animated pre-loaded sprites over a scrolling background, which became the basis for Nintendo's Radar Scope and Donkey Kong arcade hardware and home consoles such as the Nintendo Entertainment System. Parallax scrolling, which was first featured in Moon Patrol, involves several semi-transparent layers (called playfields), which scroll on top of each other at varying rates in order to give an early pseudo-3D illusion of depth. Belt scrolling is a method used in side-scrolling beat 'em up games with a downward camera angle where players can move up and down in addition to left and right. == Studies == A 1993 article by George Fitzmaurice studied spatially aware palmtop computers. These devices had a 3D sensor, and moving the device caused the contents to move as if the contents were fixed in place. This interaction could be referred to as “moving to scroll.” Also, if the user moved the device away from their body, they would zoom in; conversely, the device would zoom out if the user pulled the device closer to them. Smartphone cameras and “optical flow” image analysis utilize this technique nowadays. A 1996 research paper by Jun Rekimoto analyzed tilting operations as scrolling techniques on small screen interfaces. Users could not only tilt to scroll, but also tilt to select menu items. These techniques proved especially useful for field workers, since they only needed to hold and control the device with one hand. A study from 2013 by Selina Sharmin, Oleg Špakov, and Kari-Jouko Räihä explored the action of reading text on a screen while the text auto-scrolls based on the user's eye tracking patterns. The control group simply read text on a screen and manually scrolled. The study found that participants preferred to read primarily at the top of the screen, so the screen scrolled down whenever participants’ eyes began to look toward the bottom of the screen. This auto-scrolling caused no statistically significant difference in reading speed or performance. An undated study occurring during or after 2010 by Dede Frederick, James Mohler, Mihaela Vorvoreanu, and Ronald Glotzbach noted that parallax scrolling "may cause certain people to experience nausea."

  Read more →
• 

  Project Joshua Blue

  Joshua Blue is a project under development by IBM that focuses on advancing the artificial intelligence field by designing and programming computers to emulate human mental functions. == Goals == According to researchers at IBM's Thomas J. Watson Research Center, the main goal of Joshua Blue is "to achieve cognitive flexibility that approaches human functioning". In short, IBM is aiming to design Joshua Blue to 'think like a human', mainly in terms of emotional thought. == How it will work == A model of Joshua Blue's learning pattern has been created. Similar to how young children learn human traits through interacting with their surroundings, Joshua Blue will acquire knowledge through external stimuli present in its environment. IBM believes that if computers evolve to learn in this way and then comprehend and analyze the knowledge gained using reason, computers could begin to possess a "mind", of sorts, capable of demonstrating complex social behaviors similar to those of humans. Thus far, IBM has revealed that Joshua Blue will be a computer with a network of wires and input nodes that function as a computer nervous system. This nervous system will be used by Joshua Blue to perceive affect or personal emotional feelings. Not only will this network of input nodes help Joshua Blue discover things physically, but it will also allow Joshua Blue to interpret the significance of events. The input nodes, or proprioceptors, will enable Joshua Blue to be aware of things that happen around itself, as well as recognize and attach meaning to the emotional effect produced by interacting with an object in a certain way. In addition, Joshua Blue's proprioceptors will function as pain and pleasure sensors, allowing Joshua Blue to employ a similar "reward and punishment" system that humans use to form behaviors.

  Read more →
• 

  Catastrophic interference

  Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to abruptly and drastically forget previously learned information upon learning new information. Neural networks are an important part of the connectionist approach to cognitive science. The issue of catastrophic interference when modeling human memory with connectionist models was originally brought to the attention of the scientific community by research from McCloskey and Cohen (1989), and Ratcliff (1990). It is a radical manifestation of the 'sensitivity-stability' dilemma or the 'stability-plasticity' dilemma. Specifically, these problems refer to the challenge of making an artificial neural network that is sensitive to, but not disrupted by, new information. Lookup tables and connectionist networks lie on the opposite sides of the stability plasticity spectrum. The former remains completely stable in the presence of new information but lacks the ability to generalize, i.e. infer general principles, from new inputs. On the other hand, connectionist networks like the standard backpropagation network can generalize to unseen inputs, but they are sensitive to new information. Backpropagation models can be analogized to human memory insofar as they have a similar ability to generalize, but these networks often exhibit less stability than human memory. Notably, these backpropagation networks are susceptible to catastrophic interference. This is an issue when modelling human memory, because unlike these networks, humans typically do not show catastrophic forgetting. == Discovery == The term catastrophic interference was originally coined by McCloskey and Cohen (1989) but was also brought to the attention of the scientific community by research from Ratcliff (1990). === The Sequential Learning Problem: McCloskey and Cohen (1989) === McCloskey and Cohen (1989) noted the problem of catastrophic interference during two different experiments with backpropagation neural network modelling. Experiment 1: Learning the ones and twos addition facts In their first experiment they trained a standard backpropagation neural network on a single training set consisting of 17 single-digit ones problems (i.e., 1 + 1 through 9 + 1, and 1 + 2 through 1 + 9) until the network could represent and respond properly to all of them. The error between the actual output and the desired output steadily declined across training sessions, which reflected that the network learned to represent the target outputs better across trials. Next, they trained the network on a single training set consisting of 17 single-digit twos problems (i.e., 2 + 1 through 2 + 9, and 1 + 2 through 9 + 2) until the network could represent, respond properly to all of them. They noted that their procedure was similar to how a child would learn their addition facts. Following each learning trial on the twos facts, the network was tested for its knowledge on both the ones and twos addition facts. Like the ones facts, the twos facts were readily learned by the network. However, McCloskey and Cohen noted the network was no longer able to properly answer the ones addition problems even after one learning trial of the twos addition problems. The output pattern produced in response to the ones facts often resembled an output pattern for an incorrect number more closely than the output pattern for a correct number. This is considered to be a drastic amount of error. Furthermore, the problems 2+1 and 1+2, which were included in both training sets, even showed dramatic disruption during the first learning trials of the twos facts. Experiment 2: Replication of Barnes and Underwood (1959) study In their second connectionist model, McCloskey and Cohen attempted to replicate the study on retroactive interference in humans by Barnes and Underwood (1959). They trained the model on A-B and A-C lists and used a context pattern in the input vector (input pattern), to differentiate between the lists. Specifically the network was trained to respond with the right B response when shown the A stimulus and A-B context pattern and to respond with the correct C response when shown the A stimulus and the A-C context pattern. When the model was trained concurrently on the A-B and A-C items then the network readily learned all of the associations correctly. In sequential training the A-B list was trained first, followed by the A-C list. After each presentation of the A-C list, performance was measured for both the A-B and A-C lists. They found that the amount of training on the A-C list in Barnes and Underwood study that lead to 50% correct responses, lead to nearly 0% correct responses by the backpropagation network. Furthermore, they found that the network tended to show responses that looked like the C response pattern when the network was prompted to give the B response pattern. This indicated that the A-C list apparently had overwritten the A-B list. This could be likened to learning the word dog, followed by learning the word stool and then finding that you think of the word stool when presented with the word dog. McCloskey and Cohen tried to reduce interference through a number of manipulations including changing the number of hidden units, changing the value of the learning rate parameter, overtraining on the A-B list, freezing certain connection weights, changing target values 0 and 1 instead 0.1 and 0.9. However, none of these manipulations satisfactorily reduced the catastrophic interference exhibited by the networks. Overall, McCloskey and Cohen (1989) concluded that: at least some interference will occur whenever new learning alters the weights involved in representing old learning the greater the amount of new learning, the greater the disruption in old knowledge interference was catastrophic in the backpropagation networks when learning was sequential but not concurrent === Constraints Imposed by Learning and Forgetting Functions: Ratcliff (1990) === Ratcliff (1990) used multiple sets of backpropagation models applied to standard recognition memory procedures, in which the items were sequentially learned. After inspecting the recognition performance models he found two major problems: Well-learned information was catastrophically forgotten as new information was learned in both small and large backpropagation networks. Even one learning trial with new information resulted in a significant loss of the old information, paralleling the findings of McCloskey and Cohen (1989). Ratcliff also found that the resulting outputs were often a blend of the previous input and the new input. In larger networks, items learned in groups (e.g. AB then CD) were more resistant to forgetting than were items learned singly (e.g. A then B then C...). However, the forgetting for items learned in groups was still large. Adding new hidden units to the network did not reduce interference. Discrimination between the studied items and previously unseen items decreased as the network learned more. This finding contradicts studies on human memory, which indicated that discrimination increases with learning. Ratcliff attempted to alleviate this problem by adding 'response nodes' that would selectively respond to old and new inputs. However, this method did not work as these response nodes would become active for all inputs. A model which used a context pattern also failed to increase discrimination between new and old items. == Proposed solutions == The main cause of catastrophic interference seems to be overlap in the representations at the hidden layer of distributed neural networks. In a distributed representation, each input tends to create changes in the weights of many of the nodes. Catastrophic forgetting occurs because when many of the weights where "knowledge is stored" are changed, it is unlikely for prior knowledge to be kept intact. During sequential learning, the inputs become mixed, with the new inputs being superimposed on top of the old ones. Another way to conceptualize this is by visualizing learning as a movement through a weight space. This weight space can be likened to a spatial representation of all of the possible combinations of weights that the network could possess. When a network first learns to represent a set of patterns, it finds a point in the weight space that allows it to recognize all of those patterns. However, when the network then learns a new set of patterns, it will move to a place in the weight space for which the only concern is the recognition of the new patterns. To recognize both sets of patterns, the network must find a place in the weight space suitable for recognizing both the new and the old patterns. Below are a number of techniques which have empirical support in successfully reducing catastrophic interference in backpropagation neural networks: === Orthogonality === Many of the early techniques in reducing representational overlap involved making either the input vecto

  Read more →