Markovian discrimination is a class of spam filtering methods used in CRM114 and other spam filters to filter based on statistical patterns of transition probabilities between words or other lexical tokens in spam messages that would not be captured using simple bag-of-words naive Bayes spam filtering. == Markovian Discrimination vs. Bag-of-Words Discrimination == A bag-of-words model contains only a dictionary of legal words and their relative probabilities in spam and genuine messages. A Markovian model additionally includes the relative transition probabilities between words in spam and in genuine messages, where the relative transition probability is the likelihood that a given word will be written next, based on what the current word is. Put another way, a bag-of-words filter discriminates based on relative probabilities of single words alone regardless of phrase structure, while a Markovian word-based filter discriminates based on relative probabilities of either pairs of words, or, more commonly, short sequences of words. This allows the Markovian filter greater sensitivity to phrase structure. Neither naive Bayes nor Markovian filters are limited to the word level for tokenizing messages. They may also process letters, partial words, or phrases as tokens. In such cases, specific bag-of-words methods would correspond to general bag-of-tokens methods. Modelers can parameterize Markovian spam filters based on the relative probabilities of any such tokens' transitions appearing in spam or in legitimate messages. == Visible and Hidden Markov Models == There are two primary classes of Markov models, visible Markov models and hidden Markov models, which differ in whether the Markov chain generating token sequences is assumed to have its states fully determined by each generated token (the visible Markov models) or might also have additional state (the hidden Markov models). With a visible Markov model, each current token is modeled as if it contains the complete information about previous tokens of the message relevant to the probability of future tokens, whereas a hidden Markov model allows for more obscure conditional relationships. Since those more obscure conditional relationships are more typical of natural language messages including both genuine messages and spam, hidden Markov models are generally preferred over visible Markov models for spam filtering. Due to storage constraints, the most commonly employed model is a specific type of hidden Markov model known as a Markov random field, typically with a 'sliding window' or clique size ranging between four and six tokens.
Non-native speech database
A non-native speech database is a speech database of non-native pronunciations of English. Such databases are used in the development of: multilingual automatic speech recognition systems, text to speech systems, pronunciation trainers, and second language learning systems. == List == The actual table with information about the different databases is shown in Table 2. === Legend === In the table of non-native databases some abbreviations for language names are used. They are listed in Table 1. Table 2 gives the following information about each corpus: The name of the corpus, the institution where the corpus can be obtained, or at least further information should be available, the language which was actually spoken by the speakers, the number of speakers, the native language of the speakers, the total amount of non-native utterances the corpus contains, the duration in hours of the non-native part, the date of the first public reference to this corpus, some free text highlighting special aspects of this database and a reference to another publication. The reference in the last field is in most cases to the paper which is especially devoted to describe this corpus by the original collectors. In some cases it was not possible to identify such a paper. In these cases a paper is referenced which is using this corpus is. Some entries are left blank and others are marked with unknown. The difference here is that blank entries refer to attributes where the value is just not known. Unknown entries, however, indicate that no information about this attribute is available in the database itself. As an example, in the Jupiter weather database no information about the origin of the speakers is given. Therefore this data would be less useful for verifying accent detection or similar issues. Where possible, the name is a standard name of the corpus, for some of the smaller corpora, however, there was no established name and hence an identifier had to be created. In such cases, a combination of the institution and the collector of the database is used. In the case where the databases contain native and non-native speech, only attributes of the non-native part of the corpus are listed. Most of the corpora are collections of read speech. If the corpus instead consists either partly or completely of spontaneous utterances, this is mentioned in the Specials column.
Void Trilogy
The Void Trilogy is a space opera series by British author Peter F. Hamilton. The series is set in the same universe as The Commonwealth Saga, 1,200 years after the end of Judas Unchained. Peter F. Hamilton sold the American rights to the series to Random House. The series includes the following books: The Dreaming Void (2007) The Temporal Void (2008) The Evolutionary Void (2010) == Synopsis == === The Dreaming Void === What was formerly believed to be a supermassive black hole at the centre of the Milky Way is revealed to be an artificial construct, known as the Void. Inside, there is a strange universe where the laws of physics are very different from standard physics. It is slowly consuming the other stars of the galactic core—one day it will have devoured the entire galaxy. In AD 3320, a human member of the Commonwealth, Inigo, begins to have dreams of the wonderful existence inside the Void. His dreams inspire the disaffected, who desire to travel into the Void, where their every wish will be fulfilled. By AD 3456, the pseudo-religious Living Dream movement exceeds 5 billion members, organizing the followers into a powerful political force. Other star-faring species fear their migration will cause the Void to expand again thus devouring the galaxy. They are prepared to stop the pilgrimage fleet no matter what the cost. The Dreaming Void is broken into two distinct sections. The first follows Edeard, a young boy who lives inside the Void on a planet called Querencia, the subject of Inigo's dreams. Edeard, an orphan and apprentice, lives in Ashwell, a town in Rulan province. A gifted psychic, he is trained by Master Akeem in crafting and modding. Initially a loner, he comes to prominence in his village after designing an alternative pump mechanism for the local well. Unfortunately his luck changes for the worse after Ashwell is raided by bandits. Forced to flee, he joins the local caravan and travels to Makkathran, the capital of Querencia. In Makkathran, Edeard joins the constables and after a brutal couple of months in training, he graduates and is promoted to the commander of his Squad. He makes little progress battling the rigid and backward judicial system of Makkathran; his first real break is when his squad overcomes a trap set by the local gang, and Edeard walks on water chasing the leader of the gang. A testament to his growing psychic abilities, Edeard's stunt earns him the title of Waterwalker, and he becomes an instant star in Makkathran. The second section of The Dreaming Void is set back in the Commonwealth. Inigo, the first dreamer, and founder of Living Dream, has disappeared, leaving the 5 billion strong Living Dream movement in a state of flux. When Ethan, succeeding Inigo as the head of the movement, proclaims that the Living Dream will embark on a pilgrimage into the Void, the Commonwealth is thrown into a state of political chaos. Fearing that the human migration might cause the Void to expand (and in the process destroy whole systems or even the whole Galaxy) other spacefaring races such as the Raiel and Ocisen Empire are deeply concerned, with the latter threatening military action. This has left the Commonwealth government deeply divided, with the two largest factions in disagreement, the Accelerators faction/party supporting the pilgrimage and the Conservative faction opposing. As both parties are unable to solve the situation politically they have resolved to take matters into their own hands, with each party sending agents to further its interests. Aaron, a sleeper cell agent, is tasked with finding Inigo. He kidnaps and manipulates Corrie-Lyn, a former lover of Inigo and interrogates her for information. He also travels to Kuhmo (Inigo's homeworld) to get further information and robs Inigo's secure storage (a bank for memory). He eventually tracks Inigo to Hanko, a desolate and barren world. However, before Aaron can extract Inigo, Accelerator agents destroy Aaron's starship leaving him marooned on Hanko. Meanwhile, Accelerator agents make a deal with Ethan, agreeing to give the Living Dream movement Ultra Drives to power their ships. Accelerator plans are halted when the Delivery Man, a Conservative party agent, destroys valuable FTL Drive tech. Troblum, an Accelerator physicist, also defects, further slowing the Accelerators plans. === The Temporal Void === The Temporal Void picks up after The Dreaming Void. The Intersolar Commonwealth faces mounting turmoil as the deadline for Living Dream's Pilgrimage into the Void approaches. An Ocisen Empire fleet advances on a mission of genocide, while an internecine war erupts among post-human factions over humanity's future. Amidst the chaos, investigator Paula Myo struggles to counter the increasingly desperate actions of various agents and factions. Relentless in her pursuit, she contends with adversaries from her distant past and colleagues of uncertain loyalty, all while racing against time. At the center of the unfolding crisis is Edeard the Waterwalker, a figure from the distant past who lived deep within the Void. As the messiah of Living Dream, his life—broadcast through visions—captivates and inspires billions. His story fuels the Pilgrimage's momentum, a force seemingly impossible to stop. As Edeard approaches his ultimate victory, the true nature of the Void is finally revealed. === The Evolutionary Void === The Evolutionary Void picks up after The Temporal Void. Exposed as the Second Dreamer, Araminta has become the target of a galaxy-wide search by government agent Paula Myo and the psychopath known as the Cat, along with others equally determined to prevent, or facilitate, the pilgrimage of the Living Dream cult into the heart of the Void. An indestructible microuniverse, the Void may contain paradise, as the cultists believe, but it is also a deadly threat. For the miraculous reality that exists inside its boundaries demands energy, energy drawn from everything outside those boundaries: from planets, stars, galaxies, and everything that lives, for the Pilgrimage will trigger a super-massive expansion of the Void. Meanwhile, the parallel story of Edeard, the Waterwalker, as told through a series of dreams communicated to the gaiafield via Inigo, the First Dreamer, continues to unfold. But the inspirational tale of this idealistic young man takes a darker and more troubling turn as he finds himself faced with powerful new enemies, and temptations more powerful still, to reach fulfilment in the end. Named a Silfen Friend like her ancestress Mellanie, Araminta chooses to face her unwanted responsibilities, with no guarantee of success or survival. She takes on the role of Second Dreamer to lead the first wave of Living Dream, 24 million people, into the Void, leaving everyone confused and lost by her actions. However, in actuality, she is playing a double game. Using her original body to lead the Living Dream as a diversion, she borrows one of her fiancé's (Mr. Bovey) bodies to set out to destroy the Void. She is able to connect with a Skylord and travel the Silfen Paths. With time running out, a repentant Inigo decides to release Edeard's final dream whose message is scarcely less dangerous than the pilgrimage promises to be, where perfection is achieved, so that nothing else is left to strive for and the human race in the Void has started to devolve. He goes to the Spike to meet Ozzie and stays there to meet with Araminta, who is using one of her fiancé's bodies, and Oscar. Third Dreamer Gore Burnelli has a plan to reason with the Heart, the core of the Void. He secures the help of the Delivery Man and travels to the Anomine homeworld to retrieve the mechanism that allowed them to go post-physical. He is able to connect with Justine, his daughter, who is currently in the Void, by way of Dreams. The monomaniacal Ilanthe, leader of the breakaway Accelerator Faction, seeks dominion in the Void. It is not Fusion with the Void to attain post-physical status that she wants, but to have control over everything. Using Dark Fortress technology, she sets up a barrier around the Sol system which leaves ANA and the deterrence fleet trapped inside. It is this technology which she has equipped the ships travelling to the Void with, the ability to create a forcefield which the Warrior Raiel cannot penetrate. == Technology == The Commonwealth uses a number of advanced technologies. In the early days of the Commonwealth, humans used static and permanently opened wormholes to travel from planet to planet. However, after the events of the Starflyer War (detailed in the Commonwealth Saga), the CST corporation's monopoly on space travel was ended. With the advent of wormholes that could wrap around ships, the Commonwealth saw a shift from wormholes to spaceships. Another development in the Commonwealth is the gaiafield. Developed by Ozzie Issac in AD 3000, the gaiafield is based on Silfen technology; when Ozzie was named a friend of the Silfen during the Starflye
Oxa
Oxa (formerly Oxbotica) is an autonomous vehicle software company, headquartered in Oxfordshire, England, and founded by Paul Newman and Ingmar Posner. == History == In 2013, Newman and Posner led the RobotCar UK project as part of Oxford University's Department of Engineering Science Mobile Robotics Group. RobotCar became the first autonomous vehicle on UK roads. In 2014, the pair used the newly developed technology to found Oxbotica. Oxbotica has raised over $18 million to date and is backed by the IP Group, Parkwalk Advisors and AXA XL. In 2018, Uber's former EMEA business head, Fraser Robinson, was appointed to the board of directors. In May 2019, Ozgur Tohumcu replaced Dr Graeme Smith as Oxbotica's CEO. Also in 2019, the company opened an office in Toronto, Canada. In January 2021, Oxbotica announced it had raised $47 million in a Series B round. In August 2021, the company achieved a safety landmark as the first company to have its autonomy safety case assessed by BSI (British Standards Institution) against the requirements of the UK Code of Practice 2019, PAS 1881:2020 and PAS 1883:2020, certifying the safety conformity of its autonomous vehicle trials and testing. The assessment was completed as part of Project Endeavour, the UK's first multi-city demonstration of autonomous vehicle services and capability. In December 2021, Gavin Jackson was named CEO. In January 2023, the company raised $140 million in a Series C round. In May 2023, the company changed its name to Oxa. Oxa raised $103 million (£77 million) in March 2026, including $50 million from the UK National Wealth Fund. Nvidia's venture capital division, NVentures, also invested in the Series D funding round, along with existing Oxa shareholders IP Group, Australian pension fund Hostplus, and BP Ventures, a division of the UK oil company. == Technology == Oxa designs software and hardware for the conversion of industrial vehicles into autonomous ones. Its full stack, end-to-end Universal Autonomy software is both vehicle and platform-agnostic, with no dependence on external infrastructure such as GPS. It can be deployed in any environment and on any terrain. In addition to underground uses, the technology is also useful in natural canyons and forests, where GPS signals are weak or non-existent, but also in "urban canyons" — cities with tall buildings that obstruct GPS signals for proper navigation. == Public deployments == The LUTZ Pathfinder pod had its first public demonstration in February 2015 in Milton Keynes. The Government-funded project was designed to ensure that autonomous vehicles would comply with the Highway Code. The pod featured autonomous control software from Oxbotica, including 19 sensors, cameras, radar and Lidar. As part of the GATEway Project in 2017, Oxbotica trialled seven autonomous shuttle buses in Greenwich, navigating a two-mile riverside path near London's O2 Arena on a route that is also used by pedestrians and cyclists. Oxbotica ran the UK's first trial of autonomous grocery deliveries that year, with British online supermarket Ocado in London, as the next step in the GATEway Project. In 2018, Oxbotica deployed its autonomous vehicle software at London's Gatwick Airport, which subsequently became the first airport in the world to trial an autonomous shuttle service. The electric-powered vehicles transported staff via airside roads between the airport's North and South terminals. An airside trial of Oxbotica's autonomous driving technology was then successfully completed at Heathrow Airport in partnership with IAG Cargo, the first airside trial of an autonomous vehicle at a UK airport. The Oxbotica-designed CargoPod ran autonomously along a cargo route around the airside perimeter for three weeks. As part of the UK Centre for Connected and Autonomous Vehicles-funded DRIVEN project, Oxbotica is developing and deploying a fleet of Ford Fusion autonomous vehicles running in both London and Oxford on public roads, and in conjunction with its consortium partners, running real-time insurance. AXA XL is partnering with Oxbotica on the development of smart insurance products using Oxbotica's autonomy technology to improve road safety. In 2018, Oxbotica announced a partnership with London private taxi firm Addison Lee to develop and deploy autonomous taxis in the city of London by 2021. A 3D street mapping exercise was conducted in London's Canary Wharf. In 2019, Oxbotica deployed a fleet of their autonomous technology within Ford Mondeo cars on public roads in Stratford, London to test their use in city environments. The £13.2 million project is in collaboration with The DRIVEN Project to develop self-driving cars. == Awards == 2019 Royal Academy of Engineering Silver Medal - Paul Newman 2017 Financial Times ArcelorMittal Boldness in Business Award Barclays Award for Innovation 2016 Frost & Sullivan Award, Technology Leadership for Autonomous Driving Software
Overwatch
Overwatch (abbreviated as OW) is a multimedia franchise centered on a series of multiplayer first-person shooter (FPS) video games developed by Blizzard Entertainment. Overwatch was released in 2016. Overwatch 2 was released in 2022 and the original game was taken offline upon its release, though Blizzard renamed it back to Overwatch in 2026. Overwatch features hero-based combat between two teams of players fighting over various objectives, along with other traditional gameplay modes. Released in 2016, Overwatch lacked a traditional story mode. Instead, Blizzard employed a transmedia storytelling strategy to disseminate lore regarding the game's characters, releasing comics and other literary media, as well as animated media that includes short films. The game enjoyed both critical and commercial success, and garnered a devoted following. The fan community around the franchise has produced a large amount of content including art, cosplay, fan fiction, anime-influenced music videos, Internet memes, and pornography. Blizzard helped launch and promote an esports scene surrounding the game, including an annual Overwatch World Cup, Overwatch League a minor league, and the Overwatch Champions Series which borrowed elements found in traditional American sports leagues. == Gameplay == Both games in the Overwatch series are team-based hero shooters. Players select a hero character from a large roster (52 as of Season 2), divided among three class types. These are: Tanks, who have higher health and generally meant to help protect their teammates from damage, but are larger and easier to hit; Damage, who act as the team's offensive leads; and Support, who heal, provide buffs for teammates, or de-buff the opposing team. Each role also features sub-roles with extra passives. These sub-roles include 'Initiator', 'Stalwart', and 'Bruiser' for Tank. 'Specialist', 'Flanker', 'Recon', and 'Sharpshooter' for Damage. 'Medic', 'Tactician', and 'Survivor' for Support. Players are generally free to change to different heroes while inside their spawn room during the course of a match in response to the current tactics employed by other players. As of the development of Overwatch 2, a standard game features one tank player, two damage players and two support players, a change from having two of each class in its predecessor. Players choose their class before the match, and can only pick characters within that class for the duration of the game. There are different styles of game modes, however, that allow players to choose characters from any class throughout the game. Each hero has a skill kit that includes a primary attack, active skills that require a cooldown period before they can be used again, passive skills that remain active at all times, and an Ultimate skill that can only be used once they fill their Ultimate meter either by damaging opponents, mitigating damage, healing teammates or by passively generating it over time. An update in 2025 saw each hero receive a total of four unique abilities known as perks. Each hero has two minor and two major perks; minor perks consist of smaller changes to a hero's kit, while major perks are intended to affect the match more significantly. At the beginning of each match, all heroes are set to level 1 for each player. As the match progresses, players can individually level up their respective heroes, minor perks are unlocked at level 2, and major perks are unlocked at the maximum level 3. When perks become available, players may only select one of each type of perk; a selected perk becomes irreversibly attached to the current hero for the remainder of the match. If a player switches to another hero mid-match, the previously selected hero retains their level and perk progress. Game types of Overwatch are split between standard matches, competitive play, custom games, and arcade modes. Standard matches have matchmaking based loosely on the player's skill level as measured by the game. Competitive mode uses more strict matchmaking based on a player's current rank on the competitive ladder, with their rank increasing or decreasing when they win or lose a game, respectively. Arcade modes do not use matchmaking and are generally more experimental modes compared to standard and competitive modes. Custom games are created via the workshop and can be utilised to make game modes that are very different from the base game. The workshop, is the software in Overwatch which creates the game using either presets and settings or rules and conditions made by code. These game modes can be published directly onto Overwatch’s custom browse tab or shared off platform using a 5 digit alphanumeric code. Standard and competitive game modes are randomly selected at the start of each match, and are objective based, requiring teams to control a fixed objective point for a duration of time, or escort a payload to a target zone before match time expires. These modes include: Assault (introduced in Overwatch): Also known as 2 Capture Points (or 2CP), Assault has the attacking team tasked with capturing two target points in sequence on the map, while the defending team must stop them. Assault-style maps were removed from main gameplay rotation after Overwatch 2 released but available in the game's arcade mode. It is still available in the game's custom game modes. Since Season 2, Assault-style maps are available in Arcade Mode daily routines. Escort (introduced in Overwatch): Also known as "Payload" by the community, The attacking team is tasked with escorting a payload to a certain delivery point before time runs out, while the defending team must stop them. The payload vehicle moves along a fixed track when any player on the attacking team is close to it, increasing in speed if multiple attackers are present, the increase capping at 3, but will stop if a defending player is nearby; should no attacker be near the vehicle, it will start to move backwards along the track. The payload will also heal any attacking players by 10 health per second while they are near the payload. Passing specific checkpoints will extend the match time and prevent the payload from moving backwards from that point. Hybrid (Assault/Escort) (introduced in Overwatch): The attacking team has to capture the payload (as if it were a target point from Assault) and escort it to its destination, while the defending team tries to hold them back. Control (introduced in Overwatch): Each team tries to capture and maintain a common control point until their capture percentage reaches 100%. This game mode is played in a best-of-three format. Control maps are laid out in a symmetric fashion so no team has an intrinsic position advantage. Push (introduced in Overwatch 2's launch): Each team attempts to secure control of a large robot that pushes one of two barriers to the opposing team's side of the map, whilst being escorted by at least one team member, stopping when enemy players are nearby, similar to the payload movement system in Escort. The team that pushes the payload fully to the other side, or furthest into the enemy territory before the time runs out, wins the match. Flashpoint (introduced in Overwatch 2 in 2023): Similar to Control, each team attempts to capture and maintain a common control point until their capture percentage reaches 100%. This game mode takes place on significantly larger maps with five separate control points, which take a shorter amount of time to capture as compared to a standard Control map. A central control point is always activated first; after it is secured by one team, the remaining four are activated in a random order. The first team to secure three control points wins. Clash (introduced in Overwatch 2 in 2024): Clash maps feature symmetrical maps with five control points. Teams initially vie for control of the central point, with the winning team progressing to the next control point, towards the opponent's base. Opponents can push back by winning control points and shifting the next point away from their base. If a team captures the point closest to the opponent's base, they win. Otherwise the match plays out until one team wins control five times. Arcade modes may include variations of the above modes with experimental rules, and can also include modes like Deathmatch and Capture the Flag. Other common arcade modes include: Elimination (introduced in Overwatch in 2016): Two teams face off in a series of rounds, attempting to wipe out the other team; once a player is killed they remain out of the game until the next round, though they can be revived by Mercy's 'Resurrect' ability. If no team has won a round by a certain time, then the winners are decided by the team that can first take a neutral control point. Players cannot change heroes until the next round. Some of these can be played in "lockout" mode, in which the heroes selected by the winning team for a round are "locked" and cannot be selected in future rounds. Total Mayhem (i
FuseBase
FuseBase (previously Nimbus Note and Nimbus Platform) is a B2B SaaS platform. It is among the first to support the Model Context Protocol (MCP), an open standard enabling seamless integration of AI agents with external tools, systems, and data sources. == History == The platform was founded in 2014 as Nimbus Note, the platform started as a cross-platform note-taking and information management tool. As it evolved into Nimbus Platform, it added project management and client portal capabilities. In 2023, the company rebranded as FuseBase, pivoting to connect and automate both internal and external collaboration through AI Agents and cutting-edge protocol adoption like MCP. At the same time, FuseBase was named Product of the Year on Product Hunt. == Technical overview == The platform integrates the Model Context Protocol (MCP), an open-source framework created by Anthropic. MCP allows AI models to securely access and interact with external data, tools, and systems. This enables FuseBase AI Agents to gather relevant context, perform actions, and provide more advanced automation.
Neural architecture search
Neural architecture search (NAS) is a technique for automating the design of artificial neural networks (ANN), a widely used model in the field of machine learning. NAS has been used to design networks that are on par with or outperform hand-designed architectures. Methods for NAS can be categorized according to the search space, search strategy and performance estimation strategy used: The search space defines the type(s) of ANN that can be designed and optimized. The search strategy defines the approach used to explore the search space. The performance estimation strategy evaluates the performance of a possible ANN from its design (without constructing and training it). NAS is closely related to hyperparameter optimization and meta-learning and is a subfield of automated machine learning (AutoML). == Reinforcement learning == Reinforcement learning (RL) can underpin a NAS search strategy. Barret Zoph and Quoc Viet Le applied NAS with RL targeting the CIFAR-10 dataset and achieved a network architecture that rivals the best manually-designed architecture for accuracy, with an error rate of 3.65, 0.09 percent better and 1.05x faster than a related hand-designed model. On the Penn Treebank dataset, that model composed a recurrent cell that outperforms LSTM, reaching a test set perplexity of 62.4, or 3.6 perplexity better than the prior leading system. On the PTB character language modeling task it achieved bits per character of 1.214. Learning a model architecture directly on a large dataset can be a lengthy process. NASNet addressed this issue by transferring a building block designed for a small dataset to a larger dataset. The design was constrained to use two types of convolutional cells to return feature maps that serve two main functions when convoluting an input feature map: normal cells that return maps of the same extent (height and width) and reduction cells in which the returned feature map height and width is reduced by a factor of two. For the reduction cell, the initial operation applied to the cell's inputs uses a stride of two (to reduce the height and width). The learned aspect of the design included elements such as which lower layer(s) each higher layer took as input, the transformations applied at that layer and to merge multiple outputs at each layer. In the studied example, the best convolutional layer (or "cell") was designed for the CIFAR-10 dataset and then applied to the ImageNet dataset by stacking copies of this cell, each with its own parameters. The approach yielded accuracy of 82.7% top-1 and 96.2% top-5. This exceeded the best human-invented architectures at a cost of 9 billion fewer FLOPS—a reduction of 28%. The system continued to exceed the manually-designed alternative at varying computation levels. The image features learned from image classification can be transferred to other computer vision problems. E.g., for object detection, the learned cells integrated with the Faster-RCNN framework improved performance by 4.0% on the COCO dataset. In the so-called Efficient Neural Architecture Search (ENAS), a controller discovers architectures by learning to search for an optimal subgraph within a large graph. The controller is trained with policy gradient to select a subgraph that maximizes the validation set's expected reward. The model corresponding to the subgraph is trained to minimize a canonical cross entropy loss. Multiple child models share parameters, ENAS requires fewer GPU-hours than other approaches and 1000-fold less than "standard" NAS. On CIFAR-10, the ENAS design achieved a test error of 2.89%, comparable to NASNet. On Penn Treebank, the ENAS design reached test perplexity of 55.8. == Evolution == An alternative approach to NAS is based on evolutionary algorithms, which has been employed by several groups. An Evolutionary Algorithm for Neural Architecture Search generally performs the following procedure. First a pool consisting of different candidate architectures along with their validation scores (fitness) is initialised. At each step the architectures in the candidate pool are mutated (e.g.: 3x3 convolution instead of a 5x5 convolution). Next the new architectures are trained from scratch for a few epochs and their validation scores are obtained. This is followed by replacing the lowest scoring architectures in the candidate pool with the better, newer architectures. This procedure is repeated multiple times and thus the candidate pool is refined over time. Mutations in the context of evolving ANNs are operations such as adding or removing a layer, which include changing the type of a layer (e.g., from convolution to pooling), changing the hyperparameters of a layer, or changing the training hyperparameters. On CIFAR-10 and ImageNet, evolution and RL performed comparably, while both slightly outperformed random search. == Bayesian optimization == Bayesian Optimization (BO), which has proven to be an efficient method for hyperparameter optimization, can also be applied to NAS. In this context, the objective function maps an architecture to its validation error after being trained for a number of epochs. At each iteration, BO uses a surrogate to model this objective function based on previously obtained architectures and their validation errors. One then chooses the next architecture to evaluate by maximizing an acquisition function, such as expected improvement, which provides a balance between exploration and exploitation. Acquisition function maximization and objective function evaluation are often computationally expensive for NAS, and make the application of BO challenging in this context. Recently, BANANAS has achieved promising results in this direction by introducing a high-performing instantiation of BO coupled to a neural predictor. == Hill-climbing == Another group used a hill climbing procedure that applies network morphisms, followed by short cosine-annealing optimization runs. The approach yielded competitive results, requiring resources on the same order of magnitude as training a single network. E.g., on CIFAR-10, the method designed and trained a network with an error rate below 5% in 12 hours on a single GPU. == Multi-objective search == While most approaches solely focus on finding architecture with maximal predictive performance, for most practical applications other objectives are relevant, such as memory consumption, model size or inference time (i.e., the time required to obtain a prediction). Because of that, researchers created a multi-objective search. LEMONADE is an evolutionary algorithm that adopted Lamarckism to efficiently optimize multiple objectives. In every generation, child networks are generated to improve the Pareto frontier with respect to the current population of ANNs. Neural Architect is claimed to be a resource-aware multi-objective RL-based NAS with network embedding and performance prediction. Network embedding encodes an existing network to a trainable embedding vector. Based on the embedding, a controller network generates transformations of the target network. A multi-objective reward function considers network accuracy, computational resource and training time. The reward is predicted by multiple performance simulation networks that are pre-trained or co-trained with the controller network. The controller network is trained via policy gradient. Following a modification, the resulting candidate network is evaluated by both an accuracy network and a training time network. The results are combined by a reward engine that passes its output back to the controller network. == One-shot models == RL or evolution-based NAS require thousands of GPU-days of searching/training to achieve state-of-the-art computer vision results as described in the NASNet, mNASNet and MobileNetV3 papers. To reduce computational cost, many recent NAS methods rely on the weight-sharing idea. In this approach, a single overparameterized supernetwork (also known as the one-shot model) is defined. A supernetwork is a very large Directed Acyclic Graph (DAG) whose subgraphs are different candidate neural networks. Thus, in a supernetwork, the weights are shared among a large number of different sub-architectures that have edges in common, each of which is considered as a path within the supernet. The essential idea is to train one supernetwork that spans many options for the final design rather than generating and training thousands of networks independently. In addition to the learned parameters, a set of architecture parameters are learnt to depict preference for one module over another. Such methods reduce the required computational resources to only a few GPU days. More recent works further combine this weight-sharing paradigm, with a continuous relaxation of the search space, which enables the use of gradient-based optimization methods. These approaches are generally referred to as differentiable NAS and have proven very efficient in exploring the search space of ne