AI App Builder Free

AI App Builder Free — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

  • Hello World: How to be Human in the Age of the Machine

    Hello World: How to be Human in the Age of the Machine

    Hello World: How to Be Human in the Age of the Machine (also titled Hello World: Being Human in the Age of Algorithms) is a book on the growing influence of algorithms and artificial intelligence (AI) on human life, authored by mathematician and science communicator Hannah Fry. The book examines how algorithms are increasingly shaping decisions in critical areas such as healthcare, transportation, justice, finance, and the arts. == Overview == Fry uses real-world examples, such as driverless cars and predictive policing, to illustrate her points. She emphasizes that algorithms are not inherently objective; they reflect biases embedded in their design and data inputs. While acknowledging their potential to improve efficiency and accuracy, Fry cautions against over-reliance on machines without human judgment. Fry explores moral questions surrounding algorithmic decision-making, such as whether machines can replace human empathy in critical situations. She advocates for greater scrutiny of algorithms to ensure fairness and avoid harmful biases. The book proposes a "cyborg future", where humans work alongside algorithms to enhance decision-making while retaining ultimate control. == Reception == Hello World has been praised for its clarity, engaging storytelling, and balanced perspective. Critics have highlighted Fry's ability to make complex topics accessible to general audiences while raising important questions about technology's impact on society. The book was shortlisted for awards such as the 2018 Baillie Gifford Prize and the Royal Society Science Book Prize.

    Read more →
  • Tag (metadata)

    Tag (metadata)

    In information systems, a tag is a keyword or term assigned to a piece of information (such as an Internet bookmark, multimedia, database record, or computer file). This kind of metadata helps describe an item and allows it to be found again by browsing or searching. Tags are generally chosen informally and personally by the item's creator or by its viewer, depending on the system, although they may also be chosen from a controlled vocabulary. Tagging was popularized by websites associated with Web 2.0 and is an important feature of many Web 2.0 services. It is now also part of other database systems, desktop applications, and operating systems. == Overview == People use tags to aid classification, mark ownership, note boundaries, and indicate online identity. Tags may take the form of words, images, or other identifying marks. An analogous example of tags in the physical world is museum object tagging. People were using textual keywords to classify information and objects long before computers. Computer based search algorithms made the use of such keywords a rapid way of exploring records. Tagging gained popularity due to the growth of social bookmarking, image sharing, and social networking websites. These sites allow users to create and manage labels (or "tags") that categorize content using simple keywords. Websites that include tags often display collections of tags as tag clouds, as do some desktop applications. On websites that aggregate the tags of all users, an individual user's tags can be useful both to them and to the larger community of the website's users. Tagging systems have sometimes been classified into two kinds: top-down and bottom-up. Top-down taxonomies are created by an authorized group of designers (sometimes in the form of a controlled vocabulary), whereas bottom-up taxonomies (called folksonomies) are created by all users. This definition of "top down" and "bottom up" should not be confused with the distinction between a single hierarchical tree structure (in which there is one correct way to classify each item) versus multiple non-hierarchical sets (in which there are multiple ways to classify an item); the structure of both top-down and bottom-up taxonomies may be either hierarchical, non-hierarchical, or a combination of both. Some researchers and applications have experimented with combining hierarchical and non-hierarchical tagging to aid in information retrieval. Others are combining top-down and bottom-up tagging, including in some large library catalogs (OPACs) such as WorldCat. When tags or other taxonomies have further properties (or semantics) such as relationships and attributes, they constitute an ontology. In folder system a file cannot exist in two or more folders so tag system has been thought more convenient. But transitioning to tag system requires awareness of difference between properties of two systems. In folder system the information of classification is put outside of the file and we can change folder at once. In tag system the information of classification is put inside the file so changing its tag means changing the file and it needs to be saved again and takes time. Metadata tags as described in this article should not be confused with the use of the word "tag" in some software to refer to an automatically generated cross-reference; examples of the latter are tags tables in Emacs and smart tags in Microsoft Office. == History == The use of keywords as part of an identification and classification system long predates computers. Paper data storage devices, notably edge-notched cards, that permitted classification and sorting by multiple criteria were already in use prior to the twentieth century, and faceted classification has been used by libraries since the 1930s. In the late 1970s and early 1980s, Emacs, the text editor for Unix systems, offered a companion software program called Tags that could automatically build a table of cross-references called a tags table that Emacs could use to jump between a function call and that function's definition. This use of the word "tag" did not refer to metadata tags, but was an early use of the word "tag" in software to refer to a word index. Online databases and early websites deployed keyword tags as a way for publishers to help users find content. In the early days of the World Wide Web, the keywords meta element was used by web designers to tell web search engines what the web page was about, but these keywords were only visible in a web page's source code and were not modifiable by users. In 1997, the collaborative portal "A Description of the Equator and Some ØtherLands" produced by documenta X, Germany, used the folksonomic term Tag for its co-authors and guest authors on its Upload page. In "The Equator" the term Tag for user-input was described as an abstract literal or keyword to aid the user. However, users defined singular Tags, and did not share Tags at that point. In 2003, the social bookmarking website Delicious provided a way for its users to add "tags" to their bookmarks (as a way to help find them later); Delicious also provided browseable aggregated views of the bookmarks of all users featuring a particular tag. Within a couple of years, the photo sharing website Flickr allowed its users to add their own text tags to each of their pictures, constructing flexible and easy metadata that made the pictures highly searchable. The success of Flickr and the influence of Delicious popularized the concept, and other social software websites—such as YouTube, Technorati, and Last.fm—also implemented tagging. In 2005, the Atom web syndication standard provided a "category" element for inserting subject categories into web feeds, and in 2007 Tim Bray proposed a "tag" URN. == Examples == === Within a blog === Many systems (and other web content management systems) allow authors to add free-form tags to a post, along with (or instead of) placing the post into a predetermined category. For example, a post may display that it has been tagged with baseball and tickets. Each of those tags is usually a web link leading to an index page listing all of the posts associated with that tag. The blog may have a sidebar listing all the tags in use on that blog, with each tag leading to an index page. To reclassify a post, an author edits its list of tags. All connections between posts are automatically tracked and updated by the blog software; there is no need to relocate the page within a complex hierarchy of categories. === Within application software === Some desktop applications and web applications feature their own tagging systems, such as email tagging in Gmail and Mozilla Thunderbird, bookmark tagging in Firefox, audio tagging in iTunes or Winamp, and photo tagging in various applications. Some of these applications display collections of tags as tag clouds. === Assigned to computer files === There are various systems for applying tags to the files in a computer's file system. In Apple's Mac System 7, released in 1991, users could assign one of seven editable colored labels (with editable names such as "Essential", "Hot", and "In Progress") to each file and folder. In later iterations of the Mac operating system ever since OS X 10.9 was released in 2013, users could assign multiple arbitrary tags as extended file attributes to any file or folder, and before that time the open-source OpenMeta standard provided similar tagging functionality for Mac OS X. Several semantic file systems that implement tags are available for the Linux kernel, including Tagsistant. Microsoft Windows allows users to set tags only on Microsoft Office documents and some kinds of picture files. Cross-platform file tagging standards include Extensible Metadata Platform (XMP), an ISO standard for embedding metadata into popular image, video and document file formats, such as JPEG and PDF, without breaking their readability by applications that do not support XMP. XMP largely supersedes the earlier IPTC Information Interchange Model. Exif is a standard that specifies the image and audio file formats used by digital cameras, including some metadata tags. TagSpaces is an open-source cross-platform application for tagging files; it inserts tags into the filename. === For an event === An official tag is a keyword adopted by events and conferences for participants to use in their web publications, such as blog entries, photos of the event, and presentation slides. Search engines can then index them to make relevant materials related to the event searchable in a uniform way. In this case, the tag is part of a controlled vocabulary. === In research === A researcher may work with a large collection of items (e.g. press quotes, a bibliography, images) in digital form. If he/she wishes to associate each with a small number of themes (e.g. to chapters of a book, or to sub-themes of the overall subject), then a group of tags for these themes can be attached to each of the items in

    Read more →
  • AlphaGo Zero

    AlphaGo Zero

    AlphaGo Zero is a version of DeepMind's Go software AlphaGo. AlphaGo's team published an article in Nature in October 2017 introducing AlphaGo Zero, a version created without using data from human games, and stronger than any previous version. By playing games against itself, AlphaGo Zero: surpassed the strength of AlphaGo Lee in three days by winning 100 games to 0; reached the level of AlphaGo Master in 21 days; and exceeded all previous versions in 40 days. Training artificial intelligence (AI) without datasets derived from human experts has significant implications for the development of AI with superhuman skills, as expert data is "often expensive, unreliable, or simply unavailable." Demis Hassabis, the co-founder and CEO of DeepMind, said that AlphaGo Zero was so powerful because it was "no longer constrained by the limits of human knowledge". Furthermore, AlphaGo Zero performed better than standard deep reinforcement learning models (such as Deep Q-Network implementations) due to its integration of Monte Carlo tree search. David Silver, one of the first authors of DeepMind's papers published in Nature on AlphaGo, said that it is possible to have generalized AI algorithms by removing the need to learn from humans. Google later developed AlphaZero, a generalized version of AlphaGo Zero that could play chess and shōgi in addition to Go. In December 2017, AlphaZero beat the 3-day version of AlphaGo Zero by winning 60 games to 40, and with 8 hours of training it outperformed AlphaGo Lee on an Elo scale. AlphaZero also defeated a top chess program (Stockfish) and a top Shōgi program (Elmo). == Architecture == The network in AlphaGo Zero is a ResNet with two heads. The stem of the network takes as input a 17x19x19 tensor representation of the Go board. 8 channels are the positions of the current player's stones from the last eight time steps. (1 if there is a stone, 0 otherwise. If the time step go before the beginning of the game, then 0 in all positions.) 8 channels are the positions of the other player's stones from the last eight time steps. 1 channel is all 1 if black is to move, and 0 otherwise. The body is a ResNet with either 20 or 40 residual blocks and 256 channels. There are two heads, a policy head and a value head. Policy head outputs a logit array of size 19 × 19 + 1 {\displaystyle 19\times 19+1} , representing the logit of making a move in one of the points, plus the logit of passing. Value head outputs a number in the range ( − 1 , + 1 ) {\displaystyle (-1,+1)} , representing the expected score for the current player. -1 represents current player losing, and +1 winning. == Training == AlphaGo Zero's neural network was trained using TensorFlow, with 64 GPU workers and 19 CPU parameter servers. Only four TPUs were used for inference. The neural network initially knew nothing about Go beyond the rules. Unlike earlier versions of AlphaGo, Zero only perceived the board's stones, rather than having some rare human-programmed edge cases to help recognize unusual Go board positions. The AI engaged in reinforcement learning, playing against itself until it could anticipate its own moves and how those moves would affect the game's outcome. In the first three days AlphaGo Zero played 4.9 million games against itself in quick succession. It appeared to develop the skills required to beat top humans within just a few days, whereas the earlier AlphaGo took months of training to achieve the same level. According to Epoch.ai, training cost 3e23 FLOPs. For comparison, the researchers also trained a version of AlphaGo Zero using human games, AlphaGo Master, and found that it learned more quickly, but actually performed more poorly in the long run. DeepMind submitted its initial findings in a paper to Nature in April 2017, which was then published in October 2017. == Hardware cost == The hardware cost for a single AlphaGo Zero system in 2017, including the four TPUs, has been quoted as around $25 million. == Applications == According to Hassabis, AlphaGo's algorithms are likely to be of the most benefit to domains that require an intelligent search through an enormous space of possibilities, such as protein folding (see AlphaFold) or accurately simulating chemical reactions. AlphaGo's techniques are probably less useful in domains that are difficult to simulate, such as learning how to drive a car. DeepMind stated in October 2017 that it had already started active work on attempting to use AlphaGo Zero technology for protein folding, and stated it would soon publish new findings. == Reception == AlphaGo Zero was widely regarded as a significant advance, even when compared with its groundbreaking predecessor, AlphaGo. Oren Etzioni of the Allen Institute for Artificial Intelligence called AlphaGo Zero "a very impressive technical result" in "both their ability to do it—and their ability to train the system in 40 days, on four TPUs". The Guardian called it a "major breakthrough for artificial intelligence", citing Eleni Vasilaki of Sheffield University and Tom Mitchell of Carnegie Mellon University, who called it an impressive feat and an “outstanding engineering accomplishment" respectively. Mark Pesce of the University of Sydney called AlphaGo Zero "a big technological advance" taking us into "undiscovered territory". Gary Marcus, a psychologist at New York University, has cautioned that for all we know, AlphaGo may contain "implicit knowledge that the programmers have about how to construct machines to play problems like Go" and will need to be tested in other domains before being sure that its base architecture is effective at much more than playing Go. In contrast, DeepMind is "confident that this approach is generalisable to a large number of domains". In response to the reports, South Korean Go professional Lee Sedol said, "The previous version of AlphaGo wasn’t perfect, and I believe that’s why AlphaGo Zero was made." On the potential for AlphaGo's development, Lee said he will have to wait and see but also said it will affect young Go players. Mok Jin-seok, who directs the South Korean national Go team, said the Go world has already been imitating the playing styles of previous versions of AlphaGo and creating new ideas from them, and he is hopeful that new ideas will come out from AlphaGo Zero. Mok also added that general trends in the Go world are now being influenced by AlphaGo's playing style. "At first, it was hard to understand and I almost felt like I was playing against an alien. However, having had a great amount of experience, I’ve become used to it," Mok said. "We are now past the point where we debate the gap between the capability of AlphaGo and humans. It’s now between computers." Mok has reportedly already begun analyzing the playing style of AlphaGo Zero along with players from the national team. "Though having watched only a few matches, we received the impression that AlphaGo Zero plays more like a human than its predecessors," Mok said. Chinese Go professional Ke Jie commented on the remarkable accomplishments of the new program: "A pure self-learning AlphaGo is the strongest. Humans seem redundant in front of its self-improvement." == Comparison with predecessors == == AlphaZero == On 5 December 2017, DeepMind team released a preprint on arXiv, introducing AlphaZero, a program using generalized AlphaGo Zero's approach, which achieved within 24 hours a superhuman level of play in chess, shogi, and Go, defeating world-champion programs, Stockfish, Elmo, and 3-day version of AlphaGo Zero in each case. AlphaZero (AZ) is a more generalized variant of the AlphaGo Zero (AGZ) algorithm, and is able to play shogi and chess as well as Go. Differences between AZ and AGZ include: AZ has hard-coded rules for setting search hyperparameters. The neural network is now updated continually. Chess (unlike Go) can end in a tie; therefore AZ can take into account the possibility of a tie game. An open source program, Leela Zero, based on the ideas from the AlphaGo papers is available. It uses a GPU instead of the TPUs recent versions of AlphaGo rely on.

    Read more →
  • GermaNet

    GermaNet

    GermaNet is a semantic network for the German language. It relates nouns, verbs, and adjectives semantically by grouping lexical units that express the same concept into synsets and by defining semantic relations between these synsets. GermaNet is free for academic use, after signing a license. GermaNet shares much in common with the English WordNet and can be viewed as an online thesaurus or a light-weight ontology. GermaNet has been developed and maintained at the University of Tübingen since 1997 within the research group for General and Computational Linguistics. It has been integrated into the EuroWordNet, a multilingual lexical-semantic database. == Database == === Contents === GermaNet partitions the lexical space into a set of concepts that are interlinked by semantic relations. A semantic concept is modeled by a synset. A synset is a set of words (called lexical units) where all the words are taken to have the same or almost the same meaning. Thus, a synset is a set of synonyms grouped under one definition, or "gloss". In addition to the gloss, synsets are labeled with their syntactic function and accompanied by example sentences for each distinct meaning in the synset. Just as in WordNet, for each word category the semantic space is divided into a number of semantic fields closely related to major nodes in the semantic network: Ort, or "location", Körper, or "body", etc. As of version 20.0 (release November 2025), GermaNet contains: Synsets: 179438 Lexical units: 231500 Literals: 216517 1.29 lexical units per synset Number of conceptual relations: 194367 Number of lexical relations: 13602 (synonymy excluded) Number of split compounds: 130901 Number of Interlingual Index (ILI) records: 28561 Number of Wiktionary sense descriptions: 29539 === Format === All GermaNet data is stored in a PostgreSQL relational database. The database schema follows the internal structure of GermaNet: there are tables to store synsets, lexical units, conceptual and lexical relations, etc. GermaNet data is distributed both in this database format and as XML files. In the XML data, two types of files, one for synsets and the other for relations, represent all data available in the GermaNet database. == Interfaces == There are software libraries and APIs available for Java and Python. These programs are distributed under free-software licenses and provide easy access to all information in various versions of GermaNet. GermaNet Rover is an on-line application that can be used to search for synsets in GermaNet, explore the data associated with them, and calculate the semantic similarity of pairs of synsets. It features visualizations of the hypernym relation and advanced filtering options for synset searching. == Licenses == GermaNet 20.0 (released November 2025) can be distributed under one of the following types of license agreements: Academic Research License Agreement: for the purpose of research at academic institutions. There is no license fee for academic use. Licenses are not given to individual students, and those seeking a license are required to talk to an academic advisor. Research and Development License Agreement: applies to non-academic institutions and research consortia. To be used strictly for technology development and internal research. Commercial License Agreement: applies to non-academic institutions and commercial enterprises. It permits technology development and internal research, as well as giving the non-exclusive right to distribute and market any derived product or service. == Alternatives == Open-de-WordNet is a freely available alternative to GermaNet which is compatible with WordNet. == Linguistic applications == GermaNet has been used for a variety of applications, including: semantic analysis shallow recognition of implicit document structure compound analysis analyzing sectional preferences word sense disambiguation

    Read more →
  • NNDB

    NNDB

    The Notable Names Database (NNDB) is an online database of biographical details of over 40,000 people. Soylent Communications, a sole proprietorship that also hosted the later defunct Rotten.com, describes NNDB as an "intelligence aggregator" of noteworthy persons, highlighting their interpersonal connections. The Rotten.com domain was registered in 1996 by former Apple and Netscape software engineer Thomas E. Dell, who was also known by his internet alias, "Soylent". == Entries == Each entry has an executive summary followed by a brief narrative about their life. It also lists date and cause of death if deceased. Businesspeople and government officials are listed with chronologies of their posts, positions, and board memberships. As of 2022, the site is no longer updated. == NNDB Mapper == The NNDB Mapper, a visual tool for exploring connections between people, was made available in May 2008. It required Adobe Flash 7.

    Read more →
  • Nortel Speech Server

    Nortel Speech Server

    The Nortel Speech Server (formerly known as Periphonics Speech Processing Platform) in telecommunications is a speech processing system that was originally developed by Nortel. Following the bankruptcy of Nortel, it is now sold by Avaya. The system is primarily used for large vocabulary speech recognition, natural language understanding, text-to-speech, and speaker verification. The Nortel Speech Server was based on the Periphonics OSCAR platform. The original OSCAR Platform was based upon Solaris servers. The current range of Speech Servers is Windows based. Nortel Speech Server is a component of the MPS 500, MPS 1000, and ICP platforms. On MPS systems, it may be used to stream prerecorded audio.

    Read more →
  • Reification (computer science)

    Reification (computer science)

    In computer science, reification is the process by which an abstract idea about a program is turned into an explicit data model or other object created in a programming language. A computable/addressable object—a resource—is created in a system as a proxy for a non computable/addressable object. By means of reification, something that was previously implicit, unexpressed, and possibly inexpressible is explicitly formulated and made available to conceptual (logical or computational) manipulation. Informally, reification is often referred to as "making something a first-class citizen" within the scope of a particular system. Some aspect of a system can be reified at language design time, which is related to reflection in programming languages. It can be applied as a stepwise refinement at system design time. Reification is one of the most frequently used techniques of conceptual analysis and knowledge representation. == Reflective programming languages == In the context of programming languages, reification is the process by which a user program or any aspect of a programming language that was implicit in the translated program and the run-time system, are expressed in the language itself. This process makes it available to the program, which can inspect all these aspects as ordinary data. In reflective languages, reification data is causally connected to the related reified aspect such that a modification to one of them affects the other. Therefore, the reification data is always a faithful representation of the related reified aspect . Reification data is often said to be made a first class object. Reification, at least partially, has been experienced in many languages to date: in early Lisp dialects and in current Prolog dialects, programs have been treated as data, although the causal connection has often been left to the responsibility of the programmer. In Smalltalk-80, the compiler from the source text to bytecode has been part of the run-time system since the very first implementations of the language. The C programming language reifies the low-level detail of memory addresses.Many programming language designs encapsulate the details of memory allocation in the compiler and the run-time system. In the design of the C programming language, the memory address is reified and is available for direct manipulation by other language constructs. For example, the following code may be used when implementing a memory-mapped device driver. The buffer pointer is a proxy for the memory address 0xB8000000. Functional programming languages based on lambda-calculus reify the concept of a procedure abstraction and procedure application in the form of the Lambda expression. The Scheme programming language reifies continuations (approximately, the call stack). In C#, reification is used to make parametric polymorphism implemented in the form of generics as a first-class feature of the language. In the Java programming language, there exist "reifiable types" that are "completely available at run time" (i.e. their information is not erased during compilation). REBOL reifies code as data and vice versa. Many languages, such as Lisp, JavaScript, and Curl, provide an eval or evaluate procedure that effectively reifies the language interpreter. Smalltalk and Actor languages permit the reification of blocks and messages, which are equivalent of lambda expressions in Lisp, and thisContext in Smalltalk, which is a reification of the current executing block. Homoiconic languages reify the syntax of the language as data that is understood by the language itself. This allows the user to write programs whose inputs and outputs are code (see macros, eval). Common representations of code include S-expressions (e.g. Clojure, Lisp), and abstract syntax trees (e.g. Rust). == Data reification vs. data refinement == Data reification (stepwise refinement) involves finding a more concrete representation of the abstract data types used in a formal specification. Data reification is the terminology of the Vienna Development Method (VDM) that most other people would call data refinement. An example is taking a step towards an implementation by replacing a data representation without a counterpart in the intended implementation language, such as sets, by one that does have a counterpart (such as maps with fixed domains that can be implemented by arrays), or at least one that is closer to having a counterpart, such as sequences. The VDM community prefers the word "reification" over "refinement", as the process has more to do with concretising an idea than with refining it. For similar usages, see Reification (linguistics). == In conceptual modeling == Reification is widely used in conceptual modeling. Reifying a relationship means viewing it as an entity. The purpose of reifying a relationship is to make it explicit, when additional information needs to be added to it. Consider the relationship type IsMemberOf(member:Person, Committee). An instance of IsMemberOf is a relationship that represents the fact that a person is a member of a committee. The figure below shows an example population of IsMemberOf relationship in tabular form. Person P1 is a member of committees C1 and C2. Person P2 is a member of committee C1 only. The same fact, however, could also be viewed as an entity. Viewing a relationship as an entity, one can say that the entity reifies the relationship. This is called reification of a relationship. Like any other entity, it must be an instance of an entity type. In the present example, the entity type has been named Membership. For each instance of IsMemberOf, there is one and only one instance of Membership, and vice versa. Now, it becomes possible to add more information to the original relationship. As an example, we can express the fact that "person p1 was nominated to be the member of committee c1 by person p2". Reified relationship Membership can be used as the source of a new relationship IsNominatedBy(Membership, Person). For related usages see Reification (knowledge representation). == In Unified Modeling Language (UML) == UML provides an association class construct for defining reified relationship types. The association class is a single model element that is both a kind of association and a kind of class. The association and the entity type that reifies are both the same model element. Note that attributes cannot be reified. == On Semantic Web == === RDF and OWL === In Semantic Web languages, such as Resource Description Framework (RDF) and Web Ontology Language (OWL), a statement is a binary relation. It is used to link two individuals or an individual and a value. Applications sometimes need to describe other RDF statements, for instance, to record information like when statements were made, or who made them, which is sometimes called "provenance" information. As an example, we may want to represent properties of a relation, such as our certainty about it, severity or strength of a relation, relevance of a relation, and so on. The example from the conceptual modeling section describes a particular person with URIref person:p1, who is a member of the committee:c1. The RDF triple from that description is Consider to store two further facts: (i) to record who nominated this particular person to this committee (a statement about the membership itself), and (ii) to record who added the fact to the database (a statement about the statement). The first case is a case of classical reification like above in UML: reify the membership and store its attributes and roles etc.: Additionally, RDF provides a built-in vocabulary intended for describing RDF statements. A description of a statement using this vocabulary is called a reification of the statement. The RDF reification vocabulary consists of the type rdf:Statement, and the properties rdf:subject, rdf:predicate, and rdf:object. Using the reification vocabulary, a reification of the statement about the person's membership would be given by assigning the statement a URIref such as committee:membership12345 so that describing statements can be written as follows: These statements say that the resource identified by the URIref committee:membership12345Stat is an RDF statement, that the subject of the statement refers to the resource identified by person:p1, the predicate of the statement refers to the resource identified by committee:isMemberOf, and the object of the statement refers to the resource committee:c1. Assuming that the original statement is actually identified by committee:membership12345, it should be clear by comparing the original statement with the reification that the reification actually does describe it. The conventional use of the RDF reification vocabulary always involves describing a statement using four statements in this pattern. Therefore, they are sometimes referred to as the "reification quad". Using reification according to this convention, we could record the fact that pe

    Read more →
  • General-Purpose AI Code of Practice

    General-Purpose AI Code of Practice

    The General-Purpose AI Code of Practice (GPAI CoP) is a compliance tool released by the European Commission on 10 July 2025 to support compliance with the European Union Artificial Intelligence Act (AI Act). It provides operational guidance for providers of general-purpose AI models, particularly in relation to Articles 53 and 55 of the AI Act, which entered into application on 2 August 2025. The Code is organised into three chapters (Transparency, Copyright, and Safety and Security) and outlines how providers can meet the Act's relevant obligations. Although non-binding, providers can rely on adherence to the Code, meaning that EU regulators will assume that providers following the Code meet the corresponding legal requirements of the AI Act. As such, signatories to the Code will benefit from reduced administrative burdens and increased legal certainty compared to providers that prove compliance in other ways. While adherence to the Code is voluntary, compliance with the AI Act is not. == Background == The EU AI Act, adopted in 2024, established a risk-based regulatory regime for artificial intelligence in the European Union. The rationale for the GPAI CoP stems from Article 56 of the AI Act, which empowers the EU AI Office to develop a voluntary rulebook to guide how AI model providers can meet their legal obligations – specifically those found in Articles 53 and 55. Under Articles 53 and 55, developers of general-purpose AI models whose training compute exceeds 1023 floating-point operations (FLOPs) and that are placed on the EU market must meet transparency obligations and put in place a policy for EU copyright law. Models trained with more than 1025 FLOPs are classified as presenting systemic risk and are subject to enhanced safety requirements. The Commission may also designate a model as presenting systemic risk if it has equivalent impact or capabilities (Annex XIII criteria), even below that compute figure. Because the AI Act is relatively vague on how model providers should implement these requirements, the Code is meant to help by detailing processes and practices for compliance. == Drafting process == The development of the GPAI CoP was drawn up by 13 independent experts and involved four thematic working groups: Transparency & Copyright, Risk assessment for systemic risk, Technical risk mitigation for systemic risk, and Governance risk mitigation for systemic risk. Each group was coordinated by the European Union Artificial Intelligence Office (EU AI Office), drawing on contributions from nearly 1,000 stakeholders, including AI developers, academics, civil society organisations, national authorities, and international observers. The Code underwent three earlier iterations in November 2024, December 2024, and March 2025, before the final version was published on 10 July 2025, more than two months later than initially planned. The GPAI CoP will likely be updated continuously by the EU AI Office, alongside other tools such as the training data summary template. == Signatories == Among U.S.-based technology companies, Amazon, Anthropic, Google, IBM, Microsoft, and OpenAI have signed the GPAI CoP. xAI, founded by Elon Musk, has signed only one of the three chapters, namely the safety and security chapter. Prominent European AI companies that have signed include Aleph Alpha and Mistral AI. The European Commission maintains an updated list of signatories. As of January 2026, Meta is the most notable company that has declined to sign the Code. Major Chinese AI companies, such as Alibaba, Baidu or Deepseek, have also not signed. Providers that do not sign the GPAI CoP will still have to adhere to the binding requirements of the EU AI Act. The European Commission has indicated that it may take tougher action against companies that didn't sign the Code. == Transparency and Copyright chapters == The first two chapters of the GPAI CoP address transparency and copyright compliance and apply to all GPAI providers. They offer a way to demonstrate compliance with their obligations under Article 53 AI Act. The Transparency chapter addresses the documentation of a model's capabilities, limitations, and points of contact, and expects providers to make key documentation available to downstream providers. Signatories must also publish summaries of the content used to train their models. In the Copyright chapter, Signatories commit to follow a policy that aligns with EU copyright law. For example, they commit to mitigating the risk of copyright-infringing output. == Safety and Security chapter == The Safety and Security chapter is the most extensive chapter of the Code, and it applies to GPAI models with systemic risk, meaning it's only relevant to the small number of providers of the most advanced models. It specifies how Signatories commit to meeting Article 55(1) obligations to: Conduct model evaluations to identify systemic risks Assess and mitigate those risks Track and report serious incidents Ensure the cyber and physical security of their models The chapter outlines a comprehensive risk management process that must be applied before major deployment decisions, such as releasing a new systemic-risk GPAI model in the EU market, or substantially updating an existing one. Signatories commit to identifying systemic risks of their model, analysing and evaluating them, determining whether risk levels are acceptable, and implementing mitigation measures if necessary. This process should be repeated until models achieve an acceptable level of risk across all identified risks. === Risk identification === Signatories commit to analysing and evaluating at least four “specified” categories of systemic risk: CBRN (chemical, biological, radiological, and nuclear) Loss of control Cyber offence Harmful manipulation They are also expected to identify other systemic risks to public health, safety, and fundamental rights. The Code instructs providers to consider model capabilities, propensities, and affordances in this identification. Signatories commit to developing risk scenarios illustrating how identified risks could materialise in real-world conditions. === Risk analysis and risk evaluation === After identifying potential systemic risks, Signatories commit to analysing and evaluating the risks in order to determine whether they are acceptable or not, drawing on scientific literature, training data analysis, incident databases, expert consultation, and other sources. They also commit to conducting state-of-the-art model evaluations such as benchmarking, red teaming, and human uplift studies, targeting each risk. The risk analysis process is interconnected: insights from risk modelling should inform model evaluation design, while post-market monitoring should feed back into ongoing analysis. Signatories commit to ultimately estimating the likelihood and severity of each systemic risk. ==== Independent external model evaluations ==== Appendix 3.5 of the Safety and Security chapter requires signatories to ensure that independent external evaluators conduct model evaluations. Signatories may claim an exemption from this requirement only if they can demonstrate that their model is “similarly safe” to another model that has already been shown to comply with the Code, or if they are unable to appoint an appropriately qualified evaluator. The determination of “similarly safe” is based on comparable performance on benchmarks and the similarity of other model characteristics, such as their architecture. The CoP acknowledges that this kind of information is typically available only for models by the same provider, or potentially for open-weights or open-source models. === Risk acceptance criteria === The Code requires providers to compare estimated risks against predefined acceptance criteria, which must be measurable, based on model capabilities, and defined preemptively. While providers get to determine the level of risk they deem acceptable themselves, the pre-defined criteria and acceptance thresholds ensure providers cannot adjust their level of tolerance flexibly ahead of deployment decisions. Only if all risks are below acceptable levels should a model be deployed. === Continuous risk management and governance === The Code mandates ongoing risk management throughout the model lifecycle, including light-touch evaluations, continuous mitigation, post-market monitoring, and incident tracking and reporting. It further requires organisational governance structures assigning responsibility for risk management and expects providers to promote a “healthy risk culture,” including informing employees about the whistleblower protection policy, allowing internal challenges of decisions concerning systemic risk management, and committing to not retaliating against employees who disclose concerns about systemic risks to oversight authorities. === Documentation and transparency === Signatories commit to creating two types of documentation: Safety and Security Frame

    Read more →
  • AI washing

    AI washing

    AI washing is a deceptive marketing tactic that consists of promoting a product or a service by overstating the role of artificial intelligence (AI) and the integration of it. Companies often involve in the practice to mislead customers to boost their offerings, and to secure funding from investors. The practice raises concerns regarding transparency, and legal issues. == Definition == AI washing is a deceptive marketing practice. It involves promoting a product or a service by overstating the role of artificial intelligence (AI) and its integration in the design and manufacture of the same. The practice raises concerns regarding transparency, compliance with security regulations, and consumer trust in the AI industry potentially hampering legitimate advancements in AI. The term was first defined by the AI Now Institute, a research institute based at New York University in 2019. The term is derived from greenwashing, another deceptive marketing technique that misrepresents a product's environmental impact in a similar manner. AI washing might involve a company claiming to have used AI in the development or enhancement of its products or services without its actual involvement, or using buzzwords such as "smart" or "AI-powered" without the product actually offering it or making use of it. A company may overstate the usage of AI or misuse the term, which is also construed as AI washing. In 2026, The Washington Post defined AI washing as "a trend for bosses to blame layoffs on the productive capabilities of AI and its ability to replace workers, even when job cuts may have little to do with the technology". == Usage and effects == AI washing can lead to deception of customers and misleading of investors. It is also an illegal and unethical practice that lacks transparency regarding disclosing the details of a product or a service. Companies get involved in such a practice often in response to competition who might have used AI in their offerings. It might also be used as a ploy to secure funding and investment, assuming that it will attract them towards it. AI washing has been compared to dot-com bubble, when businesses appended "dot-com" to the end of the business name to boost their valuation. In September 2023, Coca-Cola released a new product called Coca-Cola Y3000, and the company stated that the Y3000 flavor had been "co-created with human and artificial intelligence". The company was accused of AI washing due to no proof of AI involvement in the creation of the product, and critics believed that AI was used as a way to grab consumer attention more than it was used in the actual product creation. In 2026, mass tech layoffs were attributed to AI washing from AI innovation instead of balance sheet restructuring. == Mitigation == Companies are expected to be transparent and clearer in communicating the usage of AI in their products or services. Consumers can mitigate the same by requesting for hard evidence from the companies regarding the usage of AI tools. Customers should evaluate the product or service as a whole rather than being swayed by the usage of AI. Informed decision making and purchasing can keep them from falling for such marketing gimmicks. The United States Securities and Exchange Commission (SEC) imposes penalties for companies indulging in such practices. In March 2024, the SEC imposed the first civil penalties on two companies for misleading statements about their use of AI, and in July 2024, it charged a corporate executive from a supposed AI hiring startup with fraud for the usage of buzzwords related to AI.

    Read more →
  • KataGo

    KataGo

    KataGo is a free and open-source computer Go program, capable of defeating top-level human players. First released on 27 February 2019, it is developed by David Wu, who also developed the Arimaa playing program bot_Sharp which defeated three top human players to win the Arimaa AI Challenge in 2015. KataGo's first release was trained by David Wu using resources provided by his employer Jane Street Capital, but it is now trained by a distributed effort. Members of the computer Go community provide computing resources by running the client, which generates self-play games and rating games, and submits them to a server. The self-play games are used to train newer networks and the rating games to evaluate the networks' relative strengths. KataGo supports the Go Text Protocol, with various extensions, thus making it compatible with popular GUIs such as Lizzie. As an alternative, it also implements a custom "analysis engine" protocol, which is used by the KaTrain GUI, among others. KataGo is widely used by strong human go players, including the South Korean national team, for training purposes. KataGo is also used as the default analysis engine in the online Go website AI Sensei, as well as OGS (the Online Go Server). == Technology == Based on techniques used by DeepMind's AlphaGo Zero, KataGo implements Monte Carlo tree search with a convolutional neural network providing position evaluation and policy guidance. Compared to AlphaGo, KataGo introduces many refinements that enable it to learn faster and play more strongly. Notable features of KataGo that are absent in many other Go-playing programs include score estimation; support for small boards, rectangular boards, and large boards; arbitrary values of komi and handicaps; and the ability to use various Go rulesets and adjust its play and evaluation for the small differences between them. === Network === The network used in KataGo are ResNets with pre-activation. While AlphaGo Zero has only game board history as input features (as it was designed as a general architecture for board games, subsequently becoming AlphaZero), the input to the network contains additional features designed by hand specifically for playing Go. These features include liberties, komi parity, pass-alive, and ladders. The trunk is essentially the same as in AlphaGo Zero, but with global pooling layers added to allow the network to be conditioned on global context such as ko fights. This is similar to the Squeeze-and-Excitation Network. The network has two heads: a policy head and a value head. The policy and value heads are mostly the same as in AlphaGo Zero, but both heads have auxiliary subheads to provide auxiliary loss signal for faster training: Policy head: predicts policy for the current player's move this turn, and the opponent player's move in the next turn. A policy Each is a logit array of size 19 × 19 + 1 {\displaystyle 19\times 19+1} , representing the logit of making a move in one of the points, plus the logit of passing. Value head: predicts game outcome, expected score difference, expected board ownership, etc. The network is described in detail in Appendix A of the report. The code base switched from using TensorFlow to PyTorch in version 1.12. === Training === Let its trunk have b {\displaystyle b} residual blocks and c {\displaystyle c} channels. During its first training run, multiple networks were trained with increasing ( b , c ) {\displaystyle (b,c)} . It took 19 days using a maximum of 28 Nvidia V100 GPUs at 4.2 million games. After the first training run, training became a distributed project run by volunteers, with increasing network sizes. As of August 2024, it has reached b28c512 (28 blocks, 512 channels). == Adversarial attacks == In 2022, KataGo was used as the target for adversarial attack research, designed to demonstrate the "surprising failure modes" of AI systems. The researchers were able to trick KataGo into ending the game prematurely. Adversarial training improves defense against adversarial attacks, though not perfectly.

    Read more →
  • 20Q

    20Q

    20Q is a computerized game of twenty questions that began as a test in artificial intelligence (AI). It was invented by Robin Burgener in 1988. The game was made handheld by Radica in 2003, but was discontinued in 2011 because Techno Source took the license for 20Q handheld devices. The game 20Q is based on the spoken parlor game known as twenty questions, and is both a website and a handheld device. 20Q asks the player to think of something and will then try to guess what they are thinking of with twenty yes-or-no questions. If it fails to guess in 20 questions, it will ask an additional 5 questions. If it fails to guess even with 25 (or 30) questions, the player is declared the winner. Sometimes the first guess of the object can be asked at question 14. == Principle and history == The principle is that the player thinks of something and the 20Q artificial intelligence asks a series of questions before guessing what the player is thinking. This artificial intelligence learns on its own with the information relayed back to the players who interact with it, and is not programmed. The player can answer these questions with: Yes, No, Unknown, and Sometimes. The experiment is based on the classic word game of Twenty Questions, and on the computer game "Animals," popular in the early 1970s, which used a somewhat simpler method to guess an animal. The 20Q AI uses an artificial neural network to pick the questions and to guess. After the player has answered the twenty questions posed (sometimes fewer), 20Q makes a guess. If it is incorrect, it asks more questions, then guesses again. It makes guesses based on what it has learned; it is not programmed with information or what the inventor thinks. Answers to any question are based on players’ interpretations of the questions asked. Newer editions were made for different categories, such as music 20Q which has the player think of a song, and Harry Potter 20Q, which has the player think of something from the world of the Harry Potter series. The 20Q AI can draw its own conclusions on how to interpret the information. It can be described as more of a folk taxonomy than a taxonomy. Its knowledge develops with every game played. In this regard, the online version of the 20Q AI can be inaccurate because it gathers its answers from what people think rather than from what people know. Limitations of taxonomy are often overcome by the AI itself because it can learn and adapt. For example, if the player was thinking of a "Horse" and answered "No" to the question "Is it an animal?," the AI will, nevertheless, guess correctly, despite being told that a horse is not an animal. Patent applications in the US and Europe were submitted in 2005. In August 2014, 20Q.net Inc., with Brashworks Studios, developed and released an iOS iPad version available at the Apple iTunes store. == Game show == On June 13, 2009, GSN began a TV version of the game, hosted by Cat Deeley, with Hal Sparks as the voice of Mr. Q.

    Read more →
  • Partial-order planning

    Partial-order planning

    Partial-order planning is an approach to automated planning that maintains a partial ordering between actions and only commits ordering between actions when forced to, that is, ordering of actions is partial. Also this planning doesn't specify which action will come out first when two actions are processed. By contrast, total-order planning maintains a total ordering between all actions at every stage of planning. Given a problem in which some sequence of actions is needed to achieve a goal, a partial-order plan specifies all actions that must be taken, but specifies an ordering between actions only where needed. Consider the following situation: a person must travel from the start to the end of an obstacle course. The course is composed of a bridge, a see-saw, and a swing-set. The bridge must be traversed before the see-saw and swing-set are reachable. Once reachable, the see-saw and swing-set can be traversed in any order, after which the end is reachable. In a partial-order plan, ordering between these obstacles is specified only when needed. The bridge must be traversed first. Second, either the see-saw or swing-set can be traversed. Third, the remaining obstacle can be traversed. Then the end can be traversed. Partial-order planning relies upon the principle of least commitment for its efficiency. == Partial-order plan == A partial-order plan or partial plan is a plan which specifies all actions that must be taken, but only specifies the order between actions when needed. It is the result of a partial-order planner. A partial-order plan consists of four components: A set of actions (also known as operators). A partial order for the actions. It specifies the conditions about the order of some actions. A set of causal links. It specifies which actions meet which preconditions of other actions. Alternatively, a set of bindings between the variables in actions. A set of open preconditions. It specifies which preconditions are not fulfilled by any action in the partial-order plan. To keep the possible orders of the actions as open as possible, the set of order conditions and causal links must be as small as possible. A plan is a solution if the set of open preconditions is empty. A linearization of a partial order plan is a total order plan derived from the particular partial order plan; in other words, both order plans consist of the same actions, with the order in the linearization being a linear extension of the partial order in the original partial order plan. === Example === For example, a plan for baking a cake might start: go to the store get eggs; get flour; get milk pay for all goods go to the kitchen This is a partial plan because the order for finding eggs, flour and milk is not specified, the agent can wander around the store reactively accumulating all the items on its shopping list until the list is complete. == Partial-order planner == A partial-order planner is an algorithm or program which will construct a partial-order plan and search for a solution. The input is the problem description, consisting of descriptions of the initial state, the goal and possible actions. The problem can be interpreted as a search problem where the set of possible partial-order plans is the search space. The initial state would be the plan with the open preconditions equal to the goal conditions. The final state would be any plan with no open preconditions, i.e. a solution. The initial state is the starting conditions, and can be thought of as the preconditions to the task at hand. For a task of setting the table, the initial state could be a clear table. The goal is simply the final action that needs to be accomplished, for example setting the table. The operators of the algorithm are the actions by which the task is accomplished. For this example there may be two operators: lay (tablecloth), and place (glasses, plates, and silverware). === Plan space === The plan space of the algorithm is constrained between its start and finish. The algorithm starts, producing the initial state and finishes when all parts of the goal have been achieved. In the setting a table example, two types of actions exist that must be addressed: the put-out and lay operators. Four unsolved operators also exist: Action 1, lay-tablecloth, Action 2, Put-out (plates), Action 3, Put-out (silverware), and Action 4, Put-out (glasses). However, a threat arises if Action 2, 3, or 4 comes before Action 1. This threat is that the precondition to the start of the algorithm will be unsatisfied as the table will no longer be clear. Thus, constraints exist that must be added to the algorithm that force Actions 2, 3, and 4 to come after Action 1. Once these steps are completed, the algorithm will finish and the goal will have been completed. === Threats === As seen in the algorithm presented above, partial-order planning can encounter certain threats, meaning orderings that threaten to break connected actions, thus potentially destroying the entire plan. There are two ways to resolve threats: Promotion Demotion Promotion orders the possible threat after the connection it threatens. Demotion orders the possible threat before the connection it threatens. Partial-order planning algorithms are known for being both sound and complete, with sound being defined as the total ordering of the algorithm, and complete being defined as the capability to find a solution, given that a solution does in fact exist. == Partial-order vs. total-order planning == Partial-order planning is the opposite of total-order planning, in which actions are sequenced all at once and for the entirety of the task at hand. The question arises when one has two competing processes, which one is better? Anthony Barret and Daniel Weld have argued in their 1993 book, that partial-order planning is superior to total-order planning, as it is faster and thus more efficient. They tested this theory using Korf’s taxonomy of subgoal collections, in which they found that partial-order planning performs better because it produces more trivial serializability than total-order planning. Trivial serializability facilitates a planner’s ability to perform quickly when dealing with goals that contain subgoals. Planners perform more slowly when dealing with laboriously serializable or nonserializable subgoals. The determining factor that makes a subgoal trivially or laboriously serializable is the search space of different plans. They found that partial-order planning is more adept at finding the quickest path, and is therefore the more efficient of these two main types of planning. == The Sussman anomaly == Partial-order plans are known to easily and optimally solve the Sussman anomaly. Using this type of incremental planning system solves this problem quickly and efficiently. This was a result of partial-order planning that solidified its place as an efficient planning system. == Disadvantages to partial-order planning == One drawback of this type of planning system is that it requires a lot more computational power for each node. This higher per-node cost occurs because the algorithm for partial-order planning is more complex than others. This has important artificial intelligence implications. When coding a robot to do a certain task, the creator needs to take into account how much energy is needed. Though a partial-order plan may be quicker it may not be worth the energy cost for the robot. The creator must be aware of and weigh these two options to build an efficient robot.

    Read more →
  • Large language model

    Large language model

    A large language model (LLM) is a neural network trained on a vast amount of text for natural language processing tasks, especially language generation. LLMs can typically generate, summarize, translate and analyze text in many contexts, and are a foundational technology behind modern chatbots. Biased or inaccurate training data can make an LLM's output less reliable. As of 2026, the most capable LLMs are based on transformer architectures, which, according to the 2017 paper "Attention Is All You Need", can be more efficient and parallelizable than earlier statistical and recurrent neural network models. Benchmark evaluations for LLMs attempt to measure model reasoning, factual accuracy, alignment, and safety. == History == Before the emergence of transformer-based models in 2017, some language models were considered large relative to the computational and data constraints of their time. In the early 1990s, IBM's statistical models pioneered word alignment techniques for machine translation, laying the groundwork for corpus-based language modeling. In 2001, a smoothed n-gram model, such as those employing Kneser–Ney smoothing, trained on 300 million words, achieved state-of-the-art perplexity on benchmark tests. During the 2000s, with the rise of widespread internet access, researchers began compiling massive text datasets from the web ("web as corpus") to train statistical language models. Moving beyond n-gram models, researchers started in 2000 to use neural networks as language models. Following the breakthrough of deep neural networks in image classification around 2012, similar architectures were adapted for language tasks. This shift was marked by the development of word embeddings (e.g., Word2Vec by Mikolov in 2013) and sequence-to-sequence (seq2seq) models using LSTM. In 2016, Google transitioned its translation service to neural machine translation (NMT), replacing statistical phrase-based models with deep recurrent neural networks. These early NMT systems used LSTM-based encoder-decoder architectures, as they preceded the invention of transformers. At the 2017 NeurIPS conference, Google researchers introduced the transformer architecture in their landmark paper "Attention Is All You Need". This paper's goal was to improve upon 2014 seq2seq technology, and was based mainly on the attention mechanism developed by Bahdanau et al. in 2014. The following year in 2018, BERT was introduced and quickly became "ubiquitous". Though the original transformer has both encoder and decoder blocks, BERT is an encoder-only model. Academic and research usage of BERT began to decline in 2023, following rapid improvements in the abilities of decoder-only models (such as GPT) to solve tasks via prompting. Although decoder-only GPT-1 was introduced in 2018, it was GPT-2 in 2019 that caught widespread attention because OpenAI claimed to have initially deemed it too powerful to release publicly, out of fear of malicious use. GPT-3 in 2020 went a step further and as of 2025 is available only via API with no offering of downloading the model to execute locally. But it was the consumer-facing chatbot ChatGPT in late 2022 that received extensive media coverage and public attention by 2023. The 2023 GPT-4 was praised for its increased accuracy and as a "holy grail" for its multimodal capabilities. OpenAI did not reveal the high-level architecture and the number of parameters of GPT-4. The release of ChatGPT led to an uptick in LLM usage across several research subfields of computer science, including robotics, software engineering, and societal impact work. In 2024, OpenAI released the reasoning model OpenAI o1, which generates long chains of thought before returning a final answer. Many LLMs with parameter counts comparable to those of OpenAI's GPT series have been developed. Since 2022, weights-available models have been gaining popularity, especially at first with BLOOM and LLaMA, though both have restrictions on usage and deployment. Mistral AI's open-weight models Mistral 7B and Mixtral 8x7B have a more permissive Apache License. In January 2025, DeepSeek released DeepSeek R1, a 671-billion-parameter open-weight model that performs comparably to OpenAI o1 but at a much lower price per token for users. Since 2023, many LLMs have been trained to be multimodal, having the ability to also process or generate other types of data, such as images, audio, or 3D meshes. Open-weight LLMs have become more influential since 2023. Per Vake et al. (2025), community-driven contributions to open-weight models improve their efficiency and performance via collaborative platforms such as Hugging Face. == Dataset preprocessing == === Tokenization === As machine learning algorithms process numbers rather than text, the text must be converted to numbers. In the first step, a vocabulary is decided upon, then integer indices are arbitrarily but uniquely assigned to each vocabulary entry, and finally, an embedding is associated with the integer index. Algorithms include byte-pair encoding (BPE) and WordPiece. There are also special tokens serving as control characters, such as [MASK] for masked-out token (as used in BERT), and [UNK] ("unknown") for characters not appearing in the vocabulary. Also, some special symbols are used to denote special text formatting. For example, "Ġ" denotes a preceding whitespace in RoBERTa and GPT and "##" denotes continuation of a preceding word in BERT. For example, the BPE tokenizer used by the legacy version of GPT-3 would split tokenizer: texts -> series of numerical "tokens" as Tokenization also compresses the datasets. Because LLMs generally require input to be an array that is not jagged, the shorter texts must be "padded" until they match the length of the longest one. ==== Byte-pair encoding ==== As an example, consider a tokenizer based on byte-pair encoding. In the first step, all unique characters (including blanks and punctuation marks) are treated as an initial set of n-grams (i.e. initial set of uni-grams). Successively the most frequent pair of adjacent characters is merged into a bi-gram and all instances of the pair are replaced by it. All occurrences of adjacent pairs of (previously merged) n-grams that most frequently occur together are then again merged into even lengthier n-gram, until a vocabulary of prescribed size is obtained. After a tokenizer is trained, any text can be tokenized by it, as long as it does not contain characters not appearing in the initial-set of uni-grams. === Dataset cleaning === In the context of training LLMs, datasets are typically cleaned by removing low-quality, duplicated, or toxic data. Cleaned datasets can increase training efficiency and lead to improved downstream performance. A trained LLM can be used to clean datasets for training a further LLM. With the increasing proportion of LLM-generated content on the web, data cleaning in the future may include filtering out such content. LLM-generated content can pose a problem if the content is similar to human text (making filtering difficult) but of lower quality (degrading performance of models trained on it). === Synthetic data === Training of largest language models might need more linguistic data than naturally available, or that the naturally occurring data is of insufficient quality. In these cases, synthetic data might be used. == Training == An LLM is a type of foundation model (large X model) trained on language. LLMs can be trained in different ways. In particular, GPT models are first pretrained to predict the next word on a large amount of data, before being fine-tuned. === Cost === Substantial infrastructure is necessary for training the largest models. The tendency towards larger models is visible in the list of large language models. For example, the training of GPT-2 (i.e. a 1.5-billion-parameter model) in 2019 cost $50,000, while training of the PaLM (i.e. a 540-billion-parameter model) in 2022 cost $8 million, and Megatron-Turing NLG 530B (in 2021) cost around $11 million. The qualifier "large" in "large language model" is inherently vague, as there is no definitive threshold for the number of parameters required to qualify as "large". === Fine-tuning === Before being fine-tuned, most LLMs are next-token predictors. The fine-tuning shapes the LLM's behavior via techniques like reinforcement learning from human feedback (RLHF) or constitutional AI. Instruction fine-tuning is a form of supervised learning used to teach LLMs to follow user instructions. In 2022, OpenAI demonstrated InstructGPT, a version of GPT-3 similarly fine-tuned to follow instructions. Reinforcement learning from human feedback (RLHF) involves training a reward model to predict which text humans prefer. Then, the LLM can be fine-tuned through reinforcement learning to better satisfy this reward model. Since humans typically prefer truthful, helpful and harmless answers, RLHF favors such answers. == Architecture == LLMs are generally based on the tra

    Read more →
  • OpenVINO

    OpenVINO

    OpenVINO is an open-source software toolkit developed by Intel for optimizing and deploying deep learning models. It supports several popular model formats and categories, such as large language models, computer vision, and generative AI. OpenVINO is optimized for Intel hardware, but offers support for ARM/ARM64 processors. It sees great use in AI Sound Processing drivers when tied with Intel's Gaussian & Neural Accelerator (GNA). Based in C++, it extends API support for C and Python, as well as Node.js (in early preview). OpenVINO is cross-platform and free for use under Apache License 2.0. == Workflow == The simplest OpenVINO usage involves obtaining a model and running it as is. Yet for the best results, a more complete workflow is suggested: obtain a model in one of supported frameworks, convert the model to OpenVINO IR using the OpenVINO Converter tool, optimize the model, using training-time or post-training options provided by OpenVINO's NNCF. execute inference, using OpenVINO Runtime by specifying one of several inference modes. == OpenVINO model format == OpenVINO IR is the default format used to run inference. It is saved as a set of two files, .bin and .xml, containing weights and topology, respectively. It is obtained by converting a model from one of the supported frameworks, using the application's API or a dedicated converter. Models of the supported formats may also be used for inference directly, without prior conversion to OpenVINO IR. Such an approach is more convenient but offers fewer optimization options and lower performance, since the conversion is performed automatically before inference. Some pre-converted models can be found in the Hugging Face repository. The supported model formats are: PyTorch TensorFlow TensorFlow Lite ONNX (including formats that may be serialized to ONNX) PaddlePaddle JAX/Flax == OS support == OpenVINO runs on Windows, Linux and MacOS.

    Read more →
  • Zvi Mowshowitz

    Zvi Mowshowitz

    Zvi Mowshowitz is an American writer and member of the rationalist community who primarily discusses new developments in artificial intelligence. He is a former competitive Magic: The Gathering player and was CEO of MetaMed. == Career == Mowshowitz is an alumnus of Columbia University and holds a bachelor's degree in mathematics. He co-founded and was the CEO of MetaMed, a medical research analysis firm. He has worked at Jane Street Capital, and has worked for the gambling industry in Las Vegas. He attempted to launch a blockchain game, Emergents, in 2020. === Magic: The Gathering === Mowshowitz held a developer intern position at Wizards of the Coast R&D in 2005. He created the deck TurboZvi. His first-place finishes at major competitions were the 1999 World Championships as part of the four-person United States national team, the 2001 Pro Tour Tokyo, and two 2003 Grand Prix. He has placed in the top eight of four Pro Tours, and earned over $140,000 playing Magic competitively. In 2007, Mowshowitz was elected into the Magic Hall of Fame. Last updated: 12 May 2013Source: Wizards.com Mowshowitz has written about Magic for several outlets, including the official Magic website. === Later career === Mowshowitz is on the board of directors for the Center for Applied Rationality, and is a member of the rationalist community. He also founded Balsa Research, a nonprofit think tank which advocated for the repeal of the Jones Act, increasing the housing supply, and reform of the National Environmental Policy Act. In 2023, Mowshowitz wrote an article for Vox on the topic of artificial intelligence safety. Mowshowitz has a blog on Substack under the name "Don't Worry about the Vase". He has written on topics such as artificial intelligence, economics, and the COVID-19 pandemic. == Personal life == Mowshowitz is the son of American biochemist Deborah Mowshowitz. His parents have both worked as Columbia University professors.

    Read more →