SPACEMAP (Korean: 스페이스맵) is a South Korean satellite orbit optimization and satellite communications company headquartered in Seoul, South Korea. The company was founded in 2021 by CEO, Douglas Deok-Soo Kim, as an offshoot of Hanyang University. It was funded by the Leader Research grant from the National Research Foundation of Korea with the goal of capitalizing on the growing space industry. == History == Kim initially began research into Voronoi diagrams at the University of Michigan. He met with Dr. Misoon Ma, former director of the Asia Division of the U.S. Air Force Office of Scientific Research (AFOSR) and was recruited to work with the U.S. Air force, using Voronoi diagrams for a satellite collision prevention program. After his work with the U.S. Air Force, Kim founded SPACEMAP Inc in September 2021. In 2023, the company was selected by Korea's Tech Incubator Program for Startups (TIPS) to be funded up to 17 billion KRW (approx. US$13 million) in 3 years. == Technology == The services provided by SPACEMAP are based on using dynamic Voronoi diagrams to predict satellite orbits with the aim of enhancing space mission safety and efficiency. For complex problems involving many moving points, Voronoi diagrams maintain a near-constant computation time regardless of the number of points involved. By utilizing Voronoi diagrams and artificial intelligence, the software can easily determine the number of neighboring satellites surrounding a specific satellite and calculate the distances between them, thereby predicting the probability of a collision. SPACEMAP claims their method to be superior in computational time and memory efficiency, compared to the previously established three-filter method. == Products == SPACEMAP offers satellite products and services including the following: AstroOne, a conjunction assessment, and optimal collision avoidance service for all space vehicles in both orbital and non-orbital motions. AstroOrca, providing data transmission for satellites in multiple orbits, launch optimization, shuttle logistics for space gas stations, and Active Debris Removal (ADR) itinerary. AstroLibrary, a library of RESTful APIs to access the C++ implementation of SPACEMAP's Voronoi diagram algorithms wrapped in a Python interface. It also provides real-time tracking of the North Korean reconnaissance satellite, Malligyong-1.
Local-first software
Local-first software is a software engineering approach in which an application stores its data primarily on the user's own device rather than on remote servers. Users can read and write data without an Internet connection, and changes are synchronized across devices in the background when connectivity is available. The approach differs from conventional cloud-based applications, where the server holds the authoritative copy of user data and the client acts as a thin client. The term was coined in a 2019 paper published by researchers at Ink & Switch, an independent research lab, and presented at the Onward! conference at ACM SIGPLAN. The paper, sometimes referred to as a manifesto, was authored by Martin Kleppmann, Adam Wiggins, Peter van Hardenberg, and Mark McGranaghan. == Background == Before the widespread adoption of Internet-connected software in the 2000s, most desktop applications stored data as files on the user's local disk. Users had direct access to their files and could copy, back up, or delete them at will. The rise of software as a service (SaaS) and cloud-based applications like Google Docs shifted data storage to centralized servers. While cloud applications made real-time collaboration across devices straightforward, they introduced a dependency on the service provider: if the provider discontinued the service or experienced an outage, users could lose access to their data. A related concept, "offline-first," emerged in the early 2010s and focused on making web applications resilient to network interruptions. The local-first approach built on these earlier efforts while placing greater emphasis on long-term data ownership and end-to-end encryption. == Origins == === Ink & Switch manifesto === Ink & Switch is an industrial research lab co-founded by Adam Wiggins, who had earlier co-founded Heroku. Martin Kleppmann, an associate professor in the Department of Computer Science and Technology at the University of Cambridge, was a co-author of the 2019 paper. The manifesto proposed seven "ideals" for local-first software: Fast — Operations respond without network round-trips. Multi-device — Data synchronizes across a user's devices. Offline — Users can read and write data without a network connection. Collaboration — Multiple users can work on the same data concurrently. Longevity — Data remains accessible even if the software vendor ceases operation. Privacy — End-to-end encryption protects user data. User control — The vendor cannot restrict how users access or use their data. The paper surveyed existing approaches to data storage and collaboration — ranging from email attachments and Dropbox-style file synchronization to web applications and mobile backends — and argued that none of them satisfied all seven ideals simultaneously. === Role of CRDTs === The manifesto identified conflict-free replicated data types (CRDTs) as a promising technical foundation for local-first applications. CRDTs are data structures that allow multiple replicas to be edited independently and then merged without conflicts, a property first formalized in research by Marc Shapiro and colleagues around 2011. Kleppmann and collaborators at Ink & Switch developed Automerge, an open-source CRDT library for JSON documents, to make these algorithms available to application developers. == Adoption and community == Developer interest in the local-first approach grew after the 2019 paper spread on Hacker News and at developer conferences In August 2023, Wired published a feature article on the movement, describing it as an effort to reduce reliance on large cloud providers. The first Local-First Conf took place on 30 May 2024 in Berlin, with talks by Kleppmann and developers from companies including Linear and Anytype. The community has continued to expand, with regular "LoFi" meetups, a podcast (localfirst.fm), and a third edition of the conference planned for Berlin in July 2026. == Criticisms and limitations == Developers and commentators have pointed out practical difficulties with the local-first approach. Synchronizing data between multiple devices that may be offline for extended periods introduces complexity that cloud-based architectures avoid. Conflict resolution, even with CRDTs, can produce results that are technically consistent but semantically unexpected to users. Schema migrations across thousands of client devices running different application versions pose another difficulty that does not arise with server-side databases. Web browsers impose storage limits and may evict locally stored data. Safari, for instance, has been reported to clear IndexedDB data after seven days of inactivity on a given site, which undermines the assumption that local data is persistent. There is also disagreement within the local-first community about whether a fully decentralized architecture is required. The original manifesto described decentralization as the "logical end goal," but a number of products that identify as local-first still depend on centralized servers for authentication, backup, or synchronization. In a talk at Local-First Conf 2024, Kleppmann said the seven ideals are better understood as a "gradient" rather than a strict checklist.
Agentive logic
Agentive logic (also called the logic of action or logic of agency) is the field of philosophical logic and logic in computer science that studies formal representations of agents, their actions, and their abilities. An agentive logic in the narrower sense is a formal system whose primitive operators express that an agent does something, can do something, or sees to it that something is the case. Agentive logics generalise modal logic by adding modalities indexed to agents and to actions. Typical examples include: STIT logics (from sees to it that) with operators of the form [ i s t i t : φ ] {\displaystyle [i\ {\mathsf {stit}}:\varphi ]} meaning that agent i {\displaystyle i} sees to it that φ {\displaystyle \varphi } holds; dynamic logics of action with program-like modalities [ α ] φ {\displaystyle [\alpha ]\varphi } and ⟨ α ⟩ φ {\displaystyle \langle \alpha \rangle \varphi } meaning, roughly, that after every (respectively, some) execution(s) of action α {\displaystyle \alpha } , φ {\displaystyle \varphi } holds; logics with explicit agentive operators such as "can do", "brings about", or "is able to ensure". Agentive logics are used in action theory in philosophy, in the semantics of natural language, in the theory of program verification, and in artificial intelligence, where they underpin formalisms for reasoning about actions, planning, and intelligent agents. == Terminology and scope == The adjective agentive derives from the Latin agens ("one who acts") and originally referred to the grammatical agent of a verb. In logical contexts it designates operators or predicates whose primary argument position is an agent rather than a proposition alone, for example A i φ {\displaystyle A_{i}\varphi } ("agent i {\displaystyle i} does φ {\displaystyle \varphi } ") or C i φ {\displaystyle C_{i}\varphi } ("agent i {\displaystyle i} can bring about φ {\displaystyle \varphi } "). In contemporary literature, agentive logic is sometimes used narrowly for formal reconstructions of St. Anselm's modal account of facere ("to do"). More broadly, the term is used interchangeably with logic of action or logic of agency to cover a family of modal and dynamic logics designed to capture the structure of action and choice. == Historical background == === Medieval and early modern roots === Medieval logicians already explored analogies between modalities of action and alethic modalities such as possibility and necessity, for instance, in discussions of obligation and power. An influential early agentive analysis is due to St. Anselm (11th century), who treated "doing φ {\displaystyle \varphi } " as a kind of modal operator on propositions, anticipating later modal logics of agency. Modern reconstructions of Anselm's theory show that the resulting "agentive logic" can be modelled with neighbourhood semantics and satisfies a recognisable square of opposition. === Modern logic of action === Modern study of the logic of action began in the mid-20th century, parallel to developments in deontic logic and tense logic. Early systems were proposed by Georg Henrik von Wright, Stig Kanger, and others, often motivated by questions about norms and responsibility. From the 1960s onward, two largely independent but eventually converging traditions emerged: a branching-time tradition, culminating in STIT logics, emphasising agents' choices among possible futures; and dynamic logics of programs and actions, developed within computer science to reason about program execution. In the 1990s and 2000s, action logics were further developed in connection with knowledge representation, planning, and multi-agent systems in AI, and with dynamic and update semantics in linguistics. == Core ideas == Despite their diversity, most agentive logics share some general themes: Agents are treated as explicit indices of modal operators, as in [ i d o e s ] φ {\displaystyle [i\ {\mathsf {does}}]\varphi } or C i φ {\displaystyle C_{i}\varphi } . Actions are represented either implicitly, via changes between possible worlds along an accessibility relation, or explicitly, as terms denoting primitive and composite actions. Choice and ability are captured by modalities describing what an agent can ensure, usually relative to assumptions about the environment and other agents. Formal properties such as closure under composition, interaction between different agents, and connections to obligation (what an agent ought to do) and knowledge (what an agent knows how to do) are investigated. == STIT logics == STIT ("sees to it that") logics, originating in work by Nuel Belnap and collaborators, treat agency in a branching-time framework. A STIT model consists of a partially ordered set of moments with a tree-like structure, sets of histories (maximal branches through the tree), and for each agent at each moment, a partition of the histories through that moment representing the choices available to the agent. Intuitively, an agent's action at a moment determines which equivalence class (choice cell) of histories becomes actual; a formula [ i s t i t : φ ] {\displaystyle [i\ {\mathsf {stit}}:\varphi ]} is true at a history–moment pair if φ {\displaystyle \varphi } holds on all histories in the choice cell corresponding to the agent's current action. Different STIT operators have been distinguished, notably: the Chellas STIT operator, often written [ i c s t i t : φ ] {\displaystyle [i\ {\mathsf {cstit}}:\varphi ]} , which requires only that the agent's choice guarantees φ {\displaystyle \varphi } ; and the deliberative STIT operator, [ i d s t i t : φ ] {\displaystyle [i\ {\mathsf {dstit}}:\varphi ]} , which additionally requires that φ {\displaystyle \varphi } is not already historically necessary. STIT frameworks have been extended with group agency operators, temporal modalities, epistemic operators, and deontic operators to study responsibility, collective action, and obligations under indeterminism. == Dynamic logics of action == Dynamic logic was originally developed to reason about the behaviour of computer programs, treating program execution as a kind of action. In propositional dynamic logic (PDL), action terms α , β , … {\displaystyle \alpha ,\beta ,\dots } denote abstract programs or actions, and formulas of the form [ α ] φ {\displaystyle [\alpha ]\varphi } and ⟨ α ⟩ φ {\displaystyle \langle \alpha \rangle \varphi } express that all, respectively some, terminating executions of α {\displaystyle \alpha } lead to states where φ {\displaystyle \varphi } holds. From the standpoint of agentive logic, dynamic logic provides: a language for building complex actions from primitives via sequencing, choice, and iteration (e.g., α ; β {\displaystyle \alpha ;\beta } , α ∪ β {\displaystyle \alpha \cup \beta } , α ∗ {\displaystyle \alpha ^{}} ); a Kripke semantics in which actions correspond to labelled accessibility relations; and proof systems (such as Hoare logic and weakest precondition calculi) for reasoning about the correctness of action sequences. Extensions such as concurrent dynamic logic add operators for parallel composition, allowing reasoning about interacting processes and concurrent actions. John-Jules Ch. Meyer and others have argued that dynamic logic is a natural base for logics of agents, by adding modalities for knowledge, belief, and ability on top of the action modalities. Dynamic logics have also been applied to normative reasoning, yielding dynamic deontic logics where actions are related to obligations and permissions, and to dynamic epistemic logics in which information-changing actions such as announcements are modelled as programs. == Situation calculus and other action formalisms == In artificial intelligence, reasoning about action and change is often based on first-order languages that explicitly represent situations, events, and fluents (time-varying properties). The best known is situation calculus, introduced by John McCarthy and developed extensively by Raymond Reiter. In such formalisms: action terms name primitive actions; a function symbol (often d o {\displaystyle {\mathsf {do}}} ) maps an action and a situation to a successor situation; and axioms describe which fluents hold in which situations and how actions change them. Reiter's successor state axioms give compact specifications of how each fluent changes under all actions, and precondition axioms specify when actions are possible. Related formalisms include the event calculus and fluent calculus, which provide alternative ways of representing events and their effects. While these systems are often first-order rather than modal, they are closely related to agentive logics: their action terms and transition structures can be seen as providing models for dynamic or STIT-style modalities, and conversely, dynamic logics can be used as abstract specification languages for such AI formalisms. == Ability, agency, and related modalities == Many agentive logics introduce explicit operators for ability or "can-do"
Compute (machine learning)
In machine learning and deep learning, compute is the amount of computing power or computational resources required to train machine learning models and large language models. More broadly, compute is the computational power or resources necessary for a computer or computer program to function. == Definition == Compute is commonly defined as the amount of computing power or computational resources required to train machine learning and large language models. The term "compute" has also been more broadly applied to cloud computing, referencing processing power, memory, networking, storage, and other resources required for the computation of any program. Compute is measured in petaflop/s-days and is used to document AI training. A petaflop/s-day (pfs-day) consists of performing 1015 neural net operations per second for one day, or a total of about 1020 operations. The compute-time product serves as a mental convenience, similar to kilowatt-hour for energy. An amount of compute is meant to give an idea of the number of actual operations performed. == History == In a 2018 analysis titled "AI and compute", artificial intelligence company OpenAI introduced the concept of compute. OpenAI identified two eras of training AI systems in terms of compute-usage. From 1959 to 2012, compute roughly followed Moore’s law. Between 2012 and 2018, the amount of compute used in the largest AI training runs increased exponentially, growing by more than 300,000 times — roughly doubling every 3.4 months. By comparison, Moore’s Law doubled every two years over the same period. One of the largest models, released in 2020, used 600,000 times more computing power than the 2012 model. After 2020, compute growth began to slow down, with the compute needed for the largest AI models continuing to slow down in 2023. The notion of compute has become increasingly used from the mid-2020s onwards. == Compute growth and AI progress == Larger AI models trained on more data and using more computational resources, tend to perform better. This happens even if the algorithms themselves remain unchanged. As early as 2018, OpenAI noted the exponential increase in compute to be have a key role in AI progress. OpenAI considers three factors drive the advance of AI: algorithmic innovation, data, and the amount of compute available for training. AI models with more compute not only improve in the tasks they were trained on but can develop emergent abilities. Incremental improvements can lead to more abrupt leaps in capabilities. AI provider SpaceXAI said in 2026 that their AI progress is driven by compute and used it a key metric in the AI training of its supercomputer Colossus, the which contains 1 million GPUs. Anthropic has a contract of $1.25 billion per month with SpaceXAI to buy all the compute capacity at Colossus 1 data center. === Criticism and policy === Increasing, promoting or constraining progress in artificial intelligence has often be done via controlling the amount of compute. Policymarkers have enacted policies and provided support to make compute resources more accessible to domestic AI researchers. In a January 2022 report, the Center for Security and Emerging Technology (CSET) suggested to institutions that increasingly powerful and generalizable AI (AGI) will likely require other strategies than maximizing compute. Some AI researchers are also concerned that government might exclusively focus on scaling compute instead of other strategies. The CSET has reported on the various bottlenecks which could explain why deep learning needs for compute have slow down: training is expensive and training extremely large models generates traffic jams across many processors that are difficult to manage. there is a limited supply of AI chips (see AI chip memory shortage). CSET advances that the main resource is human capital, specifically talented researchers — according to a 2023 published survey of more than 400 AI researchers, academic and private sector workers. The survey found that AI researchers are not primarily or exclusively constrained by compute access. However, both academic and industry AI researchers equally report concerns that insufficient compute could prevent them from contributing meaningfully to AI research in the future. High compute users are more concerned about compute access. When asked about which resource provided by the government would be the most useful to them, some AI researchers select compute, other prefer grant funding. For this goal, CSET advised policymakers to ensure that even researchers with smaller budgets could effectively contribute to AI research. Other proposed strategies include using contemporary AI algorithms, managing modern AI infrastructure or focusing on interdisciplinary work between the AI field and other fields of computer science. A 2024 study on compute access found that academic-only AI research teams often have less compute intensive research topics, especially foundation models, compared to industry AI labs. As a consequence, academia is likely to play a smaller role in advancing such techniques. The researchers suggest nationally-sponsored computing infrastructure as well as open science initiatives to boost academic compute access. === Data === A 2022 study found that current large language models are significantly under-trained, a consequence of focusing on scaling language models whilst keeping the amount of training data constant. By training over 400 language models of various parameter and token size, they found that "for compute-optimal training", the model size and the number of training tokens should ideally be scaled equally: for every doubling of model size the number of training tokens should also be doubled.
POP-11
POP-11 is a reflective, incrementally compiled programming language with many of the features of an interpreted language. It is the core language of the Poplog programming environment developed originally by the University of Sussex, and recently in the School of Computer Science at the University of Birmingham, which hosts the main Poplog website. POP-11 is an evolution of the language POP-2, developed in Edinburgh University, and features an open stack model (like Forth, among others). It is mainly procedural, but supports declarative language constructs, including a pattern matcher, and is mostly used for research and teaching in artificial intelligence, although it has features sufficient for many other classes of problems. It is often used to introduce symbolic programming techniques to programmers of more conventional languages like Pascal, who find POP syntax more familiar than that of Lisp. One of POP-11's features is that it supports first-class functions. POP-11 is the core language of the Poplog system. The availability of the compiler and compiler subroutines at run-time (a requirement for incremental compiling) gives it the ability to support a far wider range of extensions (including run-time extensions, such as adding new data-types) than would be possible using only a macro facility. This made it possible for (optional) incremental compilers to be added for Prolog, Common Lisp and Standard ML, which could be added as required to support either mixed language development or development in the second language without using any POP-11 constructs. This made it possible for Poplog to be used by teachers, researchers, and developers who were interested in only one of the languages. The most successful product developed in POP-11 was the Clementine data mining system, developed by ISL. After SPSS bought ISL, they renamed Clementine to SPSS Modeler and decided to port it to C++ and Java, and eventually succeeded with great effort, and perhaps some loss of the flexibility provided by the use of an AI language. POP-11 was for a time available only as part of an expensive commercial package (Poplog), but since about 1999 it has been freely available as part of the open-source software version of Poplog, including various added packages and teaching libraries. An online version of ELIZA using POP-11 is available at Birmingham. At the University of Sussex, David Young used POP-11 in combination with C and Fortran to develop a suite of teaching and interactive development tools for image processing and vision, and has made them available in the Popvision extension to Poplog. == Simple code examples == Here is an example of a simple POP-11 program: define Double(Source) -> Result; Source2 -> Result; enddefine; Double(123) => That prints out: 246 This one includes some list processing: define RemoveElementsMatching(Element, Source) -> Result; lvars Index; [[% for Index in Source do unless Index = Element or Index matches Element then Index; endunless; endfor; %]] -> Result; enddefine; RemoveElementsMatching("the", [[the cat sat on the mat]]) => ;;; outputs [[cat sat on mat]] RemoveElementsMatching("the", [[the cat] [sat on] the mat]) => ;;; outputs [[the cat] [sat on] mat] RemoveElementsMatching([[= cat]], [[the cat]] is a [[big cat]]) => ;;; outputs [[is a]] Examples using the POP-11 pattern matcher, which makes it relatively easy for students to learn to develop sophisticated list-processing programs without having to treat patterns as tree structures accessed by 'head' and 'tail' functions (CAR and CDR in Lisp), can be found in the online introductory tutorial. The matcher is at the heart of the SimAgent (sim_agent) toolkit. Some of the powerful features of the toolkit, such as linking pattern variables to inline code variables, would have been very difficult to implement without the incremental compiler facilities.
Retrieval-augmented generation
Retrieval-augmented generation (RAG) is a technique that enables large language models (LLMs) to retrieve and incorporate new information from external data sources. With RAG, LLMs first refer to a specified set of documents, then respond to user queries. These documents supplement information from the LLM's pre-existing training data. This allows LLMs to use domain-specific and/or updated information that is not available in the training data. For example, this enables LLM-based chatbots to access internal company data or generate responses based on authoritative sources. RAG improves LLMs by incorporating information retrieval before generating responses. Unlike LLMs that rely on static training data, RAG pulls relevant text from databases, uploaded documents, or web sources. According to Ars Technica, "RAG is a way of improving LLM performance, in essence by blending the LLM process with a web search or other document look-up process to help LLMs stick to the facts." This method helps reduce AI hallucinations, which have caused chatbots to describe policies that don't exist, or recommend nonexistent legal cases to lawyers that are looking for citations to support their arguments. RAG also reduces the need to retrain LLMs with new data, saving on computational and financial costs. Beyond efficiency gains, RAG also allows LLMs to include sources in their responses, so users can verify the cited sources. This provides greater transparency, as users can cross-check retrieved content to ensure accuracy and relevance. The term retrieval-augmented generation (RAG) was introduced in a 2020 paper that described combining a parametric language model with a non-parametric external memory accessed through retrieval at inference time. == RAG and LLM limitations == LLMs can provide incorrect information. For example, when Google first demonstrated its LLM tool "Google Bard" (later re-branded to Gemini), the LLM provided incorrect information about the James Webb Space Telescope. This error contributed to a $100 billion decline in Google's stock value. RAG is used to prevent these errors, but it does not solve all the problems. For example, LLMs can generate misinformation even when pulling from factually correct sources if they misinterpret the context. MIT Technology Review gives the example of an AI-generated response stating, "The United States has had one Muslim president, Barack Hussein Obama." The model retrieved this from an academic book rhetorically titled Barack Hussein Obama: America's First Muslim President? The LLM did not "know" or "understand" the context of the title, generating a false statement. LLMs with RAG are programmed to prioritize new information. This technique has been called "prompt stuffing." Without prompt stuffing, the LLM's input is generated by a user; with prompt stuffing, additional relevant context is added to this input to guide the model's response. This approach provides the LLM with key information early in the prompt, encouraging it to prioritize the supplied data over pre-existing training knowledge. == Process == Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating an information-retrieval mechanism that allows models to access and utilize additional data beyond their original training set. Ars Technica notes that "when new information becomes available, rather than having to retrain the model, all that's needed is to augment the model's external knowledge base with the updated information" ("augmentation"). IBM states that "in the generative phase, the LLM draws from the augmented prompt and its internal representation of its training data to synthesize" an answer. === RAG key stages === Typically, the data to be referenced is converted into LLM embeddings, numerical representations in the form of a large vector space. RAG can be used on unstructured (usually text), semi-structured, or structured data (for example knowledge graphs). These embeddings are then stored in a vector database to allow for document retrieval. Given a user query, a document retriever is first called to select the most relevant documents that will be used to augment the query. This comparison can be done using a variety of methods, which depend in part on the type of indexing used. The model feeds this relevant retrieved information into the LLM via prompt engineering of the user's original query. Newer implementations (as of 2023) can also incorporate specific augmentation modules with abilities such as expanding queries into multiple domains and using memory and self-improvement to learn from previous retrievals. Finally, the LLM can generate output based on both the query and the retrieved documents. Some models incorporate extra steps to improve output, such as the re-ranking of retrieved information, context selection, and fine-tuning. == Applications == Retrieval-augmented generation is used in applications where generated responses need to be grounded in external or frequently updated information. Commonly cited use cases include search engines, question-answering systems, customer support chatbots, enterprise knowledge assistants, content generation, recommendation systems, retail and e-commerce, and industrial or manufacturing workflows. In healthcare, RAG has been studied as a way to ground large language model outputs in external medical knowledge sources, although reviews have noted continuing challenges around evaluation, ethics, and clinical reliability. == Improvements == Improvements to the basic process above can be applied at different stages in the RAG flow. === Encoder === These methods focus on the encoding of text as either dense or sparse vectors. Sparse vectors, which encode the identity of a word, are typically dictionary-length and contain mostly zeros. Dense vectors, which encode meaning, are more compact and contain fewer zeros. Various enhancements can improve the way similarities are calculated in the vector stores (databases). Performance improves by optimizing how vector similarities are calculated. Dot products enhance similarity scoring, while approximate nearest neighbor (ANN) searches improve retrieval efficiency over K-nearest neighbors (KNN) searches. Accuracy may be improved with Late Interactions, which allow the system to compare words more precisely after retrieval. This helps refine document ranking and improve search relevance. Hybrid vector approaches may be used to combine dense vector representations with sparse one-hot vectors, taking advantage of the computational efficiency of sparse dot products over dense vector operations. Other retrieval techniques focus on improving accuracy by refining how documents are selected. Some retrieval methods combine sparse representations, such as SPLADE, with query expansion strategies to improve search accuracy and recall. === Retriever-centric methods === These methods aim to enhance the quality of document retrieval in vector databases: Pre-training the retriever using the Inverse Cloze Task (ICT), a technique that helps the model learn retrieval patterns by predicting masked text within documents. Supervised retriever optimization aligns retrieval probabilities with the generator model's likelihood distribution. This involves retrieving the top-k vectors for a given prompt, scoring the generated response's perplexity, and minimizing KL divergence between the retriever's selections and the model's likelihoods to refine retrieval. Reranking techniques can refine retriever performance by prioritizing the most relevant retrieved documents during training. === Language model === By redesigning the language model with the retriever in mind, a 25-time smaller network can get comparable perplexity as its much larger counterparts. Because it is trained from scratch, this method (Retro) incurs the high cost of training runs that the original RAG scheme avoided. The hypothesis is that by giving domain knowledge during training, Retro needs less focus on the domain and can devote its smaller weight resources only to language semantics. The redesigned language model is shown here. It has been reported that Retro is not reproducible, so modifications were made to make it so. The more reproducible version is called Retro++ and includes in-context RAG. === Chunking === Chunking involves various strategies for breaking up the data into vectors so the retriever can find details in it. Three types of chunking strategies are: Fixed length with overlap. This is fast and easy. Overlapping consecutive chunks helps to maintain semantic context across chunks. Syntax-based chunks can break the document up into sentences. Libraries such as spaCy or NLTK can also help. File format-based chunking. Certain file types have natural chunks built in, and it's best to respect them. For example, code files are best chunked and vectorized as whole functions or classes. HTML files should leave