AI Analytics Certification

AI Analytics Certification — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  CityEngine

  ArcGIS CityEngine is a commercial 3D modeling program. Developed by Esri R&D Center Zurich (formerly Procedural Inc.), it specializes in the generation of 3D urban environments to support the creation of detailed large-scale 3D city models. Unlike traditional 3D modeling methodology, which uses computer-aided design (CAD) tools and techniques, CityEngine takes a procedural modeling approach which shapes generation via a rules-based system. Due to its integration with the wider ArcGIS platform, CityEngine can also be used with geographic information system (GIS) datasets. CityEngine can be used for urban planning and architecture, graphics visualization, game development, entertainment, and archeology. CityEngine can be used to visualize the building information modeling (BIM) data of buildings in a larger urban context, making for more realistic construction projects. == History and releases == === Software history === ArcGIS CityEngine, originally named Esri CityEngine, was developed at Swiss technology university ETH Zurich by Pascal Mueller, the co-founder and CEO of Procedural Inc. While researching for his PhD at the ETH Computer Vision Lab, Mueller invented a number of techniques for procedural modeling of 3D architecture that make up the foundation of CityEngine. CityEngine publically debuted at the 2001 SIGGRAPH conference; since then, additional research papers have been published that have contributed to CityEngine and its features. The first commercial version of CityEngine was released in 2008. In 2007, Procedural Inc. was founded and separated from ETH Zurich, the top-ranking technology university in Switzerland. In the summer of 2011, Procedural Inc. was acquired by Esri Inc., becoming Esri R&D Center Zurich. Esri CityEngine was renamed to ArcGIS CityEngine in June 2020 to offically make it a part of the ArcGIS software suite. === Releases === === Licensing and pricing === ArcGIS CityEngine is included in the Professional and Professional Plus tiers of ArcGIS Online. Pricing may vary by region and distributors. In the US, the professional tier costs US$2,200 per year; in the UK, it is £4,200 per year (excluding VAT). CityEngine can be purchased elsewhere via a local Esri partner. . Once purchased, users can download and obtain license details from the MyEsri portal. == Features == CGA (computer generated architecture) parametric modeling rules to control mass, geometry assets, proportions, or texturing of buildings or streets on a citywide scale Select a target location and import geo-referenced satellite imagery and 3D terrain of the location to more quickly build accurate urban environments through OpenStreetMap integration Interactively control specific street or building parameters, such as height or age Import/export geo-spatial/vector data with industry-standard formats such as Esri Shapefile, File Geodatabase, and OpenStreetMap, as well as file formats for WebGL, KMZ, Collada, Autodesk FBX, Autodesk Maya, 3DS, Wavefront OBJ, RenderMan RIB, Alembic, e-on software's Vue, Universal Scene Description USD, Khronos Group GLTF, Unreal Engine, and Unreal Datasmith Script and generate rules-based reports to show socioeconomic figures (e.g., Gross Floor Area (GFA) and Floor Area Ratio (FAR)) to analyze their urban design proposals. VR viewing of modeled environments with Samsung Gear VR Use a variety of materials through the Esri materials library == Procedural modeling == ArcGIS CityEngine uses a procedural modeling approach to automatically generate models through a predefined rule set. The rules are defined through a CGA shape grammar system, enabling the creation of complex parametric models. Users can change or add the shape grammar as needed. Urban environments can be modeled within CityEngine by starting with creating a street network (either from the street drawing tool or with data imported from map data). Then, lots may be subdivided as many times as specified, resulting in a map of multiple lots and streets. CityEngine can then be instructed to start generating the buildings using defined procedural modeling rules. At this point, the city model can be re-designed and adjusted by changing the parameters or the shape grammar. === Geodesign === Though CityEngine is not an analytical tool like GIS, discussions about geodesign often mention the use of ArcGIS CityEngine. As it can be used to enhance 3D shape generation in ArcGIS, ArcGIS CityEngine is a critical product to improve the applicability of geodesign by using geospatial information to design or analyze a city. == Applications == === Urban design and planning === Garsdale Design used ArcGIS CityEngine in the creation of city master plans in Iraq before 2013, both to model existing historic areas and also model future plans. Larger companies like Foster+Partners and HOK Architects have also used CityEngine in their urban planning projects. === Urban and environmental studies === Because its primary feature is building informative city models, some urban researchers use CityEngine to compare land-use planning schemes, for example in very dense global cities such as Hong Kong and Seoul. Environmental scientists can also utilize the instant 3D model generation in CityEngine, which can make for more convenient informative research than modeling a city by creating each building individually. === Game development === CityEngine can be used as a tool in the creation of video games that require detailed 3D environments to assign interactive scripts. === Movie industry === Zootopia (also known outside of the US as Zootopolis), which won the 2016 Academy Award for Best Animated Feature Film, used CityEngine to model the city in its movie. multi-scaling city, the designers used CityEngine due to its rule-based system. CityEngine was also used to create Big Hero 6's San-Fransokyo. === Military === Due to its integration with the Esri product suite and its ability to process geospatial data to create 3D scenes/maps, CityEngine can be used within military/defense organizations. == List of movies and TV shows using CityEngine == Studios and companies rarely state what software they use in their pipelines. When CityEngine is mentioned as a tool in production, it's often in a small reference in a larger article. Movies only claimed to use CityEngine by a single Esri employee Presented at FMX 2025 workshop == Ports == ArcGIS CityEngine is built on top of Eclipse IDE, and has therefore able to be used on Windows and Linux operating systems. Support for macOS was stopped in March 2021. == Plugins and extensions == ArcGIS CityEngine currently works with a number of third party 3D modeling, rendering, and analytical software products via its SDK and API; these currently are: ArcGIS CityEngine for ArcGIS Urban: ArcGIS Urban Suite Puma: ArcGIS CityEngine for Rhinoceros 3D Palladio: ArcGIS CityEngine for Houdini Serlio: ArcGIS CityEngine for Maya PyPRT: ArcGIS CityEngine for Python ArcGIS CityEngine provides a Python scripting interface built on Jython (current version 2.7.0) which allows users to create their own tools and functionality. == Publications ==

  Read more →
• 

  Foma (software)

  Foma is a free and open source finite-state toolkit created and maintained by Mans Hulden. It includes a compiler, programming language, and C library for constructing finite-state automata and transducers (FST's) for various uses, most typically Natural Language Processing uses such as morphological analysis. Foma can replace the proprietary Xerox Finite State Toolkit for compiling and running FST's written in the lexc and xfst formalisms. The speed is comparable with the Xerox tools for most lexicons, although Foma can be 3 or 4 times slower for very large lexicons (e.g. >100,000 words). Foma is also one of the possible backends of the free and open source Helsinki Finite State Toolkit (where other backends provide support for further formalisms). There are several FOSS morphologies written in lexc/xfst compatible with foma, e.g. for the Sámi, Cornish, Faroese, Finnish, Komi, Mari, Udmurt, Buriat, Greenlandic language and Iñupiaq languages.

  Read more →
• 

  Markov chain

  In probability theory and statistics, a Markov chain or Markov process is a stochastic process describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC). Markov processes are named in honor of the Russian mathematician Andrey Markov. Markov chains have many applications as statistical models of real-world processes. They provide the basis for general stochastic simulation methods known as Markov chain Monte Carlo, which are used for simulating sampling from complex probability distributions, and have found application in areas including Bayesian statistics, biology, chemistry, economics, finance, information theory, physics, signal processing, and speech processing. The adjectives Markovian and Markov are used to describe something that is related to a Markov process. == Principles == === Definition === A Markov process is a stochastic process that satisfies the Markov property (sometimes characterized as "memorylessness"). In simpler terms, it is a process for which predictions can be made regarding future outcomes based solely on its present state and—most importantly—such predictions are just as good as the ones that could be made knowing the process's full history. In other words, conditional on the present state of the system, its future and past states are independent. A Markov chain is a type of Markov process that has either a discrete state space or a discrete index set (often representing time), but the precise definition of a Markov chain varies. For example, it is common to define a Markov chain as a Markov process in either discrete or continuous time with a countable state space (thus regardless of the nature of time), but it is also common to define a Markov chain as having discrete time in either countable or continuous state space (thus regardless of the state space). === Types of Markov chains === The system's state space and time parameter index need to be specified. The following table gives an overview of the different instances of Markov processes for different levels of state space generality for both discrete and continuous time: Note that there is no definitive agreement in the literature on the use of some of the terms that signify special cases of Markov processes. Usually the term "Markov chain" is reserved for a process with a discrete set of times, that is, a discrete-time Markov chain (DTMC), but a few authors use the term "Markov process" to refer to a continuous-time Markov chain (CTMC) without explicit mention. In addition, there are other extensions of Markov processes that are referred to as such but do not necessarily fall within any of these four categories (see Markov model). Moreover, the time index need not necessarily be real-valued; like with the state space, there are conceivable processes that move through index sets with other mathematical constructs. Notice that the general state space continuous-time Markov chain is general to such a degree that it has no designated term. While the time parameter is usually discrete, the state space of a Markov chain does not have any generally agreed-on restrictions: the term may refer to a process on an arbitrary state space. However, many applications of Markov chains employ finite or countably infinite state spaces, which have a more straightforward statistical analysis. Besides time-index and state-space parameters, there are many other variations, extensions and generalizations (see Variations). For simplicity, most of this article concentrates on the discrete-time, discrete state-space case, unless mentioned otherwise. === Transitions === The changes of state of the system are called transitions. The probabilities associated with various state changes are called transition probabilities. The process is characterized by a state space, a transition matrix describing the probabilities of particular transitions, and an initial state (or initial distribution) across the state space. By convention, we assume all possible states and transitions have been included in the definition of the process, so there is always a next state, and the process does not terminate. A discrete-time random process involves a system which is in a certain state at each step, with the state changing randomly between steps. The steps are often thought of as moments in time, but they can equally well refer to physical distance or any other discrete measurement. Formally, the steps are the integers or natural numbers, and the random process is a mapping of these to states. The Markov property states that the conditional probability distribution for the system at the next step (and in fact at all future steps) depends only on the current state of the system, and not additionally on the state of the system at previous steps. Since the system changes randomly, it is generally impossible to predict with certainty the state of a Markov chain at a given point in the future. However, the statistical properties of the system's future can be predicted. In many applications, it is these statistical properties that are important. == History == Andrey Markov studied Markov processes in the early 20th century, publishing his first paper on the topic in 1906. Markov processes in continuous time were discovered long before his work in the early 20th century in the form of the Poisson process. Markov was interested in studying an extension of independent random sequences, motivated by a disagreement with Pavel Nekrasov who claimed independence was necessary for the weak law of large numbers to hold. In his first paper on Markov chains, published in 1906, Markov showed that under certain conditions the average outcomes of the Markov chain would converge to a fixed vector of values, so proving a weak law of large numbers without the independence assumption, which had been commonly regarded as a requirement for such mathematical laws to hold. Markov later used Markov chains to study the distribution of vowels in Eugene Onegin, written by Alexander Pushkin, and proved a central limit theorem for such chains. In 1912 Henri Poincaré studied Markov chains on finite groups with an aim to study card shuffling. Other early uses of Markov chains include a diffusion model, introduced by Paul and Tatyana Ehrenfest in 1907, and a branching process, introduced by Francis Galton and Henry William Watson in 1873, preceding the work of Markov. After the work of Galton and Watson, it was later revealed that their branching process had been independently discovered and studied around three decades earlier by Irénée-Jules Bienaymé. Starting in 1928, Maurice Fréchet became interested in Markov chains, eventually resulting in him publishing in 1938 a detailed study on Markov chains. Andrey Kolmogorov developed in a 1931 paper a large part of the early theory of continuous-time Markov processes. Kolmogorov was partly inspired by Louis Bachelier's 1900 work on fluctuations in the stock market as well as Norbert Wiener's work on Einstein's model of Brownian movement. He introduced and studied a particular set of Markov processes known as diffusion processes, where he derived a set of differential equations describing the processes. Independent of Kolmogorov's work, Sydney Chapman derived in a 1928 paper an equation, now called the Chapman–Kolmogorov equation, in a less mathematically rigorous way than Kolmogorov, while studying Brownian movement. The differential equations are now called the Kolmogorov equations or the Kolmogorov–Chapman equations. Other mathematicians who contributed significantly to the foundations of Markov processes include William Feller, starting in 1930s, and then later Eugene Dynkin, starting in the 1950s. == Examples == Mark V. Shaney is a third-order Markov chain program, and a Markov text generator. It ingests the sample text (the Tao Te Ching, or the posts of a Usenet group) and creates a massive list of every sequence of three successive words (triplet) which occurs in the text. It then chooses two words at random, and looks for a word which follows those two in one of the triplets in its massive list. If there is more than one, it picks at random (identical triplets count separately, so a sequence which occurs twice is twice as likely to be picked as one which only occurs once). It then adds that word to the generated text. Then, in the same way, it picks a triplet that starts with the second and third words in the generated text, and that gives a fourth word. It adds the fourth word, then repeats with the third and fourth words, and so on. Random walks based on integers and the gambler's ruin problem are ex

  Read more →
• 

  Intelligent character recognition

  Intelligent character recognition (ICR) is a method of extracting handwritten text from images. It is a more sophisticated type of OCR technology that recognizes different handwriting styles and fonts to intelligently interpret data from physical documents. ICR is used to organize paper-based unstructured data by scanning documents, extracting information, and adapting extracted data for database storage. ICR algorithms collaborate with OCR to automate data entry from forms by removing the need for keystrokes. It has a high degree of accuracy and is a dependable method for processing various handwritten media quickly. == Capabilities == Most ICR software has a self-learning neural network-based algorithms, which automatically update the recognition database for new handwriting patterns. It extends the usefulness of scanning devices for the purpose of document processing, from printed character recognition (a function of OCR) to hand-written matter recognition. Because this process is involved in recognizing hand writing, accuracy levels may, in some circumstances, not be very good but can achieve 97%+ accuracy rates in reading handwriting in structured forms. Often to achieve these high recognition rates several read engines are used within the software and each is given elective voting rights to determine the true reading of characters. In numeric fields, engines which are designed to read numbers take preference, while in alpha fields, engines designed to read hand written letters have higher elective rights. When used in conjunction with a bespoke interface hub, hand-written data can be automatically populated into a back office system avoiding laborious manual keying and can be more accurate than traditional human data entry. === Automated forms processing === An important development of ICR was the invention of automated forms processing in 1993 by Joseph Corcoran who was awarded a patent on the invention. This involved a three-stage process of capturing the image of the form to be processed by ICR and preparing it to enable the ICR engine to give best results, then capturing the information using the ICR engine and finally processing the results to automatically validate the output from the ICR engine. This application of ICR increased the usefulness of the technology and made it applicable for use with real world forms in normal business applications. Modern software applications use ICR as a technology of recognizing text in forms filled in by hand (hand-printed). == Differences between ICR and OCR == === OCR === Optical character recognition (OCR) is commonly considered to apply to any recognition technique that reads machine printed text. An example of a traditional OCR use case would be to translate the characters from an image of a printed document, such as a book page, newspaper clipping, or legal contract, into a separate file that could be searched and updated with a word processor or document viewer. It's also quite helpful for automating the processing of forms. Information can be swiftly extracted from form fields and entered into another application, like a spreadsheet or database, by zonally applying the OCR engine to those fields. Yet, data is typically manually input rather than typed into form fields. Character identification becomes even more challenging while reading handwritten material. The diversity of more than 700,000 printed font variants is tiny compared to the near unlimited variations in hand-printed characters. The recognition program must take into account not just stylistic differences but also the kind of writing implement used, the standard of the paper, errors, hand stability, and smudges or running ink. === ICR === Intelligent character recognition (ICR) makes use of continuously improving algorithms to collect more information about the variances in hand-printed characters and more precisely identify them. ICR, which was created in the early 1990s to aid in the automation of forms processing, enables the conversion of manually entered data into text that is simple to read, search for, and change. When used to read characters that are obviously divided into distinct areas or zones, such as fixed fields seen on many structured forms, it works best. Both OCR and ICR can be configured to read a variety of languages; however, limiting the expected character set to a smaller number of languages will produce better recognition outcomes. ICR cannot read cursive handwriting since it must still be able to assess each character individually. While writing in cursive, it might be difficult to tell where one character ends and another one begins, and there are more differences across samples than when hand-printing text. A more recent method called intelligent word recognition (IWR) focuses on reading a word in context rather than recognizing individual characters. == Intelligent word recognition == Intelligent word recognition (IWR) can recognize and extract not only printed-handwritten information, cursive handwriting as well. ICR recognizes on the character-level, whereas IWR works with full words or phrases. Capable of capturing unstructured information from every day pages, IWR is said to be more evolved than hand print ICR. Not meant to replace conventional ICR and OCR systems, IWR is optimized for processing real-world documents that contain mostly free-form, hard-to-recognize data fields that are inherently unsuitable for ICR. This means that the highest and best use of IWR is to eliminate a high percentage of the manual entry of handwritten data and run-on hand print fields on documents that otherwise could be keyed only by humans.

  Read more →
• 

  Outline of brain mapping

  The following outline is provided as an overview of and topical guide to brain mapping: Brain mapping – set of neuroscience techniques predicated on the mapping of (biological) quantities or properties onto spatial representations of the (human or non-human) brain resulting in maps. Brain mapping is further defined as the study of the anatomy and function of the brain and spinal cord through the use of imaging (including intra-operative, microscopic, endoscopic and multi-modality imaging), immunohistochemistry, molecular and optogenetics, stem cell and cellular biology, engineering (material, electrical and biomedical), neurophysiology and nanotechnology. == Broad scope == History of neuroscience History of neurology Brain mapping Human brain Neuroscience Nervous system. === The neuron doctrine === Neuron doctrine – A set of carefully constructed elementary set of observations regarding neurons. For more granularity, more current, and more advanced topics, see the cellular level section Asserts that neurons fall under the broader cell theory, which postulates: All living organisms are composed of one or more cells. The cell is the basic unit of structure, function, and organization in all organisms. All cells come from preexisting, living cells. The Neuron doctrine postulates several elementary aspects of neurons: The brain is made up of individual cells (neurons) that contain specialized features such as dendrites, a cell body, and an axon. Neurons are cells differentiable from other tissues in the body. Neurons differ in size, shape, and structure according to their location or functional specialization. Every neuron has a nucleus, which is the trophic center of the cell (The part which must have access to nutrition). If the cell is divided, only the portion containing the nucleus will survive. Nerve fibers are the result of cell processes and the outgrowths of nerve cells. (Several axons are bound together to form one nerve fibril. See also: Neurofilament. Several nerve fibrils then form one large nerve fiber. Myelin, an electrical insulator, forms around selected axons. Neurons are generated by cell division. Neurons are connected by sites of contact and not via cytoplasmic continuity. (A cell membrane isolates the inside of the cell from its environment. Neurons do not communicate via direct cytoplasm to cytoplasm contact.) Law of dynamic polarization. Although the axon can conduct in both directions, in tissue there is a preferred direction of transmission from cell to cell. Elements added later to the initial Neuron doctrine A barrier to transmission exists at the site of contact between two neurons that may permit transmission. (Synapse) Unity of transmission. If a contact is made between two cells, then that contact can be either excitatory or inhibitory, but will always be of the same type. Dale's law, each nerve terminal releases a single type of neurotransmitter. Some of the basic postulates in the Neuron doctrine have been subsequently questioned, refuted, or updated. See the cellular level section topics for additional information. === Map, atlas, and database projects === Brain Activity Map Project – 2013 NIH $3 billion project to map every neuron in the human brain in ten years, based upon the Human Genome Project. NIH Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative [1] Community outreach site for above where the public may comment [2] Human Brain Project (EU) – 1 billion euro, 10-year project to simulate the human brain with supercomputers. BigBrain A high-resolution 3D atlas of the human brain created as part of the HBP. Human Connectome Project – 2009 NIH $30 million project to build a network map of the human brain, including structural (anatomical) and functional elements. Emphasis included research into dyslexia, autism, Alzheimer's disease, and schizophrenia. See also Connectome a, comprehensive map of neural connections in the brain. Allen Brain Atlas – 2003 $100 million project funded by Paul Allen (Microsoft) BrainMaps – National Institute of Health (NIH) database including 60 terabytes of image scans of primate and non-primates, integrated with information covering structure and function. NeuroNames – Defines the brain in terms of about 550 primary structures (about 850 unique structures) to which all other structures, names, and synonyms are related. About 15,000 neuroanatomical terms are cross indexed, including many synonyms in seven languages. Coverage includes the brain and spinal cord of the four species most frequently studied by neuroscientists: human, macaque (monkey), rat and mouse. The controlled, standardized vocabulary for each structure is located in an unambiguous, strict physical hierarchy, and these terms are selected based on ease of pronunciation, mnemonic value, and frequency of use in recent neuroscientific publications. Relation of each structure to its superstructures and substructures is included. The controlled vocabulary is suitable for uniquely indexing neuroanatomical information in digital databases. Decade of the Brain 1990–1999 promotion by NIH and the Library of Congress "to enhance public awareness of the benefits to be derived from brain research". Communications targeted Members of Congress, staffs, and the general public to promote funding. Talairach Atlas see Jean Talairach Harvard Whole Brain Atlas see Human brain MNI Template see Medical image computing Blue Brain Project and Artificial brain International Consortium for Brain Mapping see Brain Mapping List of neuroscience databases NIH Toolbox National Institute of Health (USA) toolbox for the assessment of neurological and behavioral function Organization for Human Brain Mapping The Organization for Human Brain Mapping (OHBM) is an international society dedicated to using neuroimaging to discover the organization of the human brain. == Imaging and recording systems == This section covers imaging and recording systems. The general section covers history, neuroimaging, and techniques for mapping specific neural connections. The specific systems section covers the various specific technologies, including experimental and widely deployed imaging and recording systems. === General === Most imaging work to date on individual neurons has been conducted outside the brain, typically on large neurons, and has been most frequently destructive. New techniques are however rapidly emerging. Search on "Single neuron imaging" and see related topics: Biological neuron model, Single-unit recording, Neural oscillation, Computational neuroscience. dMRI (above) is also promising in non-destructive imaging of single neurons inside the brain. History of neuroimaging (redirects from Brain scanner) Neuroimaging (redirects from Brain function map) Connectomics – mapping technique showing neural connections in a nervous system. === Specific systems === Cortical stimulation mapping Diffusion MRI (dMRI) – includes diffusion tensor imaging (DTI) and diffusion functional MRI (DfMRI). dMRI is a recent breakthrough in brain mapping allowing the visualization of cross connections between different anatomical parts of the brain. It allows noninvasive imaging of white matter fiber structure and in addition to mapping can be useful in clinical observations of abnormalities, including damage from stroke. Electroencephalography (EEG) – uses electrodes on the scalp and other techniques to detect the electrical flow of currents. Electrocorticography – intracranial EEG, the practice of using electrodes placed directly on the exposed surface of the brain to record electrical activity from the cerebral cortex. Electrophysiological techniques for clinical diagnosis Functional magnetic resonance imaging (fMRI) Medical image computing (brain research of leads medical and surgical uses of mapping technology) Neurostimulation (in research stimulation is frequently used in conjunction with imaging) Positron emission tomography (PET) – a nuclear medical imaging technique that produces a three-dimensional image or picture of functional processes in the body. The system detects pairs of gamma rays emitted indirectly by a positron-emitting radionuclide (tracer), which is introduced into the body on a biologically active molecule. Three-dimensional images of tracer concentration within the body are then constructed by computer analysis. In modern scanners, three dimensional imaging is often accomplished with the aid of a CT X-ray scan performed on the patient during the same session, in the same machine. === Imaging and recording componentry === ==== Electrochemical ==== Haemodynamic response – the rapid delivery of blood to active neuronal tissues. Blood Oxygenation Level Dependent signal (BOLD), corresponds to the concentration of deoxyhemoglobin. The BOLD effect is based on the fact that when neuronal activity is increased in one part of the brain, there is also an increased amount of cerebral blood flow to that area. Functional m

  Read more →
• 

  Factored language model

  The factored language model (FLM) is an extension of a conventional language model introduced by Jeff Bilmes and Katrin Kirchoff in 2003. In an FLM, each word is viewed as a vector of k factors: w i = { f i 1 , . . . , f i k } . {\displaystyle w_{i}=\{f_{i}^{1},...,f_{i}^{k}\}.} An FLM provides the probabilistic model P ( f | f 1 , . . . , f N ) {\displaystyle P(f|f_{1},...,f_{N})} where the prediction of a factor f {\displaystyle f} is based on N {\displaystyle N} parents { f 1 , . . . , f N } {\displaystyle \{f_{1},...,f_{N}\}} . For example, if w {\displaystyle w} represents a word token and t {\displaystyle t} represents a Part of speech tag for English, the expression P ( w i | w i − 2 , w i − 1 , t i − 1 ) {\displaystyle P(w_{i}|w_{i-2},w_{i-1},t_{i-1})} gives a model for predicting current word token based on a traditional Ngram model as well as the Part of speech tag of the previous word. A major advantage of factored language models is that they allow users to specify linguistic knowledge such as the relationship between word tokens and Part of speech in English, or morphological information (stems, root, etc.) in Arabic. Like N-gram models, smoothing techniques are necessary in parameter estimation. In particular, generalized back-off is used in training an FLM.

  Read more →
• 

  The Best Free AI Text-to-video Tool for Beginners

  Shopping for the best AI text-to-video tool? An AI text-to-video tool is software that uses machine learning to help you get more done — it keeps getting smarter as the underlying models improve. Pricing, accuracy, and the size of the model behind the tool are the three factors that most affect daily usefulness. Whether you are a beginner or a pro, the right AI text-to-video tool slots into your workflow and pays for itself fast. We tested the leading options and ranked them by quality, value, and ease of use.

  Read more →
• 

  François Chollet

  François Chollet (French: [fʁɑ̃swa ʃoˈlɛ]; born 20 October 1989) is a French software engineer, artificial intelligence (AI) researcher, and former Senior Staff Engineer at Google. Chollet is the creator of the Keras deep-learning library released in 2015. His research focuses on computer vision, the application of machine learning to formal reasoning, abstraction, and how to achieve greater generality in artificial intelligence (AGI). == Education and career == In 2012, Chollet graduated with a Diplôme d'Ingénieur (Master of Engineering) from ENSTA Paris, a school of the Polytechnic Institute of Paris. In 2015, Chollet started working at Google shortly after releasing Keras. In 2019, he published the Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI) benchmark, which measures the ability of AI systems to solve novel reasoning problems. In 2024, Chollet launched ARC Prize, a US$1 million competition to solve the ARC-AGI benchmark. He left Google in November 2024 after more than 9 years with the company to found with Zapier co-founder Mike Knoop a new startup focused on developing AGI with program synthesis. In early 2025, Chollet announced the expansion of ARC Prize into a full-fledged non-profit foundation, to further the mission of guiding and accelerating research progress towards artificial general intelligence. == Books and publications == Chollet's research papers in artificial intelligence have been published at major conferences in the field, including the Conference on Computer Vision and Pattern Recognition (CVPR), the Conference on Neural Information Processing Systems (NeurIPS), and the International Conference on Learning Representations (ICLR). Chollet is the author of Xception: Deep Learning with Depthwise Separable Convolutions, which is among the top ten most cited papers in CVPR proceedings at more than 18,000 citations. Chollet is the author of the book Deep Learning with Python, which sold over 100,000 copies, and the co-author with Tomasz Kalinowski of Deep Learning With R. == Awards == On December 1, 2021, Chollet won the Global Swiss AI Award for breakthroughs in AI. In September 2024, Chollet was named by TIME as one of the 100 most influential people in AI.

  Read more →
• 

  Python (programming language)

  Python is a high-level, general-purpose programming language that emphasizes code readability, simplicity, and ease-of-writing with the use of significant indentation, "plain English" naming, an extensive ("batteries-included") standard library, and garbage collection. Python supports multiple programming paradigms but with an emphasis on object-oriented programming and dynamic typing. Guido van Rossum began working on Python in the late 1980s as a successor to the ABC programming language. Python 3.0, released in 2008, was a major revision and not completely backward-compatible with earlier versions. Beginning with Python 3.5, capabilities and keywords for typing were added to the language, allowing optional static typing. As of 2026, the Python Software Foundation supports Python 3.10, 3.11, 3.12, 3.13, and 3.14, following the project's annual release cycle and five-year support policy. Python 3.15 is currently in the alpha development phase, and the stable release is expected to launch in October 2026. Earlier versions in the 3.x series have reached end-of-life and no longer receive security updates. Python has gained extensive use in the machine learning community. It is widely taught as an introductory programming language. Since 2003, Python has consistently ranked among the top ten most popular programming languages in the TIOBE Programming Community Index, which ranks programming languages based on searches across 24 platforms. == History == Python was conceived in the late 1980s by Guido van Rossum at Centrum Wiskunde & Informatica (CWI) in the Netherlands. It was designed as a successor to the ABC programming language, which was inspired by SETL, capable of exception handling and interfacing with the Amoeba operating system. Python implementation began in December 1989. Van Rossum first released it in 1991 as Python 0.9.0. Van Rossum assumed sole responsibility for the project, as the lead developer, until 12 July 2018, when he announced his "permanent vacation" from responsibilities as Python's "benevolent dictator for life" (BDFL); this title was bestowed on him by the Python community to reflect his long-term commitment as the project's chief decision-maker. (He has since come out of retirement and is self-titled "BDFL-emeritus".) In January 2019, active Python core developers elected a five-member Steering Council to lead the project. The name Python derives from the British comedy series Monty Python's Flying Circus. (See § Naming.) Python 2.0 was released on 16 October 2000, featuring many new features such as list comprehensions, cycle-detecting garbage collection, reference counting, and Unicode support. Python 2.7's end-of-life was initially set for 2015, and then postponed to 2020 out of concern that a large body of existing code could not easily be forward-ported to Python 3. It no longer receives security patches or updates. While Python 2.7 and older versions are officially unsupported, a different unofficial Python implementation, PyPy, continues to support Python 2, i.e., "2.7.18+" (plus 3.11), with the plus signifying (at least some) "backported security updates". Python 3.0 was released on 3 December 2008, and was a major revision and not completely backward-compatible with earlier versions, with some new semantics and changed syntax. Python 2.7.18, released in 2020, was the last release of Python 2. Several releases in the Python 3.x series have added new syntax to the language, and made a few (considered very minor) backward-incompatible changes. As of May 2026, Python 3.14.5 is the latest stable release. All older 3.x versions had a security update down to Python 3.9.24 then again with 3.9.25, the final version in 3.9 series. Python 3.10 is, since November 2025, the oldest supported branch. Python 3.15 has an alpha released, and Android has an official downloadable executable available for Python 3.14. Releases receive two years of full support followed by three years of security support. == Design philosophy and features == Python is a multi-paradigm programming language. Object-oriented programming and structured programming are fully supported, and many of their features support functional programming and aspect-oriented programming – including metaprogramming and metaobjects. Many other paradigms are supported via extensions, including design by contract and logic programming. Python is often referred to as a 'glue language' because it is purposely designed to be able to integrate components written in other languages. Python uses dynamic typing and a combination of reference counting and a cycle-detecting garbage collector for memory management. It uses dynamic name resolution (late binding), which binds method and variable names during program execution. Python's design offers some support for functional programming in the "Lisp tradition". It has filter, map, and reduce functions; list comprehensions, dictionaries, sets, and generator expressions. The standard library has two modules (itertools and functools) that implement functional tools borrowed from Haskell and Standard ML. Python's core philosophy is summarized in the Zen of Python (PEP 20) written by Tim Peters, which includes aphorisms such as these: Explicit is better than implicit. Simple is better than complex. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity, errors should never pass silently, unless explicitly silenced. There should be one-- and preferably only one --obvious way to do it. However, Python has received criticism for violating these principles and adding unnecessary language bloat. Responses to these criticisms note that the Zen of Python is a guideline rather than a rule. The addition of some new features had been controversial: Guido van Rossum resigned as Benevolent Dictator for Life after conflict about adding the assignment expression operator in Python 3.8. Nevertheless, rather than building all functionality into its core, Python was designed to be highly extensible through modules. This compact modularity has made it particularly popular as a means of adding programmable interfaces to existing applications. Van Rossum's vision of a small core language with a large standard library and an easily extensible interpreter stemmed from his frustrations with ABC, which represented the opposite approach. Python claims to strive for a simpler, less-cluttered syntax and grammar, while giving developers a choice in their coding methodology. Python lacks do .. while loops, which Rossum considered harmful. In contrast to Perl's motto "there is more than one way to do it", Python advocates an approach where "there should be one – and preferably only one – obvious way to do it". In practice, however, Python provides many ways to achieve a given goal. There are at least three ways to format a string literal, with no certainty as to which one a programmer should use. Alex Martelli is a Fellow at the Python Software Foundation and Python book author; he wrote that "To describe something as 'clever' is not considered a compliment in the Python culture." Python's developers typically prioritize readability over performance. For example, they reject patches to non-critical parts of the CPython reference implementation that would offer increases in speed that do not justify the cost of clarity and readability. Execution speed can be improved by moving speed-critical functions to extension modules written in languages such as C, or by using a just-in-time compiler like PyPy. Also, it is possible to transpile to other languages. However, this approach either fails to achieve the expected speed-up, since Python is a very dynamic language, or only a restricted subset of Python is compiled (with potential minor semantic changes). Python is meant to be a fun language to use. This goal is reflected in the name – a tribute to the British comedy group Monty Python – and in playful approaches to some tutorials and reference materials. For instance, some code examples use the terms "spam" and "eggs" (in reference to a Monty Python sketch), rather than the typical terms "foo" and "bar". A common neologism in the Python community is pythonic, which has a broad range of meanings related to program style: Pythonic code may use Python idioms well; be natural or show fluency in the language; or conform with Python's minimalist philosophy and emphasis on readability. === Enhancement Proposals === Python Enhancement Proposals are a design document for either providing information to the Python community, or proposal for new feature in Python. PEPs are intented to explain new processes in Python, provide naming conventions or document the processes in the language. PEPs are overseen by Python Steering Council. There are 3 kinds of PEPs, with those are being standards track PEP, Informational PEP and Process PEPs which has their own unique meanings. They were firstly introduced in 2000, in

  Read more →
• 

  Is an AI Chatbot Worth It in 2026?

  Curious about the best AI chatbot? An AI chatbot is software that uses machine learning to help you get more done — it combines speed, accuracy, and an interface that just works. Hands-on testing shows real-world results vary, so a short free trial is the smartest way to decide. Whether you are a beginner or a pro, the right AI chatbot slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.

  Read more →
• 

  IBM optical mark and character readers

  IBM designed, manufactured and sold optical mark and character readers from 1960 until 1984. The IBM 1287 is notable as being the first commercially sold scanner capable of reading handwritten numbers. == Initial development work == IBM Poughkeepsie studied machine character recognition from 1950 till 1954, developing an experimental machine that used a cathode-ray-tube attached an IBM 701 which performed the character analysis. They pursued a technique known as lakes and bays which examined different areas of dark and light where the lakes were white areas enclosed by black and the bays were partially enclosed areas. Their machine and mission was moved to IBM Endicott in 1954, where research continued. From 1955 to 1956 they then worked on the VIDOR (Visual Document Reader) program, but they could not get agreement on acceptable reject rate. The developers felt 80% recognition was acceptable (meaning 20% of documents would need to be manually processed), while product planners and IBM Marketing felt that compared to punched card, the reject rate was unacceptably high. This led to no new products being released. In 1956 the American Bankers Association chose to use Magnetic Ink Character Recognition (MICR) to automate check handling, rejecting a proposed solution generated by an IBM Poughkeepsie banking project that used optical characters formed by vertical bars and digits. IBM developed a magnetic read head to handle the new standard, releasing the IBM 1210 MICR reader/sorter in 1959. The development work for this product both with read heads and document handling, helped move optical character recognition forward, with development focusing on reading one or two lines of print from a paper document larger than an IBM punched card. The first product to be released was the IBM 1418. == IBM 123x Optical Mark Readers == The IBM 1230, IBM 1231, and IBM 1232 were optical mark readers used to input the contents of data sources such as questionnaires, test results, surveys as well as historical data that could be easily entered as marks on sheets. Educational institutes used them to score test results and they were effectively a replacement for the IBM 805 Test Scoring Machine that used electrical resistance and a mark sense pencil to score a test, rather than optical mark detection. They were developed and manufactured by IBM Rochester. They have the following features: A pneumatic input hopper that can hold approximately 600 sheets Two output stackers: the normal stacker that holds 600 sheets and the select (or reject) stacker which holds 50 sheets. Pluggable SMS printed circuit cards They can read positional marks made by a lead pencil using an optical read head that consists of photovoltaic(solar) cells and lamps The 1230 has 21 photovoltaic cells, 20 for reading the pencil marks and one to read timing marks on the right hand border of the sheet. The 1231 and 1232 have 22 photovoltaic cells, 20 to read data, one to read timing marks and one to read a special feature called a master mark. Input size is a 8+1⁄2 in × 11 in (22 cm × 28 cm) sheet called a data sheet that can have up to 1000 marked or printed positions per side. Uses electromechanical devices known as sonic delay lines to store results. === IBM 1230 Optical Mark Scoring Reader === The IBM 1230 is an offline optical mark scoring machine announced on 2 November 1962 that was designed to read and scores 1,200 answer sheets per hour. Scored results are printed via a wire matrix printer on the right margin of each answer sheet as it is processed. Two master sheets are required for the process: one that encoded the correct answers and one for the machine to record run information. Output could be sent to an IBM 534 Model 3 Card Punch as an option, which limits throughput to 750 sheets per hour when punching 80 columns of data. === IBM 1231 Optical Mark Page Reader === The IBM 1231 is an online optical mark reader that was designed to read and score 2000 test answer sheets per hour, depending on downstream operations. The correct answers for the test can either be entered using a master sheet (like the 1230) or sent to the 1231 using the optional master-mark special feature. === IBM 1232 Optical Mark Page Reader === The IBM 1232 is an offline optical mark reader that was designed to read up to 2000 marked sheets per hour. Documents can be read at up to 2000 sheets per hour, but this depends on the number of characters that need to be punched from each sheet. The IBM 1232 reads the marks and then punches them into cards using a IBM 534 Model 3 Card Punch. Together they can read up to 64,000 characters per hour or 800 fully punched cards. === Example customers === The California Test Bureau (CTB) that provided standardised achievement tests for educational institutes across the USA, began replacing their IBM 805s with IBM 1230s in 1963. They then installed two IBM 1232s in 1964. Being able to use a full 8+1⁄2 in × 11 in (22 cm × 28 cm) answer sheet rather than a 7+3⁄8 in × 3+1⁄4 in (18.7 cm × 8.3 cm) mark sense card, eliminated the need to use multiple answer cards per test per student, as well as dramatically increased the marking speed for test answers. Credit Bureau Services of Dallas used an IBM 1232 in 1966 as part of their first computerisation project. They marked credit history data onto optical scanning sheets that were fed into their IBM 1232. The attached IBM 534 then punched this data onto punched cards, which were then fed into their IBM System/360 Model 30. In 1968 the US Army Corps of Engineers Coastal Engineering Research Center (CERC) began using special log books for their coastal surveyors to record coastal survey data, which was then converted to punched cards by an IBM 1232. == IBM 2956 Optical Mark/Hole Reader == The IBM 2956 Models 2 and 3 are custom build optical mark/hole readers designed to be attached to an IBM 2740 Communications Terminal. The IBM 2956-2 can read cards that have either been hand or machine marked or that have been punched. The cards can be fed by hand or from the 400 card hopper. It has a 400 card stacker. The 2956-2 could be ordered by request for price quotation (RPQ) 843086. The IBM 2956-3 can read cards that have either been hand or machine marked or that have been punched. It can also read marked sheets up to 9 in × 14 in (230 mm × 360 mm) in size, although only a 3+1⁄4 in (83 mm) band along the side of the sheet can be read (the width of a punched card). It does not have a hopper or a stacker, so each card or sheet must be manually fed into the machine. The 2956-3 could be ordered by request for price quotation (RPQ) 843106. The 2956-3 could be attached to an IBM 3276 or IBM 3278 display station with RPQ UB9001. One use case for the IBM 2956 is to grade school tests. On completion of a learning module a student can use an optical scan-type card to record answers to up to 27 questions, with up to 5 choices per question. They are scanned by the reader and the results are then transmitted to an IBM System/360 in remote job entry mode and can also be printed on the IBM 2740. The reader can also be attached to an IBM 3735 which transmits results to an IBM System/370 and which prints results on an IBM 3286 printer. They can also be attached to an IBM System/3. Note that the IBM 2956 Model 5 (2956-5) was a banking reader/sorter. == IBM 1282 Optical Reader Card Punch == The IBM 1282 is an offline optical reader that is used to read embossed credit card receipts, a mark read field or machine printed characters in three different fonts. It then outputs this data onto a punched card. It was developed and manufactured by IBM Endicott. It proved popular and within two years of announcement 100 machines were installed or on order. === Example customer === The New York Department of Motor Vehicles reported that from 1964 until 1968 they were using an IBM 1282 to read machine printed license renewal slips that had been mailed back as part of the renewal process. They would scan the slip and then process the resulting punched card. This worked well until the DMV decided to request renewals include the drivers Social Security Number (SSN), which meant a handwritten number needed to be either manually keyed or a new scanning device procured. They switched to the IBM 1287 in 1968. == IBM 1285 Optical Reader == The IBM 1285 is an online optical reader that is used to read printed paper tapes from cash registers or adding machines. It was developed by IBM Endicott and manufactured by IBM Rochester. The IBM 1285 attaches to an IBM 1401, 1440, 1460 or System/360. It has a small round screen to display characters being read and it has a keyboard to enter header information and to optionally enter character corrections for rejected characters. It can read a 200 ft (61 m) roll or paper tape in three-and-a half minutes, reading data at speeds of up to 3000 lines per minute. It can mark the tape with a dot to indicate unreadable characters, so they can be r

  Read more →
• 

  Corpus manager

  A corpus manager (corpus browser or corpus query system) is a tool for multilingual corpus analysis, which allows effective searching in corpora. A corpus manager usually represents a complex tool that allows one to perform searches for language forms or sequences. It may provide information about the context or allow the user to search by positional attributes, such as lemma, tag, etc. These are called concordances. Other features include the ability to search for collocations, frequency statistics as well as metadata information about the processed text. The narrower meaning of corpus manager refers only to the server side or the corpus query engine, whereas the client side is simply called the user interface. A corpus manager can be software installed on a personal computer or it might be provided as a web service. == List of corpus managers == BNCweb – a web-based interface for the British National Corpus CQPweb - a web-based interface for the study of a large variety of corpora including the Spoken BNC2014 BYU-BNC – a website that allows searches of the British National Corpora and others created at Brigham Young University Coma – a tool extension of the system EXMARaLDA for working with oral corpora on a computer NoSketch Engine – a free open-source corpus management system combining Manatee (back-end) and Bonito (web interface) KonText – an extended and modified web interface to NoSketch Engine (a Bonito replacement) Sketch Engine – text corpus management and analysis software with more than 500 corpora in 90+ languages Spoco WordSmith Tools – a software package primarily for linguists

  Read more →
• 

  Game Jolt

  Game Jolt is a social community platform for video games, gamers and content creators. Founded by Yaprak and David DeCarmine, it is available on iOS, Android, and on the web and as a desktop app for Windows and Linux. Users share interactive content through a variety of formats including images, videos, live streams, chat rooms, and virtual events. == Features == === Crowd streaming === In 2021 Game Jolt revealed their own live streaming feature called Firesides. Firesides allowed multiple users to simultaneously livestream together with nearly no delay. The feature launched with a virtual concert showcasing its ability to accommodate multiple streamers. On October 16, 2023, Firesides were removed from Game Jolt. === Mobile app === Game Jolt Social by Game Jolt Inc. launched on both the Apple App Store and Google Play Store in March 2022. "It's clear to us that Gen Z is tired of generic social media and they want a place specifically for gaming that supports all types of content they're creating–art, videos, thoughts, and livestreams all in one place." said Game Jolt founder and CEO Yaprak DeCarmine, in a statement to VentureBeat. === Game API === The Game Jolt Application Programming Interface (usually known as the Game Jolt Game API) allows any developer using a game development platform that supports HTTP operations and MD5 or SHA-1. Game Jolt advertises that the API can: Create multiple "scoreboards" which collect high scores from players made publicly available on the game's profile and give user accounts EXP Award player's trophies which give user accounts EXP Store game data on Game Jolt's data servers Log whether a user is currently playing a game they're logged into via the GJAPI == Game jams and competitions == Game Jolt regularly hosts game jams where participants are encouraged to develop games for a chance to win prizes. They hosted their first game jam in 2009, Shocking Contest. In November 2014, Game Jolt announced the "Indies vs PewDiePie" game jam, partnering with the popular YouTuber Felix "PewDiePie" Kjellberg. Developers were given a weekend (21–24 November) to create a game with the theme of "fun to play, fun to watch" to suit the Let's Plays entertainment style. Users could rate entries afterwards until December 1 when the scores were counted up. The prize to the top 10 rated games was Felix playing the games on his channel as a means of promotion for the developers, although later he played other entries. One of the participants of the jam, now known as Outerminds Inc. was discovered and hired by PewDiePie to develop his mobile game, Legend of the Brofist. Game Jolt partnered with Felix, Sean "Jacksepticeye" McLoughlin and Mark "Markiplier" Fischbach to host "Indies vs Gamers" in July 2015. The requirements for entries were arcade games using the Game Jolt Game API highscore tables, to be made between the July 17–20 and the top 5 games were played on the partner's YouTube channels. Following the "Indies vs PewDiePie" game jam in 2014, Game Jolt released their internal jam hosting tools public for all users to use as a service, to create their own game jams that integrated with the main site. Today, Game Jolt focuses on hosting and co-hosting game competitions with established brands in order to bring monetary and educational opportunities to their users. On April 15, 2024, an announcement was made about a collaboration with Pocket Worlds for the "HighRise Game Jam". Pocket Worlds had sold NFTs up until roughly 2022, causing a community outburst. The situation was addressed, and the situation started to disperse. == Contests == == Events == Game Jolt hosts both physical and virtual events to entertain and prank its users, which consists of the following: == History == Game Jolt has supported independent creators with a central platform to manage their content and communities since its start in 2003. David DeCarmine began development of Game Jolt at the age of 14 for a group of hobbyists, making games and sharing on forums in an early iteration known as Holo World. The original intention was to create a platform for gamers where new games could be discoverable and quickly playable, and where feedback could be provided directly to the creators, allowing them to continue improving their games. In 2008, Game Jolt was registered as an LLC, then incorporated as Game Jolt Inc. in September 2020. A new site launched in 2015 featuring a responsive design, automated curation for both games and game news articles which weighs how recent a game was uploaded and how popular it is ("hot") and filtering options on game listings for platform, maturity rating and development status. In March 2022, Game Jolt launched a mobile application simultaneously on the Google Play Store and Apple App Store targeted at Gen Z gamers and creators. While in beta, the mobile app had 100,000 installs pre-launch. === Game store === Game Jolt continues to host a large library of independent games. Game developers can upload their games directly to the site to share or sell. They would allow distribution for downloadable games, later adding support for Adobe Flash, Unity and Java games which allowed support for browser based games. In February 2013, Game Jolt built support for browser-based HTML5 games as well. A user levelling system was released into public beta in April 2013, incorporating the GJAPI trophies and highscores, as well as site activity, to generate 'EXP' (experience points). Game Jolt Jams released in early 2014 as a service to allow users to create their own game jams that integrated with the main site. In April 2016, an online marketplace was announced and released the following month with an exclusive set of game titles, including Bendy and the Ink Machine, allowing developers to sell their games on the site. In January 2016, Game Jolt released source code of the client and site's front end on GitHub under MIT license. In January 2022, Game Jolt banned adult games from appearing on the site, stating in an email to developers that the site had become a "social media platform" and they "had to make decisions around the direction and future of the brand which has now included the removal of hosted games with explicitly adult content." In response to a tweet by Itch.io saying the site is not for prudes, they wrote in their own tweet: "Game Jolt is a platform with a large audience of 13-16 year olds. Our users asked us to clean up, so here we are." == Investments == After bootstrapping Game Jolt with revenue earned from ads on the website for years, the DeCarmines secured venture capital in 2020 from SoftBank, doing so again in 2021 from founders of Twitch, Rec Room, Modio and more.

  Read more →
• 

  How to Choose an AI Art Generator

  Looking for the best AI art generator? An AI art generator is software that uses machine learning to help you get more done — it can save you hours every week by automating repetitive work. Most options offer a generous free tier, with paid plans unlocking higher limits, faster processing, and team features. Whether you are a beginner or a pro, the right AI art generator slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.

  Read more →
• 

  Multiple sequence alignment

  Multiple sequence alignment (MSA) is the process or the result of sequence alignment of three or more biological sequences, generally protein, DNA, or RNA. These alignments are used to infer evolutionary relationships via phylogenetic analysis and can highlight homologous features between sequences. Alignments highlight mutation events such as point mutations (single amino acid or nucleotide changes), insertion mutations and deletion mutations, and alignments are used to assess sequence conservation and infer the presence and activity of protein domains, tertiary structures, secondary structures, and individual amino acids or nucleotides. Multiple sequence alignments require more sophisticated methodologies than pairwise alignments, as they are more computationally complex. Most multiple sequence alignment programs use heuristic methods rather than global optimization because identifying the optimal alignment between more than a few sequences of moderate length is prohibitively computationally expensive. However, heuristic methods generally cannot guarantee high-quality solutions and have been shown to fail to yield near-optimal solutions on benchmark test cases. == Problem statement == Given m {\displaystyle m} sequences S i {\displaystyle S_{i}} , i = 1 , ⋯ , m {\displaystyle i=1,\cdots ,m} similar to the form below: S := { S 1 = ( S 11 , S 12 , … , S 1 n 1 ) S 2 = ( S 21 , S 22 , ⋯ , S 2 n 2 ) ⋮ S m = ( S m 1 , S m 2 , … , S m n m ) {\displaystyle S:={\begin{cases}S_{1}=(S_{11},S_{12},\ldots ,S_{1n_{1}})\\S_{2}=(S_{21},S_{22},\cdots ,S_{2n_{2}})\\\,\,\,\,\,\,\,\,\,\,\vdots \\S_{m}=(S_{m1},S_{m2},\ldots ,S_{mn_{m}})\end{cases}}} A multiple sequence alignment is taken of this set of sequences S {\displaystyle S} by inserting any amount of gaps needed into each of the S i {\displaystyle S_{i}} sequences of S {\displaystyle S} until the modified sequences, S i ′ {\displaystyle S'_{i}} , all conform to length L ≥ max { n i ∣ i = 1 , … , m } {\displaystyle L\geq \max\{n_{i}\mid i=1,\ldots ,m\}} and no values in the sequences of S {\displaystyle S} of the same column consists of only gaps. The mathematical form of an MSA of the above sequence set is shown below: S ′ := { S 1 ′ = ( S 11 ′ , S 12 ′ , … , S 1 L ′ ) S 2 ′ = ( S 21 ′ , S 22 ′ , … , S 2 L ′ ) ⋮ S m ′ = ( S m 1 ′ , S m 2 ′ , … , S m L ′ ) {\displaystyle S':={\begin{cases}S'_{1}=(S'_{11},S'_{12},\ldots ,S'_{1L})\\S'_{2}=(S'_{21},S'_{22},\ldots ,S'_{2L})\\\,\,\,\,\,\,\,\,\,\,\vdots \\S'_{m}=(S'_{m1},S'_{m2},\ldots ,S'_{mL})\end{cases}}} To return from each particular sequence S i ′ {\displaystyle S'_{i}} to S i {\displaystyle S_{i}} , remove all gaps. == Graphing approach == A general approach when calculating multiple sequence alignments is to use graphs to identify all of the different alignments. When finding alignments via graph, a complete alignment is created in a weighted graph that contains a set of vertices and a set of edges. Each of the graph edges has a weight based on a certain heuristic that helps to score each alignment or subset of the original graph. === Tracing alignments === When determining the best suited alignments for each MSA, a trace is usually generated. A trace is a set of realized, or corresponding and aligned, vertices that has a specific weight based on the edges that are selected between corresponding vertices. When choosing traces for a set of sequences it is necessary to choose a trace with a maximum weight to get the best alignment of the sequences. == Alignment methods == There are various alignment methods used within multiple sequence to maximize scores and correctness of alignments. Each is usually based on a certain heuristic with an insight into the evolutionary process. Most try to replicate evolution to get the most realistic alignment possible to best predict relations between sequences. === Dynamic programming === A direct method for producing an MSA uses the dynamic programming technique to identify the globally optimal alignment solution. For proteins, this method usually involves two sets of parameters: a gap penalty and a substitution matrix assigning scores or probabilities to the alignment of each possible pair of amino acids based on the similarity of the amino acids' chemical properties and the evolutionary probability of the mutation. For nucleotide sequences, a similar gap penalty is used, but a much simpler substitution matrix, wherein only identical matches and mismatches are considered, is typical. The scores in the substitution matrix may be either all positive or a mix of positive and negative in the case of a global alignment, but must be both positive and negative, in the case of a local alignment. For n individual sequences, the naive method requires constructing the n-dimensional equivalent of the matrix formed in standard pairwise sequence alignment. The search space thus increases exponentially with increasing n and is also strongly dependent on sequence length. Expressed with the big O notation commonly used to measure computational complexity, a naïve MSA takes O(LengthNseqs) time to produce. To find the global optimum for n sequences this way has been shown to be an NP-complete problem. In 1989, based on Carrillo-Lipman Algorithm, Altschul introduced a practical method that uses pairwise alignments to constrain the n-dimensional search space. In this approach pairwise dynamic programming alignments are performed on each pair of sequences in the query set, and only the space near the n-dimensional intersection of these alignments is searched for the n-way alignment. The MSA program optimizes the sum of all of the pairs of characters at each position in the alignment (the so-called sum of pair score) and has been implemented in a software program for constructing multiple sequence alignments. In 2019, Hosseininasab and van Hoeve showed that by using decision diagrams, MSA may be modeled in polynomial space complexity. === Progressive alignment construction === The most widely used approach to multiple sequence alignments uses a heuristic search known as progressive technique (also known as the hierarchical or tree method) developed by Da-Fei Feng and Doolittle in 1987. Progressive alignment builds up a final MSA by combining pairwise alignments beginning with the most similar pair and progressing to the most distantly related. All progressive alignment methods require two stages: a first stage in which the relationships between the sequences are represented as a phylogenetic tree, called a guide tree, and a second step in which the MSA is built by adding the sequences sequentially to the growing MSA according to the guide tree. The initial guide tree is determined by an efficient clustering method such as neighbor-joining or unweighted pair group method with arithmetic mean (UPGMA), and may use distances based on the number of identical two-letter sub-sequences (as in FASTA rather than a dynamic programming alignment). Progressive alignments are not guaranteed to be globally optimal. The primary problem is that when errors are made at any stage in growing the MSA, these errors are then propagated through to the final result. Performance is also particularly bad when all of the sequences in the set are rather distantly related. Most modern progressive methods modify their scoring function with a secondary weighting function that assigns scaling factors to individual members of the query set in a nonlinear fashion based on their phylogenetic distance from their nearest neighbors. This corrects for non-random selection of the sequences given to the alignment program. Progressive alignment methods are efficient enough to implement on a large scale for many (100s to 1000s) sequences. A popular progressive alignment method has been the Clustal family. ClustalW is used extensively for phylogenetic tree construction, in spite of the author's explicit warnings that unedited alignments should not be used in such studies and as input for protein structure prediction by homology modeling. European Bioinformatics Institute (EMBL-EBI) announced that CLustalW2 will expire in August 2015. They recommend Clustal Omega which performs based on seeded guide trees and HMM profile-profile techniques for protein alignments. An alternative tool for progressive DNA alignments is multiple alignment using fast Fourier transform (MAFFT). Another common progressive alignment method named T-Coffee is slower than Clustal and its derivatives but generally produces more accurate alignments for distantly related sequence sets. T-Coffee calculates pairwise alignments by combining the direct alignment of the pair with indirect alignments that aligns each sequence of the pair to a third sequence. It uses the output from Clustal as well as another local alignment program LALIGN, which finds multiple regions of local alignment between two sequences. The resulting alignment and phylogenetic tree are used as a guide to produce new and more accurate w

  Read more →