AI Email Gen

AI Email Gen — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

  • Database application

    Database application

    A database application is a computer program whose primary purpose is retrieving information from a computerized database. From here, information can be inserted, modified or deleted which is subsequently conveyed back into the database. Early examples of database applications were accounting systems and airline reservations systems, such as SABRE, developed starting in 1957. A characteristic of modern database applications is that they facilitate simultaneous updates and queries from multiple users. Systems in the 1970s might have accomplished this by having each user in front of a 3270 terminal to a mainframe computer. By the mid-1980s it was becoming more common to give each user a personal computer and have a program running on that PC that is connected to a database server. Information would be pulled from the database, transmitted over a network, and then arranged, graphed, or otherwise formatted by the program running on the PC. Starting in the mid-1990s it became more common to build database applications with a Web interface. Rather than develop custom software to run on a user's PC, the user would use the same Web browser program for every application. A database application with a Web interface had the advantage that it could be used on devices of different sizes, with different hardware, and with different operating systems. Examples of early database applications with Web interfaces include amazon.com, which used the Oracle relational database management system, the photo.net online community, whose implementation on top of Oracle was described in the book Database-Backed Web Sites (Ziff-Davis Press; May 1997), and eBay, also running Oracle. Electronic medical records are referred to on emrexperts.com, in December 2010, as "a software database application". A 2005 O'Reilly book uses the term in its title: Database Applications and the Web. Some of the most complex database applications remain accounting systems, such as SAP, which may contain thousands of tables in only a single module. Many of today's most widely used computer systems are database applications, for example, Facebook, which was built on top of MySQL. The etymology of the phrase "database application" comes from the practice of dividing computer software into systems programs, such as the operating system, compilers, the file system, and tools such as the database management system, and application programs, such as a payroll check processor. On a standard PC running Microsoft Windows, for example, the Windows operating system contains all of the systems programs while games, word processors, spreadsheet programs, photo editing programs, etc. would be application programs. As "application" is short for "application program", "database application" is short for "database application program". Not every program that uses a database would typically be considered a "database application". For example, many physics experiments, e.g., the Large Hadron Collider, generate massive data sets that programs subsequently analyze. The data sets constitute a "database", though they are not typically managed with a standard relational database management system. The computer programs that analyze the data are primarily developed to answer hypotheses, not to put information back into the database and therefore the overall program would not be called a "database application". == Examples of database applications == Amazon Student Data CNN eBay Facebook Fandango Filemaker (Mac OS) LibreOffice Base Microsoft Access Oracle relational database SAP (Systems, Applications & Products in Data Processing) Ticketmaster Wikipedia Yelp YouTube Google MySQL

    Read more →
  • Chris Callison-Burch

    Chris Callison-Burch

    Chris Callison-Burch is an American computer scientist and professor of computer and information science at the University of Pennsylvania (Penn), specializing in natural language processing (NLP), artificial intelligence (AI), and crowdsourcing. He is recognised for his contributions to machine translation, paraphrase generation, and the application of large language models (LLMs) to AI challenges, with over 200 publications cited more than 33,000 times. Callison-Burch has influenced public policy on AI and copyright, testifying before the U.S. Congress in 2023 on generative AI’s implications. He serves as the faculty director for Penn’s Online Master of Science in Engineering in AI program. == Education == Callison-Burch earned his PhD in Computer Science from the University of Edinburgh in 2008, focusing on machine translation and paraphrasing techniques. His doctoral research developed statistical methods for generating paraphrases in machine translation systems, laying the foundation for his later NLP work. Prior to his PhD, he studied at Stanford University, where he developed an interest in computational linguistics. == Career == After his PhD, Callison-Burch joined the Centre for Language and Speech Processing at Johns Hopkins University as a research faculty member from 2008 to 2013, working on NLP projects, including machine translation and crowdsourcing for creating training data. In 2013, he joined the University of Pennsylvania as an assistant professor in the Department of Computer and Information Science and was promoted to associate professor in 2017, and to full professor in 2024. At Penn, Callison-Burch teaches courses on AI and NLP, including CIS 5300 (Natural Language Processing) and CIS 5210 (Artificial Intelligence), which attract over 500 students annually. He directs Penn’s Online Master of Science in Engineering in AI program, launched in 2025. He teaches AI and NLP courses on Coursera, reaching thousands of global learners. Callison-Burch was a part-time visiting researcher at Google in 2019 and 2020, where he collaborated on applying Google's LLM to Dungeons & Dragons dialogues. In 2023, he took a sabbatical at the Allen Institute for AI (AI2), where he contributed to vision-language models. == Research == Callison-Burch’s research focuses on NLP, AI, and crowdsourcing, with significant contributions to machine translation, paraphrase generation, and LLMs for tasks like text simplification and bias detection. His early work developed crowdsourcing methods for machine translation, leveraging non-expert annotators for paraphrase-based evaluation, influencing platforms like Amazon Mechanical Turk. Recent projects have included several notable works. Molmo and PixMo (2025) are open-weight vision-language models developed with AI2, achieving state-of-the-art multimodal performance and earning a Best Paper Honourable Mention at CVPR 2025. Also in 2025, his work on Calibrating Large Language Models with Sample Consistency improves LLM reliability via sample-based calibration, presented at NAACL 2025. The Media Bias Detector (2025) is a real-time tool analysing selection and framing bias in news, using LLMs to detect persuasive language differences (e.g., Russian vs. English Wikipedia). Holodeck (2024) is a language-guided system for generating 3D embodied AI environments, presented at CVPR 2024. BORDIRLINES (2024) is a dataset for cross-lingual retrieval-augmented generation, focusing on culturally sensitive tasks. He has co-authored over 200 publications, featured at conferences like ACL, EMNLP, and CVPR. == Awards and recognition == Callison-Burch has received numerous awards: Best Paper Honourable Mention at CVPR 2025 for "Molmo and PixMo". Best Paper Award at the Workshop on Cognitive Modelling and Computational Linguistics (CMCL) 2024 for "Evaluating Vision-Language Models on Bistable Images". Best Paper Award at STARSEM 2016 for "So-Called Non-Subsective Adjectives". Best Paper Award at the Workshop on Sense, Concept and Entity Representations 2017 for "Word Sense Filtering Improves Embedding-Based Lexical Substitution". Honourable Mention Award at CHI 2018 for "A Data-Driven Analysis of Workers’ Earnings on Amazon Mechanical Turk". Google Faculty Research Award (2013) for crowdsourcing in NLP. Sloan Research Fellowship (2014). He has received research funding from Google, Microsoft, Amazon, Facebook, Roblox, DARPA, IARPA, and NSF. His h-index is 72, with over 33,000 citations. He served as General Chair of ACL 2017 and as the Program Co-Chair EMNLP 2015. == Public policy and testimony == On May 17, 2023, Callison-Burch testified before the U.S. House Subcommittee on Courts, Intellectual Property, and the Internet on AI and copyright law. His testimony emphasised generative AI’s role in creative industries and the need for balanced copyright frameworks. He has appeared on Fox News to discuss AI’s societal impact, and discussed its impact with other print news sources. He contributes to AI ethics discussions, including workshops on AI’s effects on writing and creative professions.

    Read more →
  • Top 10 AI Paragraph Rewriters Compared (2026)

    Top 10 AI Paragraph Rewriters Compared (2026)

    Trying to pick the best AI paragraph rewriter? An AI paragraph rewriter is software that uses machine learning to help you get more done — it scales effortlessly from a single task to thousands. The best picks balance beginner-friendly simplicity with the depth power users need, and they ship updates often. Whether you are a beginner or a pro, the right AI paragraph rewriter slots into your workflow and pays for itself fast. This guide breaks down the top picks, their pros and cons, and who each one is best for.

    Read more →
  • AI Logo Makers: Free vs Paid (2026)

    AI Logo Makers: Free vs Paid (2026)

    Comparing the best AI logo maker? An AI logo maker is software that uses machine learning to help you get more done — it lowers the barrier so anyone can produce professional output. Privacy matters too: check whether your data trains the model and whether a no-log or enterprise tier is available. Whether you are a beginner or a pro, the right AI logo maker slots into your workflow and pays for itself fast. Below we compare features, pricing, and real output so you can choose with confidence.

    Read more →
  • Artificial consciousness

    Artificial consciousness

    Artificial consciousness, also known as machine consciousness, synthetic consciousness, or digital consciousness, is consciousness hypothesized to be possible for artificial intelligence. It is also the corresponding field of study, which draws insights from philosophy of mind, philosophy of artificial intelligence, cognitive science and neuroscience. The term "sentience" can be used when specifically designating ethical considerations stemming from a form of phenomenal consciousness (P-consciousness, or the ability to feel qualia). Since sentience involves the ability to experience ethically positive or negative (i.e., valenced) mental states, it may justify welfare concerns and legal protection, as with non-human animals. Some scholars believe that consciousness is generated by the interoperation of various parts of the brain; these mechanisms are labeled the neural correlates of consciousness (NCC). Some further believe that constructing a system (e.g., a computer system) that can emulate this NCC interoperation would result in a system that is conscious. Some scholars reject the possibility of non-biological conscious beings. == Philosophical views == As there are many hypothesized types of consciousness, there are many potential implementations of artificial consciousness. In the philosophical literature, perhaps the most common taxonomy of consciousness is into "access" and "phenomenal" variants. Access consciousness concerns those aspects of experience that can be apprehended, while phenomenal consciousness concerns those aspects of experience that seemingly cannot be apprehended, instead being characterized qualitatively in terms of "raw feels", "what it is like" or qualia. === Plausibility debate === Type-identity theorists and other skeptics hold the view that consciousness can be realized only in particular physical systems because consciousness has properties that necessarily depend on physical constitution. In his 2001 article "Artificial Consciousness: Utopia or Real Possibility," Giorgio Buttazzo says that a common objection to artificial consciousness is that, "Working in a fully automated mode, they [the computers] cannot exhibit creativity, unreprogrammation (which means can 'no longer be reprogrammed', from rethinking), emotions, or free will. A computer, like a washing machine, is a slave operated by its components." For other theorists (e.g., functionalists), who define mental states in terms of causal roles, any system that can instantiate the same pattern of causal roles, regardless of physical constitution, will instantiate the same mental states, including consciousness. ==== Thought experiments ==== David Chalmers proposed two thought experiments intending to demonstrate that "functionally isomorphic" systems (those with the same "fine-grained functional organization", i.e., the same information processing) will have qualitatively identical conscious experiences, regardless of whether they are based on biological neurons or digital hardware. The "fading qualia" is a reductio ad absurdum thought experiment. It involves replacing, one by one, the neurons of a brain with a functionally identical component, for example based on a silicon chip. Chalmers makes the hypothesis, knowing it in advance to be absurd, that "the qualia fade or disappear" when neurons are replaced one-by-one with identical silicon equivalents. Since the original neurons and their silicon counterparts are functionally identical, the brain's information processing should remain unchanged, and the subject's behaviour and introspective reports would stay exactly the same. Chalmers argues that this leads to an absurd conclusion: the subject would continue to report normal conscious experiences even as their actual qualia fade away. He concludes that the subject's qualia actually don't fade, and that the resulting robotic brain, once every neuron is replaced, would remain just as sentient as the original biological brain. Similarly, the "dancing qualia" thought experiment is another reductio ad absurdum argument. It supposes that two functionally isomorphic systems could have different perceptions (for instance, seeing the same object in different colors, like red and blue). It involves a switch that alternates between a chunk of brain that causes the perception of red, and a functionally isomorphic silicon chip, that causes the perception of blue. Since both perform the same function within the brain, the subject would not notice any change during the switch. Chalmers argues that this would be highly implausible if the qualia were truly switching between red and blue, hence the contradiction. Therefore, he concludes that the equivalent digital system would not only experience qualia, but it would perceive the same qualia as the biological system (e.g., seeing the same color). Greg Egan's short story Learning To Be Me (mentioned in §In fiction), illustrates how undetectable duplication of the brain and its functionality could be from a first-person perspective. Critics object that Chalmers' proposal begs the question in assuming that all mental properties and external connections are already sufficiently captured by abstract causal organization. Van Heuveln et al. argue that the dancing qualia argument contains an equivocation fallacy, conflating a "change in experience" between two systems with an "experience of change" within a single system. Mogensen argues that the fading qualia argument can be resisted by appealing to vagueness at the boundaries of consciousness and the holistic structure of conscious neural activity, which suggests consciousness may require specific biological substrates rather than being substrate-independent. Anil Seth argues that the complexity of brain neurons intrinsically matters in addition to their function and that it is not possible to replace any part of the brain with a perfect silicon equivalent. He points out that some of biological neurons exhibit activity aimed at cleaning up metabolic waste products, and writes that a perfect silicon replacement would require a silicon-based metabolism, but silicon is not suitable for creating such artificial metabolism. ==== In large language models ==== In 2022, Google engineer Blake Lemoine made a viral claim that Google's LaMDA chatbot was sentient. Lemoine supplied as evidence the chatbot's humanlike answers to many of his questions; however, the chatbot's behavior was judged by the scientific community as likely a consequence of mimicry, rather than machine sentience. Lemoine's claim was widely derided for being ridiculous. Moreover, attributing consciousness based solely on the basis of LLM outputs or the immersive experience created by an algorithm is considered a fallacy. However, while philosopher Nick Bostrom states that LaMDA is unlikely to be conscious, he additionally poses the question of "what grounds would a person have for being sure about it?" One would have to have access to unpublished information about LaMDA's architecture, and also would have to understand how consciousness works, and then figure out how to map the philosophy onto the machine: "(In the absence of these steps), it seems like one should be maybe a little bit uncertain. [...] there could well be other systems now, or in the relatively near future, that would start to satisfy the criteria." David Chalmers argued in 2023 that LLMs today display impressive conversational and general intelligence abilities, but are likely not conscious yet, as they lack some features that may be necessary, such as recurrent processing, a global workspace, and unified agency. Nonetheless, he considers that non-biological systems can be conscious, and suggested that future, extended models (LLM+s) incorporating these elements might eventually meet the criteria for consciousness, raising both profound scientific questions and significant ethical challenges. However, the view that consciousness can exist without biological phenomena is controversial and some reject it. Kristina Šekrst cautions that anthropomorphic terms such as "hallucination" can obscure important ontological differences between artificial and human cognition. While LLMs may produce human-like outputs, she argues that it does not justify ascribing mental states or consciousness to them. Instead, she advocates for an epistemological framework (such as reliabilism) that recognizes the distinct nature of AI knowledge production. She suggests that apparent understanding in LLMs may be a sophisticated form of AI hallucination. She also questions what would happen if an LLM were trained without any mention of consciousness. === Testing === Sentience is an inherently first-person phenomenon. Because of that, and due to the lack of an empirical definition of sentience, directly measuring it may be impossible. Although systems may display numerous behaviors correlated with sentience, determining whether a system is sentient is known as the hard pr

    Read more →
  • Ranking SVM

    Ranking SVM

    In machine learning, a ranking SVM is a variant of the support vector machine algorithm, which is used to solve certain ranking problems (via learning to rank). The ranking SVM algorithm was published by Thorsten Joachims in 2002. The original purpose of the algorithm was to improve the performance of an internet search engine. However, it was found that ranking SVM also can be used to solve other problems such as Rank SIFT. == Description == The ranking SVM algorithm is a learning retrieval function that employs pairwise ranking methods to adaptively sort results based on how 'relevant' they are for a specific query. The ranking SVM function uses a mapping function to describe the match between a search query and the features of each of the possible results. This mapping function projects each data pair (such as a search query and clicked web-page, for example) onto a feature space. These features are combined with the corresponding click-through data (which can act as a proxy for how relevant a page is for a specific query) and can then be used as the training data for the ranking SVM algorithm. Generally, ranking SVM includes three steps in the training period: It maps the similarities between queries and the clicked pages onto a certain feature space. It calculates the distances between any two of the vectors obtained in step 1. It forms an optimization problem which is similar to a standard SVM classification and solves this problem with the regular SVM solver. == Background == === Ranking method === Suppose C {\displaystyle \mathbb {C} } is a data set containing N {\displaystyle N} elements c i {\displaystyle c_{i}} . r {\displaystyle r} is a ranking method applied to C {\displaystyle \mathbb {C} } . Then the r {\displaystyle r} in C {\displaystyle \mathbb {C} } can be represented as a N × N {\displaystyle N\times N} binary matrix. If the rank of c i {\displaystyle c_{i}} is higher than the rank of c j {\displaystyle c_{j}} , i.e. r c i < r c j {\displaystyle r\ c_{i} Read more →

  • Stephen Muggleton

    Stephen Muggleton

    Stephen H. Muggleton (born 6 December 1959, son of Louis Muggleton) is Professor of Machine Learning and Head of the Computational Bioinformatics Laboratory at Imperial College London. == Education == Muggleton received his Bachelor of Science degree in computer science (1982) and Doctor of Philosophy in artificial intelligence (1986) supervised by Donald Michie at the University of Edinburgh. == Career == Following his PhD, Muggleton went on to work as a postdoctoral research associate at the Turing Institute in Glasgow (1987–1991) and later an EPSRC Advanced Research Fellow at Oxford University Computing Laboratory (OUCL) (1992–1997) where he founded the Machine Learning Group. In 1997 he moved to the University of York and in 2001 to Imperial College London. From 2025, Muggleton has joined Nanjing University as a full-time professor. == Research == Muggleton's research interests are primarily in Artificial intelligence. From 1997 to 2001 he held the Chair of Machine Learning at the University of York and from 2001 to 2006 the EPSRC Chair of Computational Bioinformatics at Imperial College in London. Since 2013 he holds the Syngenta/Royal Academy of Engineering Research Chair as well as the post of Director of Modelling for the Imperial College Centre for Integrated Systems Biology. He is known for founding the field of Inductive logic programming. In this field he has made contributions to theory introducing predicate invention, inverse entailment and stochastic logic programs. He has also played a role in systems development where he was instrumental in the systems Duce, Cigol, Golem, Progol and Metagol and applications – especially biological prediction tasks. He worked on a Robot Scientist together with Ross D. King that is capable of combining Inductive Logic Programming with active learning. His present work concentrates on the development of Meta-Interpretive Learning, a new form of Inductive Logic Programming which supports predicate invention and learning of recursive programs.

    Read more →
  • Top 10 AI Subtitle Generators Compared (2026)

    Top 10 AI Subtitle Generators Compared (2026)

    Curious about the best AI subtitle generator? An AI subtitle generator is software that uses machine learning to help you get more done — it combines speed, accuracy, and an interface that just works. Hands-on testing shows real-world results vary, so a short free trial is the smartest way to decide. Whether you are a beginner or a pro, the right AI subtitle generator slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.

    Read more →
  • Zero-knowledge service

    Zero-knowledge service

    In cloud computing, the term zero-knowledge (or occasionally no-knowledge or zero-access) is a commonly used term for online services that store, transfer or manipulate data with a high level of confidentiality, where the data is only accessible to the data's owner (the client), and not to the service provider. However, unlike "end-to-end encryption", the term "zero-knowledge" does not imply any specific threat model or security notion, and its use is commonly frowned-upon by the security community. The term "zero-knowledge" was popularized by backup service SpiderOak, which later switched to using the term "no knowledge", acknowledging that the previous terminology was not technically accurate. == Disadvantages == Most cloud storage services keep a copy of the client's password on their servers, allowing clients who have lost their passwords to retrieve and decrypt their data using alternative means of authentication; but since zero-knowledge services do not store copies of clients' passwords, if a client loses their password then their data cannot be decrypted, making it practically unrecoverable. Most of the most used cloud storage services, such as Google Drive, Dropbox, OneDrive or iCloud, are also able to furnish access requests from law enforcement agencies for similar reasons; zero-knowledge services, however, are unable to do so, since their systems are designed to make clients' data inaccessible without the client's explicit cooperation.

    Read more →
  • Alexei A. Efros

    Alexei A. Efros

    Alexei "Alyosha" A. Efros (born 9 April 1975) is a Russian-American computer scientist and professor at University of California, Berkeley. He has contributed to the field of computer vision, and his work has been referenced in Wired, BBC News, The New York Times, and The New Yorker. == Early life and education == Efros was born in St. Petersburg in the Soviet Union. His father is Alexei L. Efros, then a physics professor at the Ioffe Physico-Technical Institute. His family emigrated to the United States when he was 14 to accommodate his father's career and the family settled in Salt Lake City in 1991. He graduated from the University of Utah in 1997, and attended University of California, Berkeley for his PhD, where he was advised by Jitendra Malik and graduated in 2003. He then spent a year as a research fellow at the University of Oxford, where he worked with Andrew Zisserman. == Career == Efros joined the faculty at Carnegie Mellon University in Pittsburgh, where he remained until 2013 when he joined the faculty of the University of California, Berkeley. He received a Guggenheim Fellowship in 2008. He received the 2016 ACM Prize in Computing.

    Read more →
  • Hanna Hajishirzi

    Hanna Hajishirzi

    Hannaneh Hajishirzi is an Iranian-American computer scientist specializing in natural language processing. She is Torode Family Professor in Computer Science & Engineering in the Paul G. Allen School of Computer Science and Engineering at the University of Washington, head of the H2Lab in the Allen School, and a senior director of natural language processing in the Allen Institute for AI. == Education and career == After a bachelor's degree from the Sharif University of Technology, Hajishirzi completed her Ph.D. in computer science in 2011, at the University of Illinois Urbana-Champaign. Her dissertation, Action-Centered Reasoning for Probabilistic Dynamic Systems, was supervised by Eyal Amir. After postdoctoral research at Disney Research in Pittsburgh, Hajishirzi joined the University of Washington in 2012, as a research scientist in electrical engineering. In 2015 she became a research assistant professor in electrical engineering. She obtained a regular-rank assistant professorship in 2018, at the same time becoming an AI Fellow in the Allen Institute for AI, where she became a senior director of research in 2021. She was promoted to associate professor in 2022 and to full professor in 2025. == Recognition == Hajishirzi was named as a Fellow of the Association for Computational Linguistics in 2025, "for significant contributions to question answering, scientific applications, multimodal artificial intelligence, and fully open language models". == Personal life == Hajishirzi is married to Ali Farhadi, the CEO of the Allen Institute for AI.

    Read more →
  • AI Bug Finders: Free vs Paid (2026)

    AI Bug Finders: Free vs Paid (2026)

    Curious about the best AI bug finder? An AI bug finder is software that uses machine learning to help you get more done — it combines speed, accuracy, and an interface that just works. Hands-on testing shows real-world results vary, so a short free trial is the smartest way to decide. Whether you are a beginner or a pro, the right AI bug finder slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.

    Read more →
  • Artipic

    Artipic

    Artipic is a graphics editor developed for Microsoft Windows. An older version for macOS is still available but unsupported. Artipic features drawing, editing, retouching, transforming and composing images including color corrections, effects and layer-based operations. It converts all common image formats and imports camera raw formats. In the global image editing ecosystem Artipic can be positioned somewhere in the middle. It differs from simple free photo editors by more advanced capabilities, however it does not cover the complete professional-level functionality pack provided by industry leaders like Adobe Photoshop. == History == Artipic developed by Swedish company Artipic AB. Artipic 1.0 was released in March 2014 as a free version. The first commercial version on Microsoft Windows was released in November 2014, on macOS – in October 2015. == Features == Supports Microsoft Windows and macOS Standard tools: select, crop, move, rotate, transform, stamp, color picking, text Advanced tools: custom brushes, gradients, shapes, paths, layers and masks Special tools: healing brush, red-eye effect reduction, dodge and burn brushes Adjustments: Brightness & Contrast, Hue & Saturation, Curves, Levels, Color Balance, Gamma Correction, Exposure, Color Temperature, Tint, Color Enhancer, Photo Filter Simulation, Posterization, Thresholding Filters: Smoothen, Sharpen, Vignetting, High-pass, Diffuse Glow, Shadow, Gaussian Blur Reversible (non-destructive) stylization presets Batch processing White balance RAW-converter including Gray Card Adobe Photoshop images supported == Version history ==

    Read more →
  • How to Choose an AI Analytics Tool

    How to Choose an AI Analytics Tool

    Looking for the best AI analytics tool? An AI analytics tool is software that uses machine learning to help you get more done — it can save you hours every week by automating repetitive work. Most options offer a generous free tier, with paid plans unlocking higher limits, faster processing, and team features. Whether you are a beginner or a pro, the right AI analytics tool slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.

    Read more →
  • Sequential minimal optimization

    Sequential minimal optimization

    Sequential minimal optimization (SMO) is an algorithm for solving the quadratic programming (QP) problem that arises during the training of support-vector machines (SVM). It was invented by John Platt in 1998 at Microsoft Research. SMO is widely used for training support vector machines and is implemented by the popular LIBSVM tool. The publication of the SMO algorithm in 1998 has generated a lot of excitement in the SVM community, as previously available methods for SVM training were much more complex and required expensive third-party QP solvers. == Optimization problem == Consider a binary classification problem with a dataset (x1, y1), ..., (xn, yn), where xi is an input vector and yi ∈ {-1, +1} is a binary label corresponding to it. A soft-margin support vector machine is trained by solving a quadratic programming problem, which is expressed in the dual form as follows: max α ∑ i = 1 n α i − 1 2 ∑ i = 1 n ∑ j = 1 n y i y j K ( x i , x j ) α i α j , {\displaystyle \max _{\alpha }\sum _{i=1}^{n}\alpha _{i}-{\frac {1}{2}}\sum _{i=1}^{n}\sum _{j=1}^{n}y_{i}y_{j}K(x_{i},x_{j})\alpha _{i}\alpha _{j},} subject to: 0 ≤ α i ≤ C , for i = 1 , 2 , … , n , {\displaystyle 0\leq \alpha _{i}\leq C,\quad {\mbox{ for }}i=1,2,\ldots ,n,} ∑ i = 1 n y i α i = 0 {\displaystyle \sum _{i=1}^{n}y_{i}\alpha _{i}=0} where C is an SVM hyperparameter and K(xi, xj) is the kernel function, both supplied by the user; and the variables α i {\displaystyle \alpha _{i}} are Lagrange multipliers. == Algorithm == SMO is an iterative algorithm for solving the optimization problem described above. SMO breaks this problem into a series of smallest possible sub-problems, which are then solved analytically. Because of the linear equality constraint involving the Lagrange multipliers α i {\displaystyle \alpha _{i}} , the smallest possible problem involves two such multipliers. Then, for any two multipliers α 1 {\displaystyle \alpha _{1}} and α 2 {\displaystyle \alpha _{2}} , the constraints are reduced to: 0 ≤ α 1 , α 2 ≤ C , {\displaystyle 0\leq \alpha _{1},\alpha _{2}\leq C,} y 1 α 1 + y 2 α 2 = k , {\displaystyle y_{1}\alpha _{1}+y_{2}\alpha _{2}=k,} and this reduced problem can be solved analytically: one needs to find a minimum of a one-dimensional quadratic function. k {\displaystyle k} is the negative of the sum over the rest of terms in the equality constraint, which is fixed in each iteration. The algorithm proceeds as follows: Find a Lagrange multiplier α 1 {\displaystyle \alpha _{1}} that violates the Karush–Kuhn–Tucker (KKT) conditions for the optimization problem. Pick a second multiplier α 2 {\displaystyle \alpha _{2}} and optimize the pair ( α 1 , α 2 ) {\displaystyle (\alpha _{1},\alpha _{2})} . Repeat steps 1 and 2 until convergence. When all the Lagrange multipliers satisfy the KKT conditions (within a user-defined tolerance), the problem has been solved. Although this algorithm is guaranteed to converge, heuristics are used to choose the pair of multipliers so as to accelerate the rate of convergence. This is critical for large data sets since there are n ( n − 1 ) / 2 {\displaystyle n(n-1)/2} possible choices for α i {\displaystyle \alpha _{i}} and α j {\displaystyle \alpha _{j}} . == Related work == The first approach to splitting large SVM learning problems into a series of smaller optimization tasks was proposed by Bernhard Boser, Isabelle Guyon, and Vladimir Vapnik. It is known as the "chunking algorithm". The algorithm starts with a random subset of the data, solves this problem, and iteratively adds examples which violate the optimality conditions. One disadvantage of this algorithm is that it is necessary to solve QP-problems scaling with the number of SVs. On real world sparse data sets, SMO can be more than 1000 times faster than the chunking algorithm. In 1997, E. Osuna, R. Freund, and F. Girosi proved a theorem which suggests a whole new set of QP algorithms for SVMs. By the virtue of this theorem a large QP problem can be broken down into a series of smaller QP sub-problems. A sequence of QP sub-problems that always add at least one violator of the Karush–Kuhn–Tucker (KKT) conditions is guaranteed to converge. The chunking algorithm obeys the conditions of the theorem, and hence will converge. The SMO algorithm can be considered a special case of the Osuna algorithm, where the size of the optimization is two and both Lagrange multipliers are replaced at every step with new multipliers that are chosen via good heuristics. The SMO algorithm is closely related to a family of optimization algorithms called Bregman methods or row-action methods. These methods solve convex programming problems with linear constraints. They are iterative methods where each step projects the current primal point onto each constraint.

    Read more →