AI For Students Articles

AI For Students Articles — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

Brownout (software engineering)

Brownout in software engineering is a technique that involves disabling certain features of an application. == Description == Brownout is used to increase the robustness of an application to computing capacity shortage. If too many users are simultaneously accessing an application hosted online, the underlying computing infrastructure may become overloaded, rendering the application unresponsive. Users are likely to abandon the application and switch to competing alternatives, hence incurring long-term revenue loss. To better deal with such a situation, the application can be given brownout capabilities: The application will disable certain features – e.g., an online shop will no longer display recommendations of related products – to avoid overload. Although reducing features generally has a negative impact on the short-term revenue of the application owner, long-term revenue loss can be avoided. The technique is inspired by brownouts in power grids, which consists in reducing the power grid's voltage in case electricity demand exceeds production. Some consumers, such as incandescent light bulbs, will dim – hence originating the term – and draw less power, thus helping match demand with production. Similarly, a brownout application helps match its computing capacity requirements to what is available on the target infrastructure. Brownout complements elasticity. The former can help the application withstand short-term capacity shortage, but does so without changing the capacity available to the application. In contrast, elasticity consists of adding (or removing) capacity to the application, preferably in advance, so as to avoid capacity shortage altogether. The two techniques can be combined; e.g., brownout is triggered when the number of users increases unexpectedly until elasticity can be triggered, the latter usually requiring minutes to show an effect. Brownout is relatively non-intrusive for the developer, for example, it can be implemented as an advice in aspect-oriented programming. However, surrounding components, such as load-balancers, need to be made brownout-aware to distinguish between cases where an application is running normally and cases where the application maintains a low response time by triggering brownout. == Usage in phased deprecation == A related use of the brownout concept in software engineering is the deliberate introduction of temporary outages to a system, API or feature that is being phased out. This is sometimes also called a "scream test" when it is used to discover unknown dependents of a system or API. The intention is to allow detection of downstream consumers of an API or service who may otherwise have missed deprecation announcements or to uncover hidden side-effects of the deprecation that may have been overlooked. The intention is that developers of dependent systems will notice their own system failures caused by the upstream brownout. Such brownouts are typically pre-announced scheduled outages or probabilistic in nature (such as artificially failing a percentage of requests). As a brownout is only a temporary or partial outage, it provides downstream consumers of an API or service time to remove any discovered dependencies on the deprecated API before it is fully retired. For consumers that have already prepared for the deprecation, a brownout provides valuable testing that the final removal of the service won't cause any unexpected problems.
Read more →
Larry Heck

Larry Paul Heck is the Rhesa Screven Farmer, Jr., Advanced Computing Concepts Chair, Georgia Research Alliance Eminent Scholar, Co-Executive Director of the Machine Learning Center and Professor at the Georgia Institute of Technology. His career spans many of the sub-disciplines of artificial intelligence, including conversational AI, speech recognition and speaker recognition, natural language processing, web search, online advertising and acoustics. He is best known for his role as a co-founder of the Microsoft Cortana Personal Assistant and his early work in deep learning for speech processing. == Education and career == Larry Heck was born in Havre, Montana. After receiving the Bachelor of Science in electrical engineering at Texas Tech University, he was admitted to graduate school at the Georgia Institute of Technology in 1986. Heck received the MSEE in 1989 and the PhD in 1991 under advisor Prof. James H. McClellan. From 1992 to 1998, he was a senior research engineer at SRI International with the Acoustics and Radar Technology Lab (ARTL) and Speech Technology and Research (STAR) Lab, and in 1998 joined Nuance Communications, serving as vice president of R&D. Funded by the US government's NSA and DARPA from 1995-1998, Heck led the SRI team that was the first to successfully create large-scale deep neural network (DNN) deep learning technology in the field of speech processing. The deep learning technology was used to win the 1998 National Institute of Standards and Technology Speaker Recognition evaluation. The approach trained a 5-layer deep neural network, with the first two layers used as a (learned) feature extractor. To stabilize the training of the DNN, a weight normalization method was used (later rediscovered in 2010 by Xavier, et.al). Heck deployed this DNN in 1999 with Nuance Communications at the Home Shopping Network, representing the first major industrial application of deep learning with over 100K Nuance Verifier voiceprints. From 2005 to 2008, he was vice president of search & advertising quality at Yahoo!. In 2008, Heck and Ron Brachman combined search & advertising quality with Yahoo! Research to form Yahoo! Labs. Beginning in 2009, he was the chief scientist of speech products at Microsoft. In this role, he established the vision, mission and long-range plan and hired the initial team to create Microsoft’s digital-personal-assistant Cortana. Heck was named a Microsoft Distinguished Engineer in 2012 and joined Microsoft Research that same year. In 2014, he joined Google as a principal research scientist, where he founded the deep learning-based conversational AI team "Deep Dialogue". The team works on advanced research for the Google Assistant. In 2017, Heck joined Samsung as SVP and co-head of global AI Research. In 2019, he became head of Bixby (virtual assistant) North America and the CEO of Viv Labs, an independent subsidiary of Samsung. In that same year, Heck led one of the first large scale deployments of Transformer-Based LLMs as part of the Bixby Categories launch at the 2019 Samsung Developer Conference. In 2021, Heck returned to the Georgia Institute of Technology as a Professor. == Awards and honors == Larry Heck was named Fellow of the Institute of Electrical and Electronics Engineers (IEEE) in 2016 for leadership in application of machine learning to spoken and text language processing. Heck was inducted as a Fellow of the National Academy of Inventors (NAI) in 2024. Heck received the 2017 Academy of Distinguished Engineering Alumni Award from the Georgia Institute of Technology. In the same year, he also received the Texas Tech University Whitacre College of Engineering Distinguished Engineer Award. Larry Heck has several best papers including the 2020 IEEE Signal Processing Society (SPS) Best Paper Award: “Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding” published in the IEEE/ACM Transactions on Audio, Speech, and Language Processing in March 2015, and the 2020 ACM Conference on Information and Knowledge Management (CIKM) Test of Time Award for the paper "Learning Deep Structured Semantic Models for Web Search using Clickthrough Data".
Read more →
Two-way finite automaton

In computer science, in particular in automata theory, a two-way finite automaton is a finite automaton that is allowed to re-read its input. == Two-way deterministic finite automaton == A two-way deterministic finite automaton (2DFA) is an abstract machine, a generalized version of the deterministic finite automaton (DFA) which can revisit characters already processed. As in a DFA, there are a finite number of states with transitions between them based on the current character, but each transition is also labelled with a value indicating whether the machine will move its position in the input to the left, right, or stay at the same position. Equivalently, 2DFAs can be seen as read-only Turing machines with no work tape, only a read-only input tape. 2DFAs were introduced in a seminal 1959 paper by Rabin and Scott, who proved them to have equivalent power to one-way DFAs. That is, any formal language which can be recognized by a 2DFA can be recognized by a DFA which only examines and consumes each character in order. Since DFAs are obviously a special case of 2DFAs, this implies that both kinds of machines recognize precisely the class of regular languages. However, the equivalent DFA for a 2DFA may require exponentially many states, making 2DFAs a much more practical representation for algorithms for some common problems. 2DFAs are also equivalent to read-only Turing machines that use only a constant amount of space on their work tape, since any constant amount of information can be incorporated into the finite control state via a product construction (a state for each combination of work tape state and control state). == Formal description == Formally, a two-way deterministic finite automaton can be described by the following 8-tuple: M = ( Q , Σ , L , R , δ , s , t , r ) {\displaystyle M=(Q,\Sigma ,L,R,\delta ,s,t,r)} where Q {\displaystyle Q} is the finite, non-empty set of states Σ {\displaystyle \Sigma } is the finite, non-empty set of input symbols L {\displaystyle L} is the left endmarker R {\displaystyle R} is the right endmarker δ : Q × ( Σ ∪ { L , R } ) → Q × { l e f t , r i g h t } {\displaystyle \delta :Q\times (\Sigma \cup \{L,R\})\rightarrow Q\times \{\mathrm {left,right} \}} s {\displaystyle s} is the start state t {\displaystyle t} is the end state r {\displaystyle r} is the reject state In addition, the following two conditions must also be satisfied: For all q ∈ Q {\displaystyle q\in Q} δ ( q , L ) = ( q ′ , r i g h t ) {\displaystyle \delta (q,L)=(q^{\prime },\mathrm {right} )} for some q ′ ∈ Q {\displaystyle q^{\prime }\in Q} δ ( q , R ) = ( q ′ , l e f t ) {\displaystyle \delta (q,R)=(q^{\prime },\mathrm {left} )} for some q ′ ∈ Q {\displaystyle q^{\prime }\in Q} It says that there must be some transition possible when the pointer reaches either end of the input word. For all symbols σ ∈ Σ ∪ { L } {\displaystyle \sigma \in \Sigma \cup \{L\}} δ ( t , σ ) = ( t , R ) {\displaystyle \delta (t,\sigma )=(t,R)} δ ( r , σ ) = ( r , R ) {\displaystyle \delta (r,\sigma )=(r,R)} δ ( t , R ) = ( t , L ) {\displaystyle \delta (t,R)=(t,L)} δ ( r , R ) = ( r , L ) {\displaystyle \delta (r,R)=(r,L)} It says that once the automaton reaches the accept or reject state, it stays in there forever and the pointer goes to the right most symbol and cycles there infinitely. == Two-way nondeterministic finite automaton == A two-way nondeterministic finite automaton (2NFA) may have multiple transitions defined in the same configuration. Its transition function is δ : Q × ( Σ ∪ { L , R } ) → 2 Q × { l e f t , r i g h t } {\displaystyle \delta :Q\times (\Sigma \cup \{L,R\})\rightarrow 2^{Q\times \{\mathrm {left,right} \}}} . Like a standard one-way NFA, a 2NFA accepts a string if at least one of the possible computations is accepting. Like the 2DFAs, the 2NFAs also accept only regular languages. == Two-way alternating finite automaton == A two-way alternating finite automaton (2AFA) is a two-way extension of an alternating finite automaton (AFA). Its state set is Q = Q ∃ ∪ Q ∀ {\displaystyle Q=Q_{\exists }\cup Q_{\forall }} where Q ∃ ∩ Q ∀ = ∅ {\displaystyle Q_{\exists }\cap Q_{\forall }=\emptyset } . States in Q ∃ {\displaystyle Q_{\exists }} and Q ∀ {\displaystyle Q_{\forall }} are called existential resp. universal. In an existential state a 2AFA nondeterministically chooses the next state like an NFA, and accepts if at least one of the resulting computations accepts. In a universal state 2AFA moves to all next states, and accepts if all the resulting computations accept. == State complexity tradeoffs == Two-way and one-way finite automata, deterministic and nondeterministic and alternating, accept the same class of regular languages. However, transforming an automaton of one type to an equivalent automaton of another type incurs a blow-up in the number of states. Christos Kapoutsis determined that transforming an n {\displaystyle n} -state 2DFA to an equivalent DFA requires n ( n n − ( n − 1 ) n ) {\displaystyle n(n^{n}-(n-1)^{n})} states in the worst case. If an n {\displaystyle n} -state 2DFA or a 2NFA is transformed to an NFA, the worst-case number of states required is ( 2 n n + 1 ) = O ( 4 n n ) {\displaystyle {\binom {2n}{n+1}}=O\left({\frac {4^{n}}{\sqrt {n}}}\right)} . Ladner, Lipton and Stockmeyer. proved that an n {\displaystyle n} -state 2AFA can be converted to a DFA with 2 n 2 n {\displaystyle 2^{n2^{n}}} states. The 2AFA to NFA conversion requires 2 Θ ( n log ⁡ n ) {\displaystyle 2^{\Theta (n\log n)}} states in the worst case, see Geffert and Okhotin. It is an open problem whether every 2NFA can be converted to a 2DFA with only a polynomial increase in the number of states. The problem was raised by Sakoda and Sipser, who compared it to the P vs. NP problem in the computational complexity theory. Berman and Lingas discovered a formal relation between this problem and the L vs. NL open problem, see Kapoutsis for a precise relation. == Sweeping automata == Sweeping automata are 2DFAs of a special kind that process the input string by making alternating left-to-right and right-to-left sweeps, turning only at the endmarkers. Sipser constructed a sequence of languages, each accepted by an n-state NFA, yet which is not accepted by any sweeping automata with fewer than 2 n {\displaystyle 2^{n}} states. == Two-way quantum finite automaton == The concept of 2DFAs was in 1997 generalized to quantum computing by John Watrous's "On the Power of 2-Way Quantum Finite State Automata", in which he demonstrates that these machines can recognize nonregular languages and so are more powerful than DFAs. == Two-way pushdown automaton == A pushdown automaton that is allowed to move either way on its input tape is called two-way pushdown automaton (2PDA); it has been studied by Hartmanis, Lewis, and Stearns (1965). Aho, Hopcroft, Ullman (1968) and Cook (1971) characterized the class of languages recognizable by deterministic (2DPDA) and non-deterministic (2NPDA) two-way pushdown automata; Gray, Harrison, and Ibarra (1967) investigated the closure properties of these languages.
Read more →
Structured support vector machine

The structured supportvector machine is a machine learning algorithm that generalizes the support vector machine (SVM) classifier. Whereas the SVM classifier supports binary classification, multiclass classification and regression, the structured SVM allows training of a classifier for general structured output labels. As an example, a sample instance might be a natural language sentence, and the output label is an annotated parse tree. Training a classifier consists of showing pairs of correct sample and output label pairs. After training, the structured SVM model allows one to predict for new sample instances the corresponding output label; that is, given a natural language sentence, the classifier can produce the most likely parse tree. == Training == For a set of n {\displaystyle n} training instances ( x i , y i ) ∈ X × Y {\displaystyle ({\boldsymbol {x}}_{i},y_{i})\in {\mathcal {X}}\times {\mathcal {Y}}} , i = 1 , … , n {\displaystyle i=1,\dots ,n} from a sample space X {\displaystyle {\mathcal {X}}} and label space Y {\displaystyle {\mathcal {Y}}} , the structured SVM minimizes the following regularized risk function. min w ‖ w ‖ 2 + C ∑ i = 1 n max y ∈ Y ( 0 , Δ ( y i , y ) + ⟨ w , Ψ ( x i , y ) ⟩ − ⟨ w , Ψ ( x i , y i ) ⟩ ) {\displaystyle {\underset {\boldsymbol {w}}{\min }}\quad \|{\boldsymbol {w}}\|^{2}+C\sum _{i=1}^{n}{\underset {y\in {\mathcal {Y}}}{\max }}\left(0,\Delta (y_{i},y)+\langle {\boldsymbol {w}},\Psi ({\boldsymbol {x}}_{i},y)\rangle -\langle {\boldsymbol {w}},\Psi ({\boldsymbol {x}}_{i},y_{i})\rangle \right)} The function is convex in w {\displaystyle {\boldsymbol {w}}} because the maximum of a set of affine functions is convex. The function Δ : Y × Y → R + {\displaystyle \Delta :{\mathcal {Y}}\times {\mathcal {Y}}\to \mathbb {R} _{+}} measures a distance in label space and is an arbitrary function (not necessarily a metric) satisfying Δ ( y , z ) ≥ 0 {\displaystyle \Delta (y,z)\geq 0} and Δ ( y , y ) = 0 ∀ y , z ∈ Y {\displaystyle \Delta (y,y)=0\;\;\forall y,z\in {\mathcal {Y}}} . The function Ψ : X × Y → R d {\displaystyle \Psi :{\mathcal {X}}\times {\mathcal {Y}}\to \mathbb {R} ^{d}} is a feature function, extracting some feature vector from a given sample and label. The design of this function depends very much on the application. Because the regularized risk function above is non-differentiable, it is often reformulated in terms of a quadratic program by introducing one slack variable ξ i {\displaystyle \xi _{i}} for each sample, each representing the value of the maximum. The standard structured SVM primal formulation is given as follows. min w , ξ ‖ w ‖ 2 + C ∑ i = 1 n ξ i s.t. ⟨ w , Ψ ( x i , y i ) ⟩ − ⟨ w , Ψ ( x i , y ) ⟩ + ξ i ≥ Δ ( y i , y ) , i = 1 , … , n , ∀ y ∈ Y {\displaystyle {\begin{array}{cl}{\underset {{\boldsymbol {w}},{\boldsymbol {\xi }}}{\min }}&\|{\boldsymbol {w}}\|^{2}+C\sum _{i=1}^{n}\xi _{i}\\{\textrm {s.t.}}&\langle {\boldsymbol {w}},\Psi ({\boldsymbol {x}}_{i},y_{i})\rangle -\langle {\boldsymbol {w}},\Psi ({\boldsymbol {x}}_{i},y)\rangle +\xi _{i}\geq \Delta (y_{i},y),\qquad i=1,\dots ,n,\quad \forall y\in {\mathcal {Y}}\end{array}}} == Inference == At test time, only a sample x ∈ X {\displaystyle {\boldsymbol {x}}\in {\mathcal {X}}} is known, and a prediction function f : X → Y {\displaystyle f:{\mathcal {X}}\to {\mathcal {Y}}} maps it to a predicted label from the label space Y {\displaystyle {\mathcal {Y}}} . For structured SVMs, given the vector w {\displaystyle {\boldsymbol {w}}} obtained from training, the prediction function is the following. f ( x ) = argmax y ∈ Y ⟨ w , Ψ ( x , y ) ⟩ {\displaystyle f({\boldsymbol {x}})={\underset {y\in {\mathcal {Y}}}{\textrm {argmax}}}\quad \langle {\boldsymbol {w}},\Psi ({\boldsymbol {x}},y)\rangle } Therefore, the maximizer over the label space is the predicted label. Solving for this maximizer is the so-called inference problem and similar to making a maximum a-posteriori (MAP) prediction in probabilistic models. Depending on the structure of the function Ψ {\displaystyle \Psi } , solving for the maximizer can be a hard problem. == Separation == The above quadratic program involves a very large, possibly infinite number of linear inequality constraints. In general, the number of inequalities is too large to be optimized over explicitly. Instead the problem is solved by using delayed constraint generation where only a finite and small subset of the constraints is used. Optimizing over a subset of the constraints enlarges the feasible set and will yield a solution that provides a lower bound on the objective. To test whether the solution w {\displaystyle {\boldsymbol {w}}} violates constraints of the complete set inequalities, a separation problem needs to be solved. As the inequalities decompose over the samples, for each sample ( x i , y i ) {\displaystyle ({\boldsymbol {x}}_{i},y_{i})} the following problem needs to be solved. y n ∗ = argmax y ∈ Y ( Δ ( y i , y ) + ⟨ w , Ψ ( x i , y ) ⟩ − ⟨ w , Ψ ( x i , y i ) ⟩ − ξ i ) {\displaystyle y_{n}^{}={\underset {y\in {\mathcal {Y}}}{\textrm {argmax}}}\left(\Delta (y_{i},y)+\langle {\boldsymbol {w}},\Psi ({\boldsymbol {x}}_{i},y)\rangle -\langle {\boldsymbol {w}},\Psi ({\boldsymbol {x}}_{i},y_{i})\rangle -\xi _{i}\right)} The right hand side objective to be maximized is composed of the constant − ⟨ w , Ψ ( x i , y i ) ⟩ − ξ i {\displaystyle -\langle {\boldsymbol {w}},\Psi ({\boldsymbol {x}}_{i},y_{i})\rangle -\xi _{i}} and a term dependent on the variables optimized over, namely Δ ( y i , y ) + ⟨ w , Ψ ( x i , y ) ⟩ {\displaystyle \Delta (y_{i},y)+\langle {\boldsymbol {w}},\Psi ({\boldsymbol {x}}_{i},y)\rangle } . If the achieved right hand side objective is smaller or equal to zero, no violated constraints for this sample exist. If it is strictly larger than zero, the most violated constraint with respect to this sample has been identified. The problem is enlarged by this constraint and resolved. The process continues until no violated inequalities can be identified. If the constants are dropped from the above problem, we obtain the following problem to be solved. y i ∗ = argmax y ∈ Y ( Δ ( y i , y ) + ⟨ w , Ψ ( x i , y ) ⟩ ) {\displaystyle y_{i}^{}={\underset {y\in {\mathcal {Y}}}{\textrm {argmax}}}\left(\Delta (y_{i},y)+\langle {\boldsymbol {w}},\Psi ({\boldsymbol {x}}_{i},y)\rangle \right)} This problem looks very similar to the inference problem. The only difference is the addition of the term Δ ( y i , y ) {\displaystyle \Delta (y_{i},y)} . Most often, it is chosen such that it has a natural decomposition in label space. In that case, the influence of Δ {\displaystyle \Delta } can be encoded into the inference problem and solving for the most violating constraint is equivalent to solving the inference problem.
Read more →
Knowledge integration

Knowledge integration is the process of synthesizing multiple knowledge models (or representations) into a common model (representation). Compared to information integration, which involves merging information having different schemas and representation models, knowledge integration focuses more on synthesizing the understanding of a given subject from different perspectives. For example, multiple interpretations are possible of a set of student grades, typically each from a certain perspective. An overall, integrated view and understanding of this information can be achieved if these interpretations can be put under a common model, say, a student performance index. The Web-based Inquiry Science Environment (WISE), from the University of California at Berkeley has been developed along the lines of knowledge integration theory. Knowledge integration has also been studied as the process of incorporating new information into a body of existing knowledge with an interdisciplinary approach. This process involves determining how the new information and the existing knowledge interact, how existing knowledge should be modified to accommodate the new information, and how the new information should be modified in light of the existing knowledge. A learning agent that actively investigates the consequences of new information can detect and exploit a variety of learning opportunities; e.g., to resolve knowledge conflicts and to fill knowledge gaps. By exploiting these learning opportunities the learning agent is able to learn beyond the explicit content of the new information. The machine learning program KI, developed by Murray and Porter at the University of Texas at Austin, was created to study the use of automated and semi-automated knowledge integration to assist knowledge engineers constructing a large knowledge base. A possible technique which can be used is semantic matching. More recently, a technique useful to minimize the effort in mapping validation and visualization has been presented which is based on Minimal Mappings. Minimal mappings are high quality mappings such that i) all the other mappings can be computed from them in time linear in the size of the input graphs, and ii) none of them can be dropped without losing property i). The University of Waterloo operates a Bachelor of Knowledge Integration undergraduate degree program as an academic major or minor. The program started in 2008.
Read more →
Kalman filter

In statistics and control theory, Kalman filtering (also known as linear quadratic estimation) is an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, to produce estimates of unknown variables that tend to be more accurate than those based on a single measurement, by estimating a joint probability distribution over the variables for each time-step. The filter is constructed as a mean squared error minimiser, but an alternative derivation of the filter is also provided showing how the filter relates to maximum likelihood statistics. The filter is named after Rudolf E. Kálmán. Kalman filtering has numerous technological applications. A common application is for guidance, navigation, and control of vehicles, particularly aircraft, spacecraft and ships positioned dynamically. Furthermore, Kalman filtering is much applied in time series analysis tasks such as signal processing and econometrics. Kalman filtering is also important for robotic motion planning and control, and can be used for trajectory optimization. Kalman filtering also works for modeling the central nervous system's control of movement. Due to the time delay between issuing motor commands and receiving sensory feedback, the use of Kalman filters provides a realistic model for making estimates of the current state of a motor system and issuing updated commands. The algorithm works via a two-phase process: a prediction phase and an update phase. In the prediction phase, the Kalman filter produces estimates of the current state variables, including their uncertainties. Once the outcome of the next measurement (necessarily corrupted with some error, including random noise) is observed, these estimates are updated using a weighted average, with more weight given to estimates with greater certainty. The algorithm is recursive. It can operate in real time, using only the present input measurements and the state calculated previously and its uncertainty matrix; no additional past information is required. Optimality of Kalman filtering assumes that errors have a normal (Gaussian) distribution. In the words of Rudolf E. Kálmán, "The following assumptions are made about random processes: Physical random phenomena may be thought of as due to primary random sources exciting dynamic systems. The primary sources are assumed to be independent gaussian random processes with zero mean; the dynamic systems will be linear." Regardless of Gaussianity, however, if the process and measurement covariances are known, then the Kalman filter is the best possible linear estimator in the minimum mean-square-error sense, although there may be better nonlinear estimators. It is a common misconception (perpetuated in the literature) that the Kalman filter cannot be rigorously applied unless all noise processes are assumed to be Gaussian. Extensions and generalizations of the method have also been developed, such as the extended Kalman filter and the unscented Kalman filter which work on nonlinear systems. The basis is a hidden Markov model such that the state space of the latent variables is continuous and all latent and observed variables have Gaussian distributions. Kalman filtering has been used successfully in multi-sensor fusion, and distributed sensor networks to develop distributed or consensus Kalman filtering. == History == The filtering method is named for Hungarian émigré Rudolf E. Kálmán, although Thorvald Nicolai Thiele and Peter Swerling developed a similar algorithm earlier. Richard S. Bucy of the Johns Hopkins Applied Physics Laboratory contributed to the theory, causing it to be known sometimes as Kalman–Bucy filtering. Kalman was inspired to derive the Kalman filter by applying state variables to the Wiener filtering problem. Stanley F. Schmidt is generally credited with developing the first implementation of a Kalman filter. He realized that the filter could be divided into two distinct parts, with one part for time periods between sensor outputs and another part for incorporating measurements. It was during a visit by Kálmán to the NASA Ames Research Center that Schmidt saw the applicability of Kálmán's ideas to the nonlinear problem of trajectory estimation for the Apollo program resulting in its incorporation in the Apollo navigation computer. This digital filter is sometimes termed the Stratonovich–Kalman–Bucy filter because it is a special case of a more general, nonlinear filter developed by the Soviet mathematician Ruslan Stratonovich. In fact, some of the special case linear filter's equations appeared in papers by Stratonovich that were published before the summer of 1961, when Kalman met with Stratonovich during a conference in Moscow. This Kalman filtering was first described and developed partially in technical papers by Swerling (1958), Kalman (1960) and Kalman and Bucy (1961). The Apollo computer used 2k of magnetic core RAM and 36k wire rope [...]. The CPU was built from ICs [...]. Clock speed was under 100 kHz [...]. The fact that the MIT engineers were able to pack such good software (one of the very first applications of the Kalman filter) into such a tiny computer is truly remarkable. Kalman filters have been vital in the implementation of the navigation systems of U.S. Navy nuclear ballistic missile submarines, and in the guidance and navigation systems of cruise missiles such as the U.S. Navy's Tomahawk missile and the U.S. Air Force's Air Launched Cruise Missile. They are also used in the guidance and navigation systems of reusable launch vehicles and the attitude control and navigation systems of spacecraft which dock at the International Space Station. == Overview of the calculation == Kalman filtering uses a system's dynamic model (e.g., physical laws of motion), known control inputs to that system, and multiple sequential measurements (such as from sensors) to form an estimate of the system's varying quantities (its state) that is better than the estimate obtained by using only one measurement alone. As such, it is a common sensor fusion and data fusion algorithm. Noisy sensor data, approximations in the equations that describe the system evolution, and external factors that are not accounted for, all limit how well it is possible to determine the system's state. The Kalman filter deals effectively with the uncertainty due to noisy sensor data and, to some extent, with random external factors. The Kalman filter produces an estimate of the state of the system as an average of the system's predicted state and of the new measurement using a weighted average. The purpose of the weights is that values with better (i.e., smaller) estimated uncertainty are "trusted" more. The weights are calculated from the covariance, a measure of the estimated uncertainty of the prediction of the system's state. The result of the weighted average is a new state estimate that lies between the predicted and measured state, and has a better estimated uncertainty than either alone. This process is repeated at every time step, with the new estimate and its covariance informing the prediction used in the following iteration. This means that Kalman filter works recursively and requires only the last "best guess", rather than the entire history, of a system's state to calculate a new state. The measurements' certainty-grading and current-state estimate are important considerations. It is common to discuss the filter's response in terms of the Kalman filter's gain. The Kalman gain is the weight given to the measurements and current-state estimate, and can be "tuned" to achieve a particular performance. With a high gain, the filter places more weight on the most recent measurements, and thus conforms to them more responsively. With a low gain, the filter conforms to the model predictions more closely. At the extremes, a high gain (close to one) will result in a more jumpy estimated trajectory, while a low gain (close to zero) will smooth out noise but decrease the responsiveness. When performing the actual calculations for the filter (as discussed below), the state estimate and covariances are coded into matrices because of the multiple dimensions involved in a single set of calculations. This allows for a representation of linear relationships between different state variables (such as position, velocity, and acceleration) in any of the transition models or covariances. == Example application == As an example application, consider the problem of determining the precise location of a truck. The truck can be equipped with a GPS unit that provides an estimate of the position within a few meters. The GPS estimate is likely to be noisy; readings 'jump around' rapidly, though remaining within a few meters of the real position. In addition, since the truck is expected to follow the laws of physics, its position can also be estimated by integrating its velocity over time, determined by keeping track of wheel revolutions and the
Read more →
The Best Free AI Image Generator for Beginners

In search of the best AI image generator? An AI image generator is software that uses machine learning to help you get more done — it turns a rough idea into a polished result in seconds. When choosing one, weigh output quality, pricing, export formats, and how well it fits the tools you already use. Whether you are a beginner or a pro, the right AI image generator slots into your workflow and pays for itself fast. Below we compare features, pricing, and real output so you can choose with confidence.
Read more →
AI Pair Programmers: Free vs Paid (2026)

Trying to pick the best AI pair programmer? An AI pair programmer is software that uses machine learning to help you get more done — it scales effortlessly from a single task to thousands. The best picks balance beginner-friendly simplicity with the depth power users need, and they ship updates often. Whether you are a beginner or a pro, the right AI pair programmer slots into your workflow and pays for itself fast. This guide breaks down the top picks, their pros and cons, and who each one is best for.
Read more →
Colloquis

Colloquis, previously known as ActiveBuddy and Conversagent, was a company that created conversation-based interactive agents originally distributed via instant messaging platforms. The company had offices in New York, New York, and Sunnyvale, California. == History == Founded in 2000, the company was the brainchild of Robert Hoffer, Timothy Kay, and Peter Levitan. The idea for interactive agents (also known as Internet bots) came from the team's vision to add functionality to increasingly popular instant messaging services. The original implementation took shape as a word-based adventure game but quickly grew to include a wide range of database applications, including access to news, weather, stock information, movie times, Yellow Pages listings, and detailed sports data, as well as a variety of tools (calculators, translator, etc.). These various applications were bundled into one entity and launched as SmarterChild in 2001. SmarterChild acted as a showcase for the quick data access and possibilities for fun conversation that the company planned to turn into customized, niche-specific products. The rapid success of SmarterChild led to targeted promotional products for Radiohead, Austin Powers, The Sporting News, and others. ActiveBuddy sought to strengthen its hold on the interactive agent market for the future by filing for, and receiving, a controversial patent on their creation in 2002. The company also released the BuddyScript SDK, a free developer kit that allow programmers to design and launch their own interactive agents using ActiveBuddy's proprietary scripting language, in 2002. Ultimately, however, the decline in ad spending in 2001 and 2002 led to a shift in corporate strategy towards business focused Automated Service Agents, building products for clients including Cingular, Comcast and Cox Communications. The company subsequently changed its name from ActiveBuddy to Conversagent in 2003, and then again to Colloquis in 2006. Colloquis was purchased by Microsoft in October 2006.
Read more →
Aapo Hyvärinen

Aapo Johannes Hyvärinen (born 1970 in Helsinki) is a Finnish professor of computer science at the University of Helsinki and known for his research in independent component analysis. == Education and career == Hyvärinen was born in Helsinki and studied mathematics at the University of Helsinki and received his Doctor of Technology in information science in 1997 at the Helsinki University of Technology under the supervision of Erkki Oja. His doctoral thesis, titled "Independent component analysis: A neural network approach", introduced the FastICA algorithm. Since then, Hyvärinen has conducted research especially in relation to the independent component analysis, as well as score matching (also known as Hyvärinen scoring rule). In November 2007, he was appointed as a professor at the University of Helsinki. Hyvärinen has been a member of the Finnish Academy of Sciences since 2016. From August 2016 to March 2019, he held a professorship in machine learning at the Gatsby Computational Neuroscience Unit of the University College London.
Read more →
The Best Free AI Art Generator for Beginners

Trying to pick the best AI art generator? An AI art generator is software that uses machine learning to help you get more done — it scales effortlessly from a single task to thousands. The best picks balance beginner-friendly simplicity with the depth power users need, and they ship updates often. Whether you are a beginner or a pro, the right AI art generator slots into your workflow and pays for itself fast. This guide breaks down the top picks, their pros and cons, and who each one is best for.
Read more →
Brainware

Brainware was an American software company that marketed Automatic identification and data capture and data extraction products. The company was acquired by Hyland Software in 2017. Brainware originally spun out of Dulles, Virginia-based SER Solutions Inc. in February 2006 when SER was acquired by The Gores Group LLC. From February 2006 to March 2012, Brainware's majority owner was San Francisco-based private equity firm Vista Equity Partners. == History == On March 5, 2012, Lexmark International announced it had acquired the company for a cash price of approximately $148 million. The company was added to Lexmark's Perceptive Software division. On July 10, 2017, Hyland Software finalized its acquisition of the Perceptive Business Unit of Lexmark International, Inc. All enterprise software business assets in the Perceptive business unit, including Perceptive Content (formerly ImageNow), Perceptive Intelligent Capture (formerly Brainware), Acuo VNA, PACSGEAR, Claron, Nolij, Saperion, Pallas Athena, ISYS and Twistage, now operate under Hyland's portfolio of products. Brainware was headquartered in Ashburn, Virginia, USA, with sales, support, professional services and R&D offices in London, UK; Kirchzarten, Germany; and Neuchâtel, Switzerland. The company had partnerships with most major enterprise software providers, including Oracle, SAP and Microsoft, and said its software integrated with most available enterprise content management platforms. Brainware also partnered with a number of hardware providers, including Hewlett-Packard, Fujitsu and OPEX. Brainware's core solution, Distiller, "disrupted the data capture industry by using contextual document data to deliver higher automated processing than earlier technology" said Henry Ijams, Managing Director and Founder, PayStream Advisors. Brainware was awarded a Technology Excellence Award by PayStream Advisors and their Advisory Board to honor those providers who are delivering industry leading solutions. Brainware said its software "could relieve a company of 60 percent to 80 percent of the work of manually keying in information from unstructured documents," and serviced companies such as NEC, Mayo Clinic, Bechtel, Royal Dutch Shell, and Rabobank. In a 2011 comparison report, Real Story Group classifies Brainware as a "Capture Solutions" vendor, competing directly with Kofax and ReadSoft. Brainware and its customers were profiled in publications including Profit Online, Business Finance, imageSource, Managing Automation, Industryweek, Treasury & Risk and others. The company's enterprise search technology has been profiled by InfoWorld.
Read more →
Sample (graphics)

In computer graphics, a sample is an intersection of a channel and a pixel. The diagram below depicts a 24-bit pixel, consisting of 3 samples for Red, Green, and Blue. In this particular diagram, the Red sample occupies 9 bits, the Green sample occupies 7 bits and the Blue sample occupies 8 bits, totaling 24 bits per pixel. Note that the samples do not have to be equal size and not all samples are mandatory in a pixel. Also, a pixel can consist of more than 3 samples (e.g. 4 samples of the RGBA color space). A sample is related to a subpixel on a physical display.
Read more →
Best AI Resume Builders in 2026

Looking for the best AI resume builder? An AI resume builder is software that uses machine learning to help you get more done — it can save you hours every week by automating repetitive work. Most options offer a generous free tier, with paid plans unlocking higher limits, faster processing, and team features. Whether you are a beginner or a pro, the right AI resume builder slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.
Read more →
Best AI Resume Builders in 2026

Looking for the best AI resume builder? An AI resume builder is software that uses machine learning to help you get more done — it can save you hours every week by automating repetitive work. Most options offer a generous free tier, with paid plans unlocking higher limits, faster processing, and team features. Whether you are a beginner or a pro, the right AI resume builder slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.
Read more →