AS2 (Applicability Statement 2) is a specification on how to transport structured business-to-business data securely and reliably over the Internet. Security is achieved by using digital certificates and encryption. == Background == AS2 was created in 2002 by the IETF to replace AS1, which they created in the early 1990s. The adoption of AS2 grew rapidly throughout the early 2000s because major players in the retail and fast-moving consumer goods industries championed AS2. Walmart was the first major retailer to require its suppliers to use the AS2 protocol instead of relying on dial-up modems for ordering goods. Amazon, Target, Lowe's, Bed, Bath, & Beyond and thousands of others followed suit. Many other industries use the AS2 protocol, including healthcare, as AS2 meets legal HIPAA requirements. In some cases, AS2 is a way to bypass expensive value-added networks previously used for data interchange. == Technical overview == AS2 is specified in RFC 4130, and is based on HTTP and S/MIME. It was the second AS protocol developed and uses the same signing, encryption and MDN (as defined by RFC3798) conventions used in the original AS1 protocol introduced in the late 1990s by IETF. In other words: Files are encoded as "attachments" in a standardized S/MIME message (an AS2 message). AS2 does not specify the contents of the files. Usually, the file contents are in a standardized format that is separately agreed upon, such as XML or EDIFACT. AS2 messages are always sent using the HTTP or HTTPS protocol (Secure Sockets Layer — also known as SSL — is implied by HTTPS) and usually use the "POST" method (use of "GET" is rare). Messages can be signed, but do not have to be. Messages can be encrypted, but do not have to be. Messages may request a Message Disposition Notification (MDN) back if all went well, but do not have to request such a message. If the original AS2 message requested an MDN: Upon the receipt of the message and its successful decryption or signature validation (as necessary) a "success" MDN will be sent back to the original sender. This MDN is typically signed but never encrypted (unless temporarily encrypted in transit via HTTPS). Upon the receipt and successful verification of the signature on the MDN, the original sender will "know" that the recipient got their message (this provides the "Non-repudiation" element of AS2). If there are any problems receiving or interpreting the original AS2 message, a "failed" MDN may be sent back. However, part of the AS2 protocol states that the client must treat a lack of an MDN as a failure as well, so some AS2 receivers will not return an MDN in this case. Like any other AS file transfer, AS2 file transfers typically require both sides of the exchange to trade X.509 certificates and specific "trading partner" names before any transfers can take place. AS2 trading partner names can usually be any valid phrase. === MDN options === Unlike AS1 or AS3 file transfers, AS2 file transfers offer several "MDN return" options instead of the traditional options of "yes" or "no". Specifically, the choices are: ==== AS2 w/ "Sync" MDNs ==== Return Synchronous MDN via HTTP(S) ("AS2 Sync") - This popular option allows AS2 MDNs to be returned to AS2 message sender clients over the same HTTP connection they used to send the original message. This "MDN while you wait" capability makes "AS2 Sync" transfers the fastest of any type of AS file transfer, but it also keeps this flavor of MDN requests from being used with large files (which may time out in low-bandwidth situations). ==== AS2 w/ "ASync" MDNs ==== Return Asynchronous MDN via HTTP(S) (a.k.a. "AS2 Async") - This popular option allows AS2 MDNs to be returned to the AS2 message sender's server later over a different HTTP connection. This flavor of MDN request is usually used if large files are involved or if your trading partner's AS2 server has poor Internet service. ==== AS2 w/ "Email" MDNs ==== Return (Asynchronous) MDN via Email - This rarely used option allows AS2 MDNs to be returned to AS2 message senders via email rather than HTTP. Otherwise, it is similar to "AS2 Async (HTTP)". ==== AS2 w/ No MDNs ==== Do not return MDN - This option works like it does in any other AS protocol: the receiver of an AS2 message with this option set simply does not try to return an MDN to the AS2 message sender. ==== Filename preservation ==== AS2 filename preservation feature will be used to communicate the filename to the trading partner. The banking industry relies on filenames being communicated between trading partners. AS2 vendors are currently certifying that implementation of filename communication conforms to the standard and is interoperable. There are two profiles for filename preservation being optionally tested under AS2 testing: Filename preservation without MDN responses Filename preservation with an associated MDN response certification Walmart recommends contacting Drummond Group, LLC for more information on EDIINT AS2, or for a list of interoperable-testing AS2 software providers. == Benefits == For many businesses, the use of AS2 and electronic data interchange (EDI) is not a choice so much as it is a requirement of doing business with a large customer or partner. That said, AS2 is a universal protocol that has benefits, from both business and technology vantage points. === Business case === Cut costs by using the web for EDI file transfers, AS2 reduces the cost of transactions from expensive VANs. Extend EDI to more partners; with lower costs and universal web connectivity, AS2 allows organizations to implement EDI with partners worldwide that have little EDI infrastructure. Save time by eliminating the need to manually process orders. Eliminate errors by turning manual processes into automated processes. Universal solution — AS2 is established and tested, so no one has to re-invent the wheel. === Technological advantages === Leverage the web: if an organization can share data securely via the web, they already have much of the infrastructure for AS2. Unlimited EDI data — there are no practical limitations on transaction sizes via the web, and AS2 includes features for managing large transfers. Payload Agnostic — AS2 can be used to transport any type of document. While EDI X12, EDIFACT and XML are common, any mutually agreed-upon format may be transferred.
Flektor
Flektor was a web application that allowed users the ability to create and "mashup" their own content (photos, videos, music, etc.) and share it via email, on social networking websites MySpace, Facebook, Blogger, Digg, eBay or on personal blogs. The company's website (Flektor.com) launched on April 2, 2007, and over 40,000 people began utilizing its features just one month later. Flektor closed down in January 2009. Flektor offered tools and widgets that included audio, video, photos, text, and approximately 100 effects, transitions and filters to be used with media. Users could create personalized slideshows, polls, postcards, and streaming video projects which the website calls "fleks". Flektor also offered Chat (used as a MySpace addon) and Movie Editor, which provided the ability to edit content and assets together. Users of Flektor could import media from websites like Photobucket and Google's YouTube, and then edit their content with the site's editing tools. Flektor's erstwhile competitors include Slide.com (founded by PayPal co-founder Max Levchin), RockYou!, Yahoo's JumpCut and Brightcove. == History == Flektor was created by Jason Rubin, Andy Gavin and former HBO executive Jason R. Kay. Both Rubin and Gavin spent most of their careers in the video game industry developing games for publishers like Electronic Arts, Universal Interactive Studios and Sony Computer Entertainment America. They founded a successful game development studio called Naughty Dog and were responsible for games such as Crash Bandicoot and Jak and Daxter. After selling Naughty Dog to Sony, Rubin focused on a comic book series called Iron and the Maiden before teaming up again with Gavin to venture into the web industry with Flektor. Jason Kay spent four years at Home Box Office, working as a consultant to the EVP of Business Development. They recruited former employee and then Naughty Dog Lead Programmer Scott Shumaker to lead the technology team along with Gavin. Ryan Evans joined shortly thereafter, spearheading product development. Flektor is based in Culver City, California. In May 2007, the company was sold to Fox Interactive Media, which is a division of News Corp., for more than $20 million. The deal coincided with Fox's acquisition of Photobucket, an image-hosting and sharing website. Fox Interactive Media already holds possession of MySpace, IGN Entertainment, FOXSports.com, AmericanIdol.com and Rotten Tomatoes. After the acquisition, Rubin, Gavin and Kay departed, leaving the studio in the hands of Shumaker and Evans. In the fall of 2007, Flektor partnered with its sister company, MySpace, and MTV to provide instant audience feedback via polls for the interactive MySpace/ MTV Presidential Dialogues series with presidential candidates Senator Barack Obama, Senator John McCain and John Edwards. Use of Flektor's polling system, enabled hosts John McLaughlin and Geoffrey Garin to cater their questions towards subjects of voter-interest. In the fall of 2008, Flektor built the official site for the 2008 Presidential debates, hosted at MyDebates. In January 2009, due to a company directive to focus on the core MySpace property, Fox Interactive announced that Flektor would be shut down, with some of its technology being incorporated into MySpace.
The Master Algorithm
The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World is a book by Pedro Domingos released in 2015. Domingos wrote the book in order to generate interest from people outside the field. == Overview == The book outlines five approaches of machine learning: inductive reasoning, connectionism, evolutionary computation, Bayes' theorem and analogical modelling. The author explains these tribes to the reader by referring to more understandable processes of logic, connections made in the brain, natural selection, probability and similarity judgments. Throughout the book, it is suggested that each different tribe has the potential to contribute to a unifying "master algorithm". Towards the end of the book the author pictures a "master algorithm" in the near future, where machine learning algorithms asymptotically grow to a perfect understanding of how the world and people in it work. Although the algorithm doesn't yet exist, he briefly reviews his own invention of the Markov logic network. == In the media == In 2016 Bill Gates recommended the book, alongside Nick Bostrom's Superintelligence, as one of two books everyone should read to understand AI. In 2018 the book was noted to be on Chinese Communist Party general secretary Xi Jinping's bookshelf. === Reception === A computer science educator stated in Times Higher Education that the examples are clear and accessible. In contrast, The Economist agreed Domingos "does a good job" but complained that he "constantly invents metaphors that grate or confuse". Kirkus Reviews praised the book, stating that "Readers unfamiliar with logic and computer theory will have a difficult time, but those who persist will discover fascinating insights." A New Scientist review called it "compelling but rather unquestioning".
AlphaChip (controversy)
The AlphaChip controversy refers to a series of public, scholarly, and legal disputes surrounding a 2021 Nature paper by Google-affiliated researchers. The paper describes an approach to macro placement, a stage of chip floorplanning, based on reinforcement learning (RL), a machine learning method in which a system iteratively improves its decisions by optimizing performance-based reward signals. The primary technical question is whether the new techniques are better than existing (non-AI) techniques. Both internal Google studies and external attempts to replicate the algorithm have failed to show the claimed benefits. No head-to-head comparison is available because the data used in the paper is proprietary, and Google has not released any results from running its algorithm on public benchmarks. This has resulted in considerable skepticism over the paper's claims. In addition, the inability of others (both inside and outside of Google) to replicate the claimed results have sparked concerns about the paper’s methodology, reproducibility, and scientific integrity. The lead researchers of the Nature paper were affiliated with Google Brain, which became part of Google DeepMind, and later spun off into the company Ricursive. == Motivation for research: Macro placement in chip layout == Chip design for modern integrated circuits is a complex, expert-driven process that relies on electronic design automation. It determines the performance of the final chip, and takes weeks or months to complete. Advances that produce better designs, or complete the process faster, are commercially and academically significant. Macro placement is a step during chip design that determines the locations of large circuit components (macros) within a chip. It is followed by detailed placement, which places the far more numerous but much smaller standard cells. Alternatively, mixed-size placement simultaneously places both large macros and millions of small cells, requiring algorithms to handle objects that differ by several orders of magnitude in area and mobility. The number of macros per circuit typically ranges from several to thousands. Wiring must be performed after placement, and the details of this wiring strongly influence the power, performance, and area (PPA) of the completed chip. The full wiring calculation is very resource intensive, so placement tools typically use a proxy cost, a simplified objective function used to guide the placement algorithm during training and evaluation. The faithfulness of the chosen proxy cost to the final objective cost is a critical aspect of placer performance. === State of the art as of 2021 === Chips have been designed since the 1960s, so there were many existing methods as of 2021. Available options included manual design, academic tools, and commercial offerings. Academic methods include combinatorial optimization techniques such as simulated annealing, analytical placement, hierarchical heuristics, and as of 2019 reinforcement learning and broader machine learning techniques.. Existing (non-AI) academic tools for solving the same problem include APlace, NTUplace3, ePlace, RePlace, and DREAMPlace. Commercial EDA vendors also offered automated software tools for floorplanning and mixed-size placement. For instance, as of 2019 Cadence’s Innovus implementation software offered a Concurrent Macro Placer (CMP) feature to automatically place large blocks and standard cells. == The 2021 Nature paper and its claims == In 2021, Nature published a paper under the title “A graph‑placement methodology for fast chip design” co‑authored by 21 Google-affiliated researchers. The paper reported that an RL agent could generate macro placements for integrated circuits "in under six hours" and achieve improvements over human-designed layouts in power, timing performance, and area (PPA), standard chip-quality metrics referring respectively to energy consumption, chip operating speed, and silicon footprint (evaluated after wire routing). It introduced a sequential macro placement algorithm in which macros are placed one at a time instead of optimizing their locations concurrently. At each step, the algorithm selects a location for a single macro on a discretized chip canvas, conditioning its decision on the placements of previously placed macros. This sequential formulation converts macro placement into a long-horizon decision process in which early placement choices constrain later ones. After macro placement, force-directed placement is applied to place standard cells connected to the macros. Deep reinforcement learning is used to train a policy network to place macros by maximizing a reward that reflects final placement quality (for example, wirelength and congestion). Policy learning occurs during self‑play for one or multiple circuit designs. Further placement optimizations refine the overall layout by balancing wirelength, density, and overlap constraints, while treating the macro locations produced by the RL policy as fixed obstacles. The approach relies on pre-training, in which the RL model is first trained on a corpus of prior designs (twenty in the Nature paper) to learn general placement patterns before being fine-tuned on a specific chip. Circuit examples used in the study were parts of proprietary Google TPU designs, called blocks (or floorplan partitions). The paper reported results on five blocks and described the approach as generalizable across chip designs. == Controversy == Soon after the paper's publication, controversy arose over whether the claims were true, whether they were sufficiently proven, and whether academic standards were followed. These controversies arose both within Google and among external academic experts. === Internal dispute at Google and legal proceedings === In 2022, Satrajit Chatterjee, a Google engineer involved in reviewing the AlphaChip work, raised concerns internally and drafted an alternative analysis, (Stronger Baselines) arguing that established methods outperformed the RL approach under fair comparison. In March 2022, Google declined to publish this analysis and terminated Chatterjee's employment. Chatterjee filed a wrongful dismissal lawsuit, alleging that representations related to the AlphaChip research involved fraud and scientific misconduct. According to court documents, Chatterjee's study was conducted "in the context of a large potential Google Cloud deal". He noted that it "would have been unethical to imply that we had revolutionary technology when our tests showed otherwise" and claimed Google was deliberately withholding material information. Furthermore, the committee that reviewed his paper and disapproved its publication was allegedly chaired by subordinates of Jeff Dean, a senior co-author of the Nature paper. Google’s subsequent motion to dismiss was denied, holding that Chatterjee had plausibly alleged retaliation for refusing to engage in conduct he believed would violate state or federal law. === External controversy === The external questions can be summarized in four main points: (a) Are the claims supported by the evidence provided? (b) Did the paper provide enough information to allow the results to be independently reproduced and verified? If so, are the results an improvement over existing academic and commercial tools? (c) Were the comparisons in the paper done fairly and with full disclosure? (d) Were academic standards followed? Each of these is discussed below. ==== Are the claims supported by the evidence provided? ==== The Nature paper described the reduction in design-process time as going from "days or weeks" to "hours", but did not provide per-design time breakdowns or specify the number of engineers, their level of expertise, or the baseline tools and workflow against which this comparison was made. It was also unclear whether the "days or weeks" baseline included time spent on other tasks such as functional design changes. The paper also evaluated the method on fewer benchmarks (five) than is common in the field, and showed mixed results across different evaluation goals While the approach was described as improving circuit area, this claim seems unsupported, as the RL optimization did not alter the overall circuit area, as it adjusted only the locations of fixed-shape non-overlapping circuit components within a fixed rectangular layout boundary. ==== Comparison with existing methods, and replicating the algorithm ==== Because macro placement is largely geometric and its fundamental algorithms are not tied to a specific process node, competing approaches can be evaluated on public benchmarks (tests) across technologies, rather than primarily on proprietary internal designs. This is standard procedure when comparing academic placers, see . In contrast, Google has only reported results only on internal proprietary designs, and as of 2026 has not offered comparisons with prior methods on common benchmarks. Researchers at the University of Califor
Symbolic regression
Symbolic regression (SR) is a type of regression analysis that searches the space of mathematical expressions to find the model that best fits a given dataset, both in terms of accuracy and simplicity. No particular model is provided as a starting point for symbolic regression. Instead, initial expressions are formed by randomly combining mathematical building blocks such as mathematical operators, analytic functions, constants, and state variables. Usually, a subset of these primitives will be specified by the person operating it, but that's not a requirement of the technique. The symbolic regression problem for mathematical functions has been tackled with a variety of methods, including recombining equations most commonly using genetic programming, as well as more recent methods utilizing Bayesian methods and neural networks. Another non-classical alternative method to SR is called Universal Functions Originator (UFO), which has a different mechanism, search-space, and building strategy. Further methods such as Exact Learning attempt to transform the fitting problem into a moments problem in a natural function space, usually built around generalizations of the Meijer-G function. By not requiring a priori specification of a model, symbolic regression isn't affected by human bias, or unknown gaps in domain knowledge. It attempts to uncover the intrinsic relationships of the dataset, by letting the patterns in the data itself reveal the appropriate models, rather than imposing a model structure that is deemed mathematically tractable from a human perspective. The fitness function that drives the evolution of the models takes into account not only error metrics (to ensure the models accurately predict the data), but also special complexity measures, thus ensuring that the resulting models reveal the data's underlying structure in a way that's understandable from a human perspective. This facilitates reasoning and favors the odds of getting insights about the data-generating system, as well as improving generalisability and extrapolation behaviour by preventing overfitting. Accuracy and simplicity may be left as two separate objectives of the regression—in which case the optimum solutions form a Pareto front—or they may be combined into a single objective by means of a model selection principle such as minimum description length. It has been proven that symbolic regression is an NP-hard problem. Nevertheless, if the sought-for equation is not too complex it is possible to solve the symbolic regression problem exactly by generating every possible function (built from some predefined set of operators) and evaluating them on the dataset in question. == Difference from classical regression == While conventional regression techniques seek to optimize the parameters for a pre-specified model structure, symbolic regression avoids imposing prior assumptions, and instead infers the model from the data. In other words, it attempts to discover both model structures and model parameters. This approach has the disadvantage of having a much larger space to search, because not only the search space in symbolic regression is infinite, but there are an infinite number of models which will perfectly fit a finite data set (provided that the model complexity isn't artificially limited). This means that it will possibly take a symbolic regression algorithm longer to find an appropriate model and parametrization, than traditional regression techniques. This can be attenuated by limiting the set of building blocks provided to the algorithm, based on existing knowledge of the system that produced the data; but in the end, using symbolic regression is a decision that has to be balanced with how much is known about the underlying system. Nevertheless, this characteristic of symbolic regression also has advantages: because the evolutionary algorithm requires diversity in order to effectively explore the search space, the result is likely to be a selection of high-scoring models (and their corresponding set of parameters). Examining this collection could provide better insight into the underlying process, and allows the user to identify an approximation that better fits their needs in terms of accuracy and simplicity. == Benchmarking == === SRBench === In 2021, SRBench was proposed as a large benchmark for symbolic regression. In its inception, SRBench featured 14 symbolic regression methods, 7 other ML methods, and 252 datasets from PMLB. The benchmark intends to be a living project: it encourages the submission of improvements, new datasets, and new methods, to keep track of the state of the art in SR. === SRBench Competition 2022 === In 2022, SRBench announced the competition Interpretable Symbolic Regression for Data Science, which was held at the GECCO conference in Boston, MA. The competition pitted nine leading symbolic regression algorithms against each other on a novel set of data problems and considered different evaluation criteria. The competition was organized in two tracks, a synthetic track and a real-world data track. ==== Synthetic Track ==== In the synthetic track, methods were compared according to five properties: re-discovery of exact expressions; feature selection; resistance to local optima; extrapolation; and sensitivity to noise. Rankings of the methods were: QLattice PySR (Python Symbolic Regression) uDSR (Deep Symbolic Optimization) ==== Real-world Track ==== In the real-world track, methods were trained to build interpretable predictive models for 14-day forecast counts of COVID-19 cases, hospitalizations, and deaths in New York State. These models were reviewed by a subject expert and assigned trust ratings and evaluated for accuracy and simplicity. The ranking of the methods was: uDSR (Deep Symbolic Optimization) QLattice geneticengine (Genetic Engine) == Non-standard methods == Most symbolic regression algorithms prevent combinatorial explosion by implementing evolutionary algorithms that iteratively improve the best-fit expression over many generations. Recently, researchers have proposed algorithms utilizing other tactics in AI. Silviu-Marian Udrescu and Max Tegmark developed the "AI Feynman" algorithm, which attempts symbolic regression by training a neural network to represent the mystery function, then runs tests against the neural network to attempt to break up the problem into smaller parts. For example, if f ( x 1 , . . . , x i , x i + 1 , . . . , x n ) = g ( x 1 , . . . , x i ) + h ( x i + 1 , . . . , x n ) {\displaystyle f(x_{1},...,x_{i},x_{i+1},...,x_{n})=g(x_{1},...,x_{i})+h(x_{i+1},...,x_{n})} , tests against the neural network can recognize the separation and proceed to solve for g {\displaystyle g} and h {\displaystyle h} separately and with different variables as inputs. This is an example of divide and conquer, which reduces the size of the problem to be more manageable. AI Feynman also transforms the inputs and outputs of the mystery function in order to produce a new function which can be solved with other techniques, and performs dimensional analysis to reduce the number of independent variables involved. The algorithm was able to "discover" 100 equations from The Feynman Lectures on Physics, while a leading software using evolutionary algorithms, Eureqa, solved only 71. AI Feynman, in contrast to classic symbolic regression methods, requires a very large dataset in order to first train the neural network and is naturally biased towards equations that are common in elementary physics.
Evaluation of binary classifiers
Evaluation of a binary classifier typically assigns a numerical value, or values, to a classifier that represent its accuracy. An example is error rate, which measures how frequently the classifier makes a mistake. There are many metrics that can be used; different fields have different preferences. For example, in medicine sensitivity and specificity are often used, while in computer science precision and recall are preferred. An important distinction is between metrics that are independent of the prevalence or skew (how often each class occurs in the population), and metrics that depend on the prevalence – both types are useful, but they have very different properties. Often, evaluation is used to compare two methods of classification, so that one can be adopted and the other discarded. Such comparisons are more directly achieved by a form of evaluation that results in a single unitary metric rather than a pair of metrics. == Contingency table == Given a data set, a classification (the output of a classifier on that set) gives two numbers: the number of positives and the number of negatives, which add up to the total size of the set. To evaluate a classifier, one compares its output to another reference classification – ideally a perfect classification, but in practice the output of another gold standard test – and cross tabulates the data into a 2×2 contingency table, comparing the two classifications. One then evaluates the classifier relative to the gold standard by computing summary statistics of these 4 numbers. Generally these statistics will be scale invariant (scaling all the numbers by the same factor does not change the output), to make them independent of population size, which is achieved by using ratios of homogeneous functions, most simply homogeneous linear or homogeneous quadratic functions. Say we test some people for the presence of a disease. Some of these people have the disease, and our test correctly says they are positive. They are called true positives (TP). Some have the disease, but the test incorrectly claims they don't. They are called false negatives (FN). Some don't have the disease, and the test says they don't – true negatives (TN). Finally, there might be healthy people who have a positive test result – false positives (FP). These can be arranged into a 2×2 contingency table (confusion matrix), conventionally with the test result on the vertical axis and the actual condition on the horizontal axis. These numbers can then be totaled, yielding both a grand total and marginal totals. Totaling the entire table, the number of true positives, false negatives, true negatives, and false positives add up to 100% of the set. Totaling the columns (adding vertically) the number of true positives and false positives add up to 100% of the test positives, and likewise for negatives. Totaling the rows (adding horizontally), the number of true positives and false negatives add up to 100% of the condition positives (conversely for negatives). The basic marginal ratio statistics are obtained by dividing the 2×2=4 values in the table by the marginal totals (either rows or columns), yielding 2 auxiliary 2×2 tables, for a total of 8 ratios. These ratios come in 4 complementary pairs, each pair summing to 1, and so each of these derived 2×2 tables can be summarized as a pair of 2 numbers, together with their complements. Further statistics can be obtained by taking ratios of these ratios, ratios of ratios, or more complicated functions. The contingency table and the most common derived ratios are summarized below; see sequel for details. Note that the rows correspond to the condition actually being positive or negative (or classified as such by the gold standard), as indicated by the color-coding, and the associated statistics are prevalence-independent, while the columns correspond to the test being positive or negative, and the associated statistics are prevalence-dependent. There are analogous likelihood ratios for prediction values, but these are less commonly used, and not depicted above. == Pairs of metrics == Often accuracy is evaluated with a pair of metrics composed in a standard pattern. === Sensitivity and specificity === The fundamental prevalence-independent statistics are sensitivity and specificity. Sensitivity or True Positive Rate (TPR), also known as recall, is the proportion of people that tested positive and are positive (True Positive, TP) of all the people that actually are positive (Condition Positive, CP = TP + FN). It can be seen as the probability that the test is positive given that the patient is sick. With higher sensitivity, fewer actual cases of disease go undetected (or, in the case of the factory quality control, fewer faulty products go to the market). Specificity (SPC) or True Negative Rate (TNR) is the proportion of people that tested negative and are negative (True Negative, TN) of all the people that actually are negative (Condition Negative, CN = TN + FP). As with sensitivity, it can be looked at as the probability that the test result is negative given that the patient is not sick. With higher specificity, fewer healthy people are labeled as sick (or, in the factory case, fewer good products are discarded). The relationship between sensitivity and specificity, as well as the performance of the classifier, can be visualized and studied using the Receiver Operating Characteristic (ROC) curve. In theory, sensitivity and specificity are independent in the sense that it is possible to achieve 100% in both (such as in the red/blue ball example given above). In more practical, less contrived instances, however, there is usually a trade-off, such that they are inversely proportional to one another to some extent. This is because we rarely measure the actual thing we would like to classify; rather, we generally measure an indicator of the thing we would like to classify, referred to as a surrogate marker. The reason why 100% is achievable in the ball example is because redness and blueness is determined by directly detecting redness and blueness. However, indicators are sometimes compromised, such as when non-indicators mimic indicators or when indicators are time-dependent, only becoming evident after a certain lag time. The following example of a pregnancy test will make use of such an indicator. Modern pregnancy tests do not use the pregnancy itself to determine pregnancy status; rather, human chorionic gonadotropin is used, or hCG, present in the urine of gravid females, as a surrogate marker to indicate that a woman is pregnant. Because hCG can also be produced by a tumor, the specificity of modern pregnancy tests cannot be 100% (because false positives are possible). Also, because hCG is present in the urine in such small concentrations after fertilization and early embryogenesis, the sensitivity of modern pregnancy tests cannot be 100% (because false negatives are possible). === Positive and negative predictive values === In addition to sensitivity and specificity, the performance of a binary classification test can be measured with positive predictive value (PPV), also known as precision, and negative predictive value (NPV). The positive prediction value answers the question "If the test result is positive, how well does that predict an actual presence of disease?". It is calculated as TP/(TP + FP); that is, it is the proportion of true positives out of all positive results. The negative prediction value is the same, but for negatives, naturally. ==== Impact of prevalence on predictive values ==== Prevalence has a significant impact on prediction values. As an example, suppose there is a test for a disease with 99% sensitivity and 99% specificity. If 2000 people are tested and the prevalence (in the sample) is 50%, 1000 of them are sick and 1000 of them are healthy. Thus about 990 true positives and 990 true negatives are likely, with 10 false positives and 10 false negatives. The positive and negative prediction values would be 99%, so there can be high confidence in the result. However, if the prevalence is only 5%, so of the 2000 people only 100 are really sick, then the prediction values change significantly. The likely result is 99 true positives, 1 false negative, 1881 true negatives and 19 false positives. Of the 19+99 people tested positive, only 99 really have the disease – that means, intuitively, that given that a patient's test result is positive, there is only 84% chance that they really have the disease. On the other hand, given that the patient's test result is negative, there is only 1 chance in 1882, or 0.05% probability, that the patient has the disease despite the test result. === Precision and recall === Precision and recall can be interpreted as (estimated) conditional probabilities: Precision is given by P ( C = P | C ^ = P ) {\displaystyle P(C=P|{\hat {C}}=P)} while recall is given by P ( C ^ = P | C = P ) {\displaystyle P({\hat {C}}=P|C=P)} , where C ^ {\
Data annotation
Data annotation is the process of labeling or tagging relevant metadata within a dataset to enable machines to interpret the data accurately. The dataset can take various forms, including images, audio files, video footage, or text. == Applications == Data is a fundamental component in the development of artificial intelligence (AI). Training AI models, particularly in computer vision and natural language processing, requires large volumes of annotated data. Proper annotation ensures that machine learning algorithms can recognize patterns and make accurate predictions. Common types of data annotation include classification, bounding boxes, semantic segmentation, and keypoint annotation. Data annotation is used in AI-driven fields, including healthcare, autonomous vehicles, retail, security, and entertainment. By accurately labeling data, machine learning models can perform complex tasks such as object detection, sentiment analysis, and speech recognition with greater precision. This growing demand has led to the emergence of specialized sectors and platforms dedicated to AI training and human-in-the-loop workflows, which often utilize Reinforcement Learning from Human Feedback (RLHF) to refine model behavior. == In computer vision == === Image classification === Image classification, also known as image categorization, involves assigning predefined labels to images. Machine learning algorithms trained on classified images can later recognize objects and differentiate between categories. For instance, an AI model trained to recognize furniture styles can distinguish between Georgian and Rococo armchairs. === Semantic segmentation === Semantic segmentation assigns each pixel in an image to a specific class, such as trees, vehicles, humans, or buildings. This type of annotation enables machine learning models to differentiate objects by grouping similar pixels, allowing for a detailed understanding of an image. === Bounding boxes === Bounding box annotation involves drawing rectangular boxes around objects in an image. This technique is commonly used in autonomous driving, security surveillance, and retail analytics to detect and classify objects such as pedestrians, vehicles, and products on store shelves. === 3D cuboids === 3D cuboid annotation enhances traditional bounding boxes by adding depth, enabling models to predict an object's spatial orientation, movement, and size. This method is particularly useful for autonomous vehicles and robotics, where understanding object dimensions and depth is critical. === Polygonal annotation === For objects with irregular shapes, such as curved or multi-sided items, polygonal annotation provides more precise labeling than bounding boxes. This technique is often used in applications that require detailed object recognition, such as medical imaging or aerial mapping. === Keypoint annotation === Keypoint annotation marks specific points on an object, such as facial landmarks or body joints, to enable tracking and motion analysis. This method is widely used in facial recognition, emotion detection, sports analytics, and augmented reality applications.