In machine learning, the kernel embedding of distributions (also called the kernel mean or mean map) comprises a class of nonparametric methods in which a probability distribution is represented as an element of a reproducing kernel Hilbert space (RKHS). A generalization of the individual data-point feature mapping done in classical kernel methods, the embedding of distributions into infinite-dimensional feature spaces can preserve all of the statistical features of arbitrary distributions, while allowing one to compare and manipulate distributions using Hilbert space operations such as inner products, distances, projections, linear transformations, and spectral analysis. This learning framework is very general and can be applied to distributions over any space Ω {\displaystyle \Omega } on which a sensible kernel function (measuring similarity between elements of Ω {\displaystyle \Omega } ) may be defined. For example, various kernels have been proposed for learning from data which are: vectors in R d {\displaystyle \mathbb {R} ^{d}} , discrete classes/categories, strings, graphs/networks, images, time series, manifolds, dynamical systems, and other structured objects. The theory behind kernel embeddings of distributions has been primarily developed by Alex Smola, Le Song, Arthur Gretton, and Bernhard Schölkopf. A review of recent works on kernel embedding of distributions can be found in. The analysis of distributions is fundamental in machine learning and statistics, and many algorithms in these fields rely on information theoretic approaches such as entropy, mutual information, or Kullback–Leibler divergence. However, to estimate these quantities, one must first either perform density estimation, or employ sophisticated space-partitioning/bias-correction strategies which are typically infeasible for high-dimensional data. Commonly, methods for modeling complex distributions rely on parametric assumptions that may be unfounded or computationally challenging (e.g. Gaussian mixture models), while nonparametric methods like kernel density estimation (Note: the smoothing kernels in this context have a different interpretation than the kernels discussed here) or characteristic function representation (via the Fourier transform of the distribution) break down in high-dimensional settings. Methods based on the kernel embedding of distributions sidestep these problems and also possess the following advantages: Data may be modeled without restrictive assumptions about the form of the distributions and relationships between variables Intermediate density estimation is not needed Practitioners may specify the properties of a distribution most relevant for their problem (incorporating prior knowledge via choice of the kernel) If a characteristic kernel is used, then the embedding can uniquely preserve all information about a distribution, while thanks to the kernel trick, computations on the potentially infinite-dimensional RKHS can be implemented in practice as simple Gram matrix operations Dimensionality-independent rates of convergence for the empirical kernel mean (estimated using samples from the distribution) to the kernel embedding of the true underlying distribution can be proven. Learning algorithms based on this framework exhibit good generalization ability and finite sample convergence, while often being simpler and more effective than information theoretic methods Thus, learning via the kernel embedding of distributions offers a principled drop-in replacement for information theoretic approaches and is a framework which not only subsumes many popular methods in machine learning and statistics as special cases, but also can lead to entirely new learning algorithms. == Definitions == Let X {\displaystyle X} denote a random variable with domain Ω {\displaystyle \Omega } and distribution P {\displaystyle P} . Given a symmetric, positive-definite kernel k : Ω × Ω → R {\displaystyle k:\Omega \times \Omega \rightarrow \mathbb {R} } the Moore–Aronszajn theorem asserts the existence of a unique RKHS H {\displaystyle {\mathcal {H}}} on Ω {\displaystyle \Omega } (a Hilbert space of functions f : Ω → R {\displaystyle f:\Omega \to \mathbb {R} } equipped with an inner product ⟨ ⋅ , ⋅ ⟩ H {\displaystyle \langle \cdot ,\cdot \rangle _{\mathcal {H}}} and a norm ‖ ⋅ ‖ H {\displaystyle \|\cdot \|_{\mathcal {H}}} ) for which k {\displaystyle k} is a reproducing kernel, i.e., in which the element k ( x , ⋅ ) {\displaystyle k(x,\cdot )} satisfies the reproducing property ⟨ f , k ( x , ⋅ ) ⟩ H = f ( x ) ∀ f ∈ H , ∀ x ∈ Ω . {\displaystyle \langle f,k(x,\cdot )\rangle _{\mathcal {H}}=f(x)\qquad \forall f\in {\mathcal {H}},\quad \forall x\in \Omega .} One may alternatively consider x ↦ k ( x , ⋅ ) {\displaystyle x\mapsto k(x,\cdot )} as an implicit feature mapping φ : Ω → H {\displaystyle \varphi :\Omega \rightarrow {\mathcal {H}}} (which is therefore also called the feature space), so that k ( x , x ′ ) = ⟨ φ ( x ) , φ ( x ′ ) ⟩ H {\displaystyle k(x,x')=\langle \varphi (x),\varphi (x')\rangle _{\mathcal {H}}} can be viewed as a measure of similarity between points x , x ′ ∈ Ω . {\displaystyle x,x'\in \Omega .} While the similarity measure is linear in the feature space, it may be highly nonlinear in the original space depending on the choice of kernel. === Kernel embedding === The kernel embedding of the distribution P {\displaystyle P} in H {\displaystyle {\mathcal {H}}} (also called the kernel mean or mean map) is given by: μ X := E [ k ( X , ⋅ ) ] = E [ φ ( X ) ] = ∫ Ω φ ( x ) d P ( x ) {\displaystyle \mu _{X}:=\mathbb {E} [k(X,\cdot )]=\mathbb {E} [\varphi (X)]=\int _{\Omega }\varphi (x)\ \mathrm {d} P(x)} If P {\displaystyle P} allows a square integrable density p {\displaystyle p} , then μ X = E k p {\displaystyle \mu _{X}={\mathcal {E}}_{k}p} , where E k {\displaystyle {\mathcal {E}}_{k}} is the Hilbert–Schmidt integral operator. A kernel is characteristic if the mean embedding μ : { family of distributions over Ω } → H {\displaystyle \mu :\{{\text{family of distributions over }}\Omega \}\to {\mathcal {H}}} is injective. Each distribution can thus be uniquely represented in the RKHS and all statistical features of distributions are preserved by the kernel embedding if a characteristic kernel is used. === Empirical kernel embedding === Given n {\displaystyle n} training examples { x 1 , … , x n } {\displaystyle \{x_{1},\ldots ,x_{n}\}} drawn independently and identically distributed (i.i.d.) from P , {\displaystyle P,} the kernel embedding of P {\displaystyle P} can be empirically estimated as μ ^ X = 1 n ∑ i = 1 n φ ( x i ) {\displaystyle {\widehat {\mu }}_{X}={\frac {1}{n}}\sum _{i=1}^{n}\varphi (x_{i})} === Joint distribution embedding === If Y {\displaystyle Y} denotes another random variable (for simplicity, assume the co-domain of Y {\displaystyle Y} is also Ω {\displaystyle \Omega } with the same kernel k {\displaystyle k} which satisfies ⟨ φ ( x ) ⊗ φ ( y ) , φ ( x ′ ) ⊗ φ ( y ′ ) ⟩ = k ( x , x ′ ) k ( y , y ′ ) {\displaystyle \langle \varphi (x)\otimes \varphi (y),\varphi (x')\otimes \varphi (y')\rangle =k(x,x')k(y,y')} ), then the joint distribution P ( x , y ) ) {\displaystyle P(x,y))} can be mapped into a tensor product feature space H ⊗ H {\displaystyle {\mathcal {H}}\otimes {\mathcal {H}}} via C X Y = E [ φ ( X ) ⊗ φ ( Y ) ] = ∫ Ω × Ω φ ( x ) ⊗ φ ( y ) d P ( x , y ) {\displaystyle {\mathcal {C}}_{XY}=\mathbb {E} [\varphi (X)\otimes \varphi (Y)]=\int _{\Omega \times \Omega }\varphi (x)\otimes \varphi (y)\ \mathrm {d} P(x,y)} By the equivalence between a tensor and a linear map, this joint embedding may be interpreted as an uncentered cross-covariance operator C X Y : H → H {\displaystyle {\mathcal {C}}_{XY}:{\mathcal {H}}\to {\mathcal {H}}} from which the cross-covariance of functions f , g ∈ H {\displaystyle f,g\in {\mathcal {H}}} can be computed as Cov ( f ( X ) , g ( Y ) ) := E [ f ( X ) g ( Y ) ] − E [ f ( X ) ] E [ g ( Y ) ] = ⟨ f , C X Y g ⟩ H = ⟨ f ⊗ g , C X Y ⟩ H ⊗ H {\displaystyle \operatorname {Cov} (f(X),g(Y)):=\mathbb {E} [f(X)g(Y)]-\mathbb {E} [f(X)]\mathbb {E} [g(Y)]=\langle f,{\mathcal {C}}_{XY}g\rangle _{\mathcal {H}}=\langle f\otimes g,{\mathcal {C}}_{XY}\rangle _{{\mathcal {H}}\otimes {\mathcal {H}}}} Given n {\displaystyle n} pairs of training examples { ( x 1 , y 1 ) , … , ( x n , y n ) } {\displaystyle \{(x_{1},y_{1}),\dots ,(x_{n},y_{n})\}} drawn i.i.d. from P {\displaystyle P} , we can also empirically estimate the joint distribution kernel embedding via C ^ X Y = 1 n ∑ i = 1 n φ ( x i ) ⊗ φ ( y i ) {\displaystyle {\widehat {\mathcal {C}}}_{XY}={\frac {1}{n}}\sum _{i=1}^{n}\varphi (x_{i})\otimes \varphi (y_{i})} === Conditional distribution embedding === Given a conditional distribution P ( y ∣ x ) , {\displaystyle P(y\mid x),} one can define the corresponding RKHS embedding as μ Y ∣ x = E [ φ ( Y ) ∣ X ] = ∫ Ω φ ( y ) d P ( y ∣ x ) {\displaystyle \mu _{Y\mid x}=\mathbb {E} [\varphi (Y)\mid X]=\int _{\Omega
Imieliński–Lipski algebra
In database theory, Imieliński–Lipski algebra is an extension of relational algebra onto tables with different types of null values. It is used to operate on relations with incomplete information. Imieliński–Lipski algebras are defined to satisfy precise conditions for semantically meaningful extension of the usual relational operators, such as projection, selection, union, and join, from operators on relations to operators on relations with various kinds of "null values". These conditions require that the system be safe in the sense that no incorrect conclusion is derivable by using a specified subset F of the relational operators; and that it be complete in the sense that all valid conclusions expressible by relational expressions using operators in F are in fact derivable in this system. For example, it is well known that the three-valued logic approach to deal with null values, supported treatment of nulls values by SQL is not complete, see Ullman book. To show this, let T be: Take SQL query Q SQL query Q will return empty set (no results) under 3-valued semantics currently adopted by all variants of SQL. This is the case because in SQL, NULL is never equal to any constant – in this case, neither to “Spring” nor “Fall” nor “Winter” (if there is Winter semester in this school). NULL='Spring' will evaluate to MAYBE and so will NULL='Fall'. The disjunction MAYBE OR MAYBE evaluates to MAYBE (not TRUE). Thus Igor will not be part of the answer (and of course neither will Rohit). But Igor should be returned as the answer. Indeed, regardless what semester Igor took the Networks class (no matter what was the unknown value of NULL), the selection condition will be true. This “Igor” will be missed by SQL and the SQL answer would be incomplete according to completeness requirements specified in Tomasz Imieliński, Witold Lipski, 'Incomplete Information in Relational Databases'. It is also argued there that 3-valued logic (TRUE, FALSE, MAYBE) can never provide guarantee of complete answer for tables with incomplete information. Three algebras which satisfy conditions of safety and completeness are defined as Imielinski–Lipski algebras: the Codd-Tables algebra, the V-tables algebra and the Conditional tables (C-tables) algebra. == Codd-tables algebra == Codd-tables algebra is based on the usual Codd's single NULL values. The table T above is an example of Codd-table. Codd-table algebra supports projection and positive selections only. It is also demonstrated in [IL84 that it is not possible to correctly extend more relational operators over Codd-Tables. For example, such basic operation as join is not extendable over Codd-tables. It is not possible to define selections with Boolean conditions involving negation and preserve completeness. For example, queries like the above query Q cannot be supported. In order to be able to extend more relational operators, more expressive form of null value representation is needed in tables which are called V-table. == V-tables algebra == V-tables algebra is based on many different ("marked") null values or variables allowed to appear in a table. V-tables allow to show that a value may be unknown but the same for different tuples. For example, in the table below Gaurav and Igor order the same (but unknown) beer in two unknown bars (which may, or may not be different – but remain unknown). Gaurav and Jane frequent the same unknown bar (Y1). Thus, instead one NULL value, we use indexed variables, or Skolem constants . V-tables algebra is shown to correctly support projection, positive selection (with no negation occurring in the selection condition), union, and renaming of attributes, which allows for processing arbitrary conjunctive queries. A very desirable property enjoyed by the V-table algebra is that all relational operators on tables are performed in exactly the same way as in the case of the usual relations. === Conditional tables (c-tables) algebra === Example of conditional table (c-table) is shown below. It has additional column “con” which is a Boolean condition involving variables, null values – same as in V-tables. over the following table c-table Conditional tables algebra, mainly of theoretical interest, supports projection, selection, union, join, and renaming. Under closed-world assumption, it can also handle the operator of difference, thus it can support all relational operators. == History == Imieliński–Lipski algebras were introduced by Tomasz Imieliński and Witold Lipski Jr. in Incomplete Information in Relational Databases.
Personal knowledge base
A personal knowledge base (PKB) is an electronic tool used by an individual to express, capture, and later retrieve personal knowledge. It differs from a traditional database in that it contains subjective material particular to the owner, that others may not agree with nor care about. Importantly, a PKB consists primarily of knowledge, rather than information; in other words, it is not a collection of documents or other sources an individual has encountered, but rather an expression of the distilled knowledge the owner has extracted from those sources or from elsewhere. The term personal knowledge base was mentioned as early as the 1980s, but the term came to prominence in the 2000s when it was described at length in publications by computer scientist Stephen Davies and colleagues, who compared PKBs on a number of different dimensions, the most important of which is the data model that each PKB uses to organize knowledge. == Data models == Davies and colleagues examined three aspects of the data models of PKBs: their structural framework, which prescribes rules about how knowledge elements can be structured and interrelated (as a tree, graph, tree plus graph, spatially, categorically, as n-ary links, chronologically, or ZigZag); their knowledge elements, or basic building blocks of information that a user creates and works with, and the level of granularity of those knowledge elements (such as word/concept, phrase/proposition, free text notes, links to information sources, or composite); and their schema, which involves the level of formal semantics introduced into the data model (such as a type system and related schemas, keywords, attribute–value pairs, etc.). Davies and colleagues also emphasized the principle of transclusion, "the ability to view the same knowledge element (not a copy) in multiple contexts", which they considered to be "pivotal" to an ideal PKB. They concluded, after reviewing many design goals, that the ideal PKB was still to come in the future. === Personal knowledge graph === In their publications on PKBs, Davies and colleagues discussed knowledge graphs as they were implemented in some software of the time. Later, other writers used the term personal knowledge graph (PKG) to refer to a PKB featuring a graph structure and graph visualization. However, the term personal knowledge graph is also used by software engineers to refer to the different subject of a knowledge graph about a person, in contrast to a knowledge graph created by a person in a PKB. == Software architecture == Davies and colleagues also differentiated PKBs according to their software architecture: file-based, database-based, or client–server systems (including Internet-based systems accessed through desktop computers and/or handheld mobile devices). == History == Non-electronic personal knowledge bases have probably existed in some form for centuries: Leonardo da Vinci's journals and notes are a famous example of the use of notebooks. Commonplace books, florilegia, annotated private libraries, and card files (in German, Zettelkästen) of index cards and edge-notched cards are examples of formats that have served this function in the pre-electronic age. Undoubtedly the most famous early formulation of an electronic PKB was Vannevar Bush's description of the "memex" in 1945. In a 1962 technical report, human–computer interaction pioneer Douglas Engelbart (who would later become famous for his 1968 "Mother of All Demos" that demonstrated almost all the fundamental elements of modern personal computing) described his use of edge-notched cards to partially model Bush's memex. == Examples == The following software applications have been used to build PKBs using various data models and architectures. The list includes software mentioned by Davies and colleagues in their 2005 paper, and additional software. Open source Compendium Haystack (MIT project) Joplin Logseq NoteCards Org-mode QOwnNotes TiddlyWiki Closed source Evernote Microsoft OneNote MindManager MyLifeBits Notion Obsidian Personal Knowbase PersonalBrain Roam Tinderbox
Cortica
Headquartered in Tel Aviv Cortica utilizes unsupervised learning methods to recognize and analyze digital images and video. The technology developed by the Cortica team is based on research of the function of the human brain. == Company Founding == Cortica was founded in 2007 by Igal Raichelgauz, Karina Odinaev and Yehoshua Zeevi. Together, the founders developed the company’s core technology while at Technion – Israel Institute of Technology. By combining discoveries in neuroscience with developments in computer programming, the team created technology that possesses the ability to interpret large amounts of visual data with increased accuracy. This technology, called Image2Text, is based on the founders’ work in digitally replicating cortical neural networks’ ability to identify complex patterns within massive quantities of ambiguous and noisy data. Cortica’s offerings have application in the automotive industry, media industries, as well as the smart city and medical industries. Industry experts suggest that the self-driving automotive industry alone will be worth upwards of $7 trillion while each connected car is expected to generate 4,000 GB of data per day. Beyond that, industry analysts expect the proliferation of surveillance cameras to continue leading to an expected 2,500 Petabytes of data being generated daily by new surveillance cameras. Cortica operates in these high scale industries. The company currently employs professionals from many domains including AI researchers as well as veterans of intelligence units within the Israeli Defense Forces. == Research and Technology == In 2006, Founders Raichelgauz, Odinaev, and Zeevi shared their findings with the 28th IEEE EMBS Annual International Conference in New York in a paper titled, “Natural Signal Classification by Neural Cliques and Phase-Locked Attractors”. That same year, the team also published “Cliques in Neural Ensembles as Perception Carriers" CB Insights recently identified Cortica as the number one patent holder among AI companies. Cortica is researching to develop a machine-learning driving system which can identify objects and pedestrians. Connecting to it, Elon Musk has been rumored to partner with Cortica for his electric car company, Tesla. However, Tesla denies it stating that Musk did not discuss a collaboration with artificial intelligence firm Cortica. == Funding == Cortica raised $7 million in its Series A funding round, announced in August 2012. Investors included Horizons Ventures (the investment firm of Hong Kong billionaire Li Ka-Shing), and Ynon Kreiz, the former chairman and CEO of the Endemol Group. In May 2013, it was announced that Cortica had raised $1.5 million from Russian firm Mail.ru Group. It later transpired that this was a part of Cortica's Series B funding round for $6.4 million, announced in June 2013. The round was led by Horizons Ventures, with participation from the Russian firm Mail.ru Group and other angel investors. In its fourth funding round, Cortica has raised $20 million, bringing the total investments to $38 million. According to a report from The Israeli lead Daily economic newspaper, TheMarker, the fourth round was led by a strategic Chinese investor who will probably help the company expand into the Asian market. == Media coverage == GigaOm listed Cortica as one of the top deep learning startups in a November 2013 article surveying the field, along with AlchemyAPI, Ersatz, and Semantria. Business Insider ranked Cortica as one of the coolest tech companies in Israel. CB Insights has identified Cortica as the top patent holding AI company. In 2017 several leading automotive media outlets covered the launch of Cortica's automotive business unit
Minion (solver)
Minion is a solver for satisfaction problems. Unlike constraint programming toolkits, which expect users to write programs in a traditional programming language like C++, Java or Prolog, Minion takes a text file which specifies the problem, and solves using only this. This makes using Minion much simpler, at the cost of much less customization. Minion has been shown to be faster than major commercial constraint solvers including CPLEX (formerly IBM ILOG). == Overview == Minion was introduced in 2006 by researchers at the University of St Andrews as a “fast, scalable” solver for large and hard CSP instances. The project provides a compact input language and a low-overhead C++ implementation aimed at throughput and memory efficiency. == Design and features == Minion implements a range of variable and constraint types commonly used in CSP modelling, plus search heuristics and optimisation support. The solver architecture prioritises cache-friendly data structures and specialised propagators. Notably, the developers adapted watched literal techniques from SAT solving to speed up constraint propagation for, among others, Boolean sums, the element global constraint, and table constraints. The modelling approach relies on a plain-text format (parsed by Minion) rather than embedding models into a host programming language. This reduces overhead and supports rapid “model-and-run” experimentation for large benchmark sets. == Performance == In the original evaluation on standard benchmarks, the authors reported that Minion often ran between one and two orders of magnitude faster than state-of-the-art toolkits of the time (including ILOG Solver and Gecode) on large, hard instances, with smaller gains—or slowdowns—on easier problems. Subsequent research has used Minion as a baseline solver in empirical studies and test generation tasks, reflecting its adoption within parts of the constraint programming community. == Applications == Minion has been applied in academic work on combinatorial search, scheduling and test generation, and is available to other environments via wrappers (for example, from the R language).
Intel Threat Detection Technology
Intel Threat Detection Technology (TDT) is a CPU-level technology created by Intel in 2018 to enable host endpoint protections to use a CPU's low-level access to detect threats to a system. TDT consists of multiple components including Accelerated Memory Scanning, which uses the CPU's integrated GPU to scan memory, and Advanced Platform Telemetry, which uses processor-level activity monitoring to detect unusual activity. It is supported on sixth-generation or newer Intel Core CPUs and additional capabilities were added to the 11th generation Core processors. Intel TDT is integrated into several third-party anti-malware solutions including Microsoft Defender, Check Point Harmony Endpoint, CrowdStrike Falcon, and others. == Accelerated Memory Scanning == Accelerated Memory Scanning (also referred to as "Advanced Memory Scanning") uses the CPU's integrated GPU to scan memory for malicious code, instead of using the CPU directly. This improves system responsiveness during anti-malware scanning. and lowers power consumption. Features include pattern matching, using random forest decision trees, string extraction, entropy calculation, and Euclidean clustering. == Advanced Platform Telemetry == Advanced Platform Telemetry collects CPU-level telemetry to detect uncommon activity patterns which might be indicative of malware. The telemetry data is collected from the CPU performance monitoring unit (PMU) and doesn't require a large signature database to detect malware. Instead, it uses machine-learning based correlations to identify indicators of attack For example, Microsoft Defender is able to use TDT's Advanced Platform Telemetry features to detect processor usage patterns indicative of ransomware and cryptojacking with TDT so it can detect them.
International Journal of Pattern Recognition and Artificial Intelligence
The International Journal of Pattern Recognition and Artificial Intelligence was founded in 1987 and is published by World Scientific. The journal covers developments in artificial intelligence, and its sub-field, pattern recognition. This includes articles on image and language processing, robotics and neural networks. == Abstracting and indexing == The journal is abstracted and indexed in: SciSearch ISI Alerting Services CompuMath Citation Index Current Contents/Engineering, Computing & Technology Inspec io-port.net Compendex Computer Abstracts