In computer science, an enumeration algorithm is an algorithm that enumerates the answers to a computational problem. Formally, such an algorithm applies to problems that take an input and produce a list of solutions, similarly to function problems. For each input, the enumeration algorithm must produce the list of all solutions, without duplicates, and then halt. The performance of an enumeration algorithm is measured in terms of the time required to produce the solutions, either in terms of the total time required to produce all solutions, or in terms of the maximal delay between two consecutive solutions and in terms of a preprocessing time, counted as the time before outputting the first solution. This complexity can be expressed in terms of the size of the input, the size of each individual output, or the total size of the set of all outputs, similarly to what is done with output-sensitive algorithms. == Formal definitions == An enumeration problem P {\displaystyle P} is defined as a relation R {\displaystyle R} over strings of an arbitrary alphabet Σ {\displaystyle \Sigma } : R ⊆ Σ ∗ × Σ ∗ {\displaystyle R\subseteq \Sigma ^{}\times \Sigma ^{}} An algorithm solves P {\displaystyle P} if for every input x {\displaystyle x} the algorithm produces the (possibly infinite) sequence y {\displaystyle y} such that y {\displaystyle y} has no duplicate and z ∈ y {\displaystyle z\in y} if and only if ( x , z ) ∈ R {\displaystyle (x,z)\in R} . The algorithm should halt if the sequence y {\displaystyle y} is finite. == Common complexity classes == Enumeration problems have been studied in the context of computational complexity theory, and several complexity classes have been introduced for such problems. A very general such class is EnumP, the class of problems for which the correctness of a possible output can be checked in polynomial time in the input and output. Formally, for such a problem, there must exist an algorithm A which takes as input the problem input x, the candidate output y, and solves the decision problem of whether y is a correct output for the input x, in polynomial time in x and y. For instance, this class contains all problems that amount to enumerating the witnesses of a problem in the class NP. Other classes that have been defined include the following. In the case of problems that are also in EnumP, these problems are ordered from least to most specific: Output polynomial, the class of problems whose complete output can be computed in polynomial time. Incremental polynomial time, the class of problems where, for all i, the i-th output can be produced in polynomial time in the input size and in the number i. Polynomial delay, the class of problems where the delay between two consecutive outputs is polynomial in the input (and independent from the output). Strongly polynomial delay, the class of problems where the delay before each output is polynomial in the size of this specific output (and independent from the input or from the other outputs). The preprocessing is generally assumed to be polynomial. Constant delay, the class of problems where the delay before each output is constant, i.e., independent from the input and output. The preprocessing phase is generally assumed to be polynomial in the input. == Common techniques == Backtracking: The simplest way to enumerate all solutions is by systematically exploring the space of possible results (partitioning it at each successive step). However, performing this may not give good guarantees on the delay, i.e., a backtracking algorithm may spend a long time exploring parts of the space of possible results that do not give rise to a full solution. Flashlight search: This technique improves on backtracking by exploring the space of all possible solutions but solving at each step the problem of whether the current partial solution can be extended to a partial solution. If the answer is no, then the algorithm can immediately backtrack and avoid wasting time, which makes it easier to show guarantees on the delay between any two complete solutions. In particular, this technique applies well to self-reducible problems. Closure under set operations: If we wish to enumerate the disjoint union of two sets, then we can solve the problem by enumerating the first set and then the second set. If the union is non disjoint but the sets can be enumerated in sorted order, then the enumeration can be performed in parallel on both sets while eliminating duplicates on the fly. If the union is not disjoint and both sets are not sorted then duplicates can be eliminated at the expense of a higher memory usage, e.g., using a hash table. Likewise, the cartesian product of two sets can be enumerated efficiently by enumerating one set and joining each result with all results obtained when enumerating the second step. == Examples of enumeration problems == The vertex enumeration problem, where we are given a polytope described as a system of linear inequalities and we must enumerate the vertices of the polytope. Enumerating the minimal transversals of a hypergraph. This problem is related to monotone dualization and is connected to many applications in database theory and graph theory. Enumerating the answers to a database query, for instance a conjunctive query or a query expressed in monadic second-order. There have been characterizations in database theory of which conjunctive queries could be enumerated with linear preprocessing and constant delay. The problem of enumerating maximal cliques in an input graph, e.g., with the Bron–Kerbosch algorithm Listing all elements of structures such as matroids and greedoids Several problems on graphs, e.g., enumerating independent sets, paths, cuts, etc. Enumerating the satisfying assignments of representations of Boolean functions, e.g., a Boolean formula written in conjunctive normal form or disjunctive normal form, a binary decision diagram such as an OBDD, or a Boolean circuit in restricted classes studied in knowledge compilation, e.g., NNF. == Connection to computability theory == The notion of enumeration algorithms is also used in the field of computability theory to define some high complexity classes such as RE, the class of all recursively enumerable problems. This is the class of sets for which there exist an enumeration algorithm that will produce all elements of the set: the algorithm may run forever if the set is infinite, but each solution must be produced by the algorithm after a finite time.
Learnable function class
In statistical learning theory, a learnable function class is a set of functions for which an algorithm can be devised to asymptotically minimize the expected risk, uniformly over all probability distributions. The concept of learnable classes are closely related to regularization in machine learning, and provides large sample justifications for certain learning algorithms. == Definition == === Background === Let Ω = X × Y = { ( x , y ) } {\displaystyle \Omega ={\mathcal {X}}\times {\mathcal {Y}}=\{(x,y)\}} be the sample space, where y {\displaystyle y} are the labels and x {\displaystyle x} are the covariates (predictors). F = { f : X ↦ Y } {\displaystyle {\mathcal {F}}=\{f:{\mathcal {X}}\mapsto {\mathcal {Y}}\}} is a collection of mappings (functions) under consideration to link x {\displaystyle x} to y {\displaystyle y} . L : Y × Y ↦ R {\displaystyle L:{\mathcal {Y}}\times {\mathcal {Y}}\mapsto \mathbb {R} } is a pre-given loss function (usually non-negative). Given a probability distribution P ( x , y ) {\displaystyle P(x,y)} on Ω {\displaystyle \Omega } , define the expected risk I P ( f ) {\displaystyle I_{P}(f)} to be: I P ( f ) = ∫ L ( f ( x ) , y ) d P ( x , y ) {\displaystyle I_{P}(f)=\int L(f(x),y)dP(x,y)} The general goal in statistical learning is to find the function in F {\displaystyle {\mathcal {F}}} that minimizes the expected risk. That is, to find solutions to the following problem: f ^ = arg min f ∈ F I P ( f ) {\displaystyle {\hat {f}}=\arg \min _{f\in {\mathcal {F}}}I_{P}(f)} But in practice the distribution P {\displaystyle P} is unknown, and any learning task can only be based on finite samples. Thus we seek instead to find an algorithm that asymptotically minimizes the empirical risk, i.e., to find a sequence of functions { f ^ n } n = 1 ∞ {\displaystyle \{{\hat {f}}_{n}\}_{n=1}^{\infty }} that satisfies lim n → ∞ P ( I P ( f ^ n ) − inf f ∈ F I P ( f ) > ϵ ) = 0 {\displaystyle \lim _{n\rightarrow \infty }\mathbb {P} (I_{P}({\hat {f}}_{n})-\inf _{f\in {\mathcal {F}}}I_{P}(f)>\epsilon )=0} One usual algorithm to find such a sequence is through empirical risk minimization. === Learnable function class === We can make the condition given in the above equation stronger by requiring that the convergence is uniform for all probability distributions. That is: The intuition behind the more strict requirement is as such: the rate at which sequence { f ^ n } {\displaystyle \{{\hat {f}}_{n}\}} converges to the minimizer of the expected risk can be very different for different P ( x , y ) {\displaystyle P(x,y)} . Because in real world the true distribution P {\displaystyle P} is always unknown, we would want to select a sequence that performs well under all cases. However, by the no free lunch theorem, such a sequence that satisfies (1) does not exist if F {\displaystyle {\mathcal {F}}} is too complex. This means we need to be careful and not allow too "many" functions in F {\displaystyle {\mathcal {F}}} if we want (1) to be a meaningful requirement. Specifically, function classes that ensure the existence of a sequence { f ^ n } {\displaystyle \{{\hat {f}}_{n}\}} that satisfies (1) are known as learnable classes. It is worth noting that at least for supervised classification and regression problems, if a function class is learnable, then the empirical risk minimization automatically satisfies (1). Thus in these settings not only do we know that the problem posed by (1) is solvable, we also immediately have an algorithm that gives the solution. == Interpretations == If the true relationship between y {\displaystyle y} and x {\displaystyle x} is y ∼ f ∗ ( x ) {\displaystyle y\sim f^{}(x)} , then by selecting the appropriate loss function, f ∗ {\displaystyle f^{}} can always be expressed as the minimizer of the expected loss across all possible functions. That is, f ∗ = arg min f ∈ F ∗ I P ( f ) {\displaystyle f^{}=\arg \min _{f\in {\mathcal {F}}^{}}I_{P}(f)} Here we let F ∗ {\displaystyle {\mathcal {F}}^{}} be the collection of all possible functions mapping X {\displaystyle {\mathcal {X}}} onto Y {\displaystyle {\mathcal {Y}}} . f ∗ {\displaystyle f^{}} can be interpreted as the actual data generating mechanism. However, the no free lunch theorem tells us that in practice, with finite samples we cannot hope to search for the expected risk minimizer over F ∗ {\displaystyle {\mathcal {F}}^{}} . Thus we often consider a subset of F ∗ {\displaystyle {\mathcal {F}}^{}} , F {\displaystyle {\mathcal {F}}} , to carry out searches on. By doing so, we risk that f ∗ {\displaystyle f^{}} might not be an element of F {\displaystyle {\mathcal {F}}} . This tradeoff can be mathematically expressed as In the above decomposition, part ( b ) {\displaystyle (b)} does not depend on the data and is non-stochastic. It describes how far away our assumptions ( F {\displaystyle {\mathcal {F}}} ) are from the truth ( F ∗ {\displaystyle {\mathcal {F}}^{}} ). ( b ) {\displaystyle (b)} will be strictly greater than 0 if we make assumptions that are too strong ( F {\displaystyle {\mathcal {F}}} too small). On the other hand, failing to put enough restrictions on F {\displaystyle {\mathcal {F}}} will cause it to be not learnable, and part ( a ) {\displaystyle (a)} will not stochastically converge to 0. This is the well-known overfitting problem in statistics and machine learning literature. == Example: Tikhonov regularization == A good example where learnable classes are used is the so-called Tikhonov regularization in reproducing kernel Hilbert space (RKHS). Specifically, let F ∗ {\displaystyle {\mathcal {F^{}}}} be an RKHS, and | | ⋅ | | 2 {\displaystyle ||\cdot ||_{2}} be the norm on F ∗ {\displaystyle {\mathcal {F^{}}}} given by its inner product. It is shown in that F = { f : | | f | | 2 ≤ γ } {\displaystyle {\mathcal {F}}=\{f:||f||_{2}\leq \gamma \}} is a learnable class for any finite, positive γ {\displaystyle \gamma } . The empirical minimization algorithm to the dual form of this problem is arg min f ∈ F ∗ { ∑ i = 1 n L ( f ( x i ) , y i ) + λ | | f | | 2 } {\displaystyle \arg \min _{f\in {\mathcal {F}}^{}}\left\{\sum _{i=1}^{n}L(f(x_{i}),y_{i})+\lambda ||f||_{2}\right\}} This was first introduced by Tikhonov to solve ill-posed problems. Many statistical learning algorithms can be expressed in such a form (for example, the well-known ridge regression). The tradeoff between ( a ) {\displaystyle (a)} and ( b ) {\displaystyle (b)} in (2) is geometrically more intuitive with Tikhonov regularization in RKHS. We can consider a sequence of { F γ } {\displaystyle \{{\mathcal {F}}_{\gamma }\}} , which are essentially balls in F ∗ {\displaystyle {\mathcal {F^{}}}} with centers at 0. As γ {\displaystyle \gamma } gets larger, F γ {\displaystyle {\mathcal {F}}_{\gamma }} gets closer to the entire space, and ( b ) {\displaystyle (b)} is likely to become smaller. However we will also suffer smaller convergence rates in ( a ) {\displaystyle (a)} . The way to choose an optimal γ {\displaystyle \gamma } in finite sample settings is usually through cross-validation. == Relationship to empirical process theory == Part ( a ) {\displaystyle (a)} in (2) is closely linked to empirical process theory in statistics, where the empirical risk { ∑ i = 1 n L ( y i , f ( x i ) ) , f ∈ F } {\displaystyle \{\sum _{i=1}^{n}L(y_{i},f(x_{i})),f\in {\mathcal {F}}\}} are known as empirical processes. In this field, the function class F {\displaystyle {\mathcal {F}}} that satisfies the stochastic convergence are known as uniform Glivenko–Cantelli classes. It has been shown that under certain regularity conditions, learnable classes and uniformly Glivenko-Cantelli classes are equivalent. Interplay between ( a ) {\displaystyle (a)} and ( b ) {\displaystyle (b)} in statistics literature is often known as the bias-variance tradeoff. However, note that in the authors gave an example of stochastic convex optimization for General Setting of Learning where learnability is not equivalent with uniform convergence.
Eduard Hovy
Eduard Hovy is a Research Professor in the Language Technologies Institute at Carnegie Mellon University. He is one of the original 17 Fellows of the Association for Computational Linguistics. == Biography == Eduard Hovy received M.S. (December 1982) and Ph.D. (May 1987) degrees in Computer Science from Yale University. He was awarded honorary doctorates from the National University of Distance Education (UNED) in Madrid in 2013 and the University of Antwerp in 2015.
Vlado Keselj
Vlado Keselj (Vlado Kešelj) is a Serbian-Canadian computer scientist known for his research in natural language processing and authorship attribution. He is a professor at Dalhousie University. == Education == As a high school student in Yugoslavia, Keselj competed in the 1987 International Mathematical Olympiad, earning a bronze medal. He earned his Ph.D. in 2002 at the University of Waterloo, with the dissertation Modular Stochastic HPSGs for Question Answering supervised by Nick Cercone. == Awards == Vlado Keselj is a recipient of the 2019 CAIAC Distinguished Service Award, awarded by the Canadian Artificial Intelligence Association (CAIAC). == Selected publications == Kešelj, V., Peng, F., Cercone, N., & Thomas, C. (2003, August). N-gram-based author profiles for authorship attribution. In Proceedings of the Conference of the Pacific Association for Computational Linguistics, PACLING 2003 (Vol. 3, pp. 255–264).
Lübke English
The term Lübke English (or, in German, Lübke-Englisch) refers to nonsensical English created by literal word-by-word translation of German phrases, disregarding differences between the languages in syntax and meaning. Lübke English is named after Heinrich Lübke, a president of Germany in the 1960s, whose limited English made him a target of German humorists. In 2006, the German magazine konkret revealed that most of the statements ascribed to Lübke were in fact invented by the editorship of Der Spiegel, mainly by staff writer Ernst Goyke and subsequent letters to the editor. In the 1980s, comedian Otto Waalkes had a routine called "English for Runaways", which is a nonsensical literal translation of Englisch für Fortgeschrittene (actually an idiom for 'English for advanced speakers' in German – note that fortschreiten divides into fort, meaning "away" or "forward", and schreiten, meaning "to walk in steps"). In this mock "course", he translates every sentence back or forth between English and German at least once (usually from German literally into English). Though there are also other, more complex language puns, the title of this routine has gradually replaced the term Lübke English when a German speaker wants to point out naive literal translations.
T-vertices
T-vertices is a term used in computer graphics to describe a problem that can occur during mesh refinement or mesh simplification. The most common case occurs in naive implementations of continuous level of detail, where a finer-level mesh is "sewn" together with a coarser-level mesh by simply aligning the finer vertices on the edges of the coarse polygons. The result is a continuous mesh, however due to the nature of the z-buffer and certain lighting algorithms such as Gouraud shading, visual artifacts can often be detected. Some modeling algorithms such as subdivision surfaces will fail when a model contains T-vertices.
Li Sheng (computer scientist)
Li Sheng (Chinese: 李生; born 1943), is a professor at the School of Computer Science and Engineering, Harbin Institute of Technology (HIT), China. He began his research on Chinese-English machine translation in 1985, making himself one of the earliest Chinese scholars in this field. After that, he pursued in vast topics of natural language processing, including machine translation, information retrieval, question answering and applied artificial intelligence. He was the final review committee member for computer area in NSF China. Born and raised in Heilongjiang province, he graduated in 1965 from the computer specialty of HIT, which is one of the earliest computer specialties in Chinese universities. Then he started to work as a staff in the Computer specialty of HIT, which was finally granted as a department in 1985. Also from 1985, he was appointed to undertake a series administrative positions in HIT, e.g. Dean of Computer Department(1987–1988), Director of R&D Division (1988–1990), Chief R&D Officer and several other key leading positions in HIT. Resigned all his administrative positions in 2004, Li devoted himself as the director of MOE-Microsoft Join Key Lab of NLP& Speech (HIT), making it a leading NLP research group with more than 100 staffs and students working on various aspects of NLP. So far, the lab has already been granted for dozens of technology awards by the ministries of central government and local provincial government of China. Its research progresses are reported annually in top tier conferences including ACL, IJCAI, SIGIR etc. As one of the pioneers in NLP research in China, he contributes NLP in China not only in technology innovations but also in talents education. So far, his research group has graduated more than 60 Ph.D. and almost 200 M.E with NLP major. Most of them are now working as the chief researcher in various NLP groups of universities and companies in China, including several world-known NLP scholars, such as Wang Haifeng of Baidu, Zhou Ming of Microsoft Research, Zhang Min (张民) of Soochow University (China), and Zhao Tiejun (赵铁军) and Liu Ting (刘挺) of HIT. Owing to his contributions in Chinese language processing, Li was elected as the President of Chinese Information Processing Society of China (CIPSC) in 2011. He scaled this top level academic organization in China up to more than 3000 registered members, and promoted NLP into several national projects for research or industry development. In addition, the CIPSC is now enhancing its co-operations with world NLP organizations including ACL. == Machine Intelligence & Translation Laboratory (MI&TLAB) == Originates from Machine Translation Research Group of Computer Science Department, Harbin Institute of Technology, which was started Li in 1985. It is one of the earliest institutions engaged in MT research in China, featured by its investigations into Chinese-English machine translation. It is now running under the Research Center on Language Technology, School of Computer Science and Technology, HIT. Details for staffs and publications can be found at https://mitlab.hit.edu.cn. == MOE-MS Joint Key Lab of Natural Language Processing and Speech (HIT) == In June, 2000, the Joint HIT-Microsoft Machine Translation Lab was founded by MI&T Lab and Microsoft Research (China). It was the third joint lab established by Microsoft Research (China) with Chinese universities, and the only one focusing on Machine Translation. Based on this jointly lab, the cooperation between HIT and Microsoft gradually extended to the areas of machine translation, information retrieval, speech recognition and processing, natural language understanding. In Oct, 2004, the joint key lab was granted as one of the 10 joint key labs supported by the Microsoft Research of Asia and Ministry of Education in China. In July 2006, the Shenzhen extension of the lab was launched. More than 200 staff and students have undertaken research projects, including some sponsored by the National Natural Science Foundation of China and the National 863 program of China. Since 2005, the lab has also been organizing a summer camp in Harbin Institute of Technology, and approximately 150 faculty members and students from universities in China have participated. This summer workshop was organized annually until 2014, when it was organized formally as the summer school series by Chinese Information Processing Society, China. Through the lab, a Microsoft Research of Asia-HIT joint PhD program was implemented in 2012. == CEMT-I MT System == In May 1989, CEMT-I passed the formal project appraisal in Harbin, China. Capable of translating technical paper titles from Chinese to English, it is not only the first MT system completed by Li and his group, but also the first Chinese-English Translation system that passed the technical appraisal by Chinese government according to the public reports. It was then awarded the Second Prize of Ministry Level Technology Innovation by the former National Aerospace Industry Corporation in 1990. == Daya Translation Workstation == Owing to the technical achievements by Li's group in Chinese-English machine translation, the former National Aerospace Industry Corporation of China sponsored a commercial system development of "Daya Translation Station (MT)" in 1993. Designed as a comprehensive English composition aid for Chinese users, this system was finished and put into the market in 1995. And in 1997, this system was awarded the Second Prize of Ministry Level Technology Innovation by the former National Aerospace Industry Corporation. == BT863 MT System == From 1994, the researches in Li's lab were supported by National 863 Hi-tech Research and Development Program. During this period, the BT863 system was explored to employ one engine for both Chinese-English and English-Chinese translation. This system was proved to be the best performance among Chinese-English MT systems in the formal technical evaluation of National 863 program, yielding the Third Prize of Ministry Level Technology Innovation by the former National Aerospace Industry Corporation in 1997. == Next Generation IR == This is a key project granted by NSF China (with a joint sponsorship from MSRA) started form 2008. In contrast to his previous NSF grants for different NLP issues, Li explored in his last PI project on key technologies in personalized IR, together with researchers from Tsinghua University and Institute of Software, Chinese Academy of Science. With impressive publications in top tier journals and conferences (including breakthrough publications in SIGIR of his own group), this projected was approved "A-level" achievements by the NSF China office in 2012.