Cross-language information retrieval

Cross-language information retrieval

Cross-language information retrieval (CLIR) is a subfield of information retrieval dealing with retrieving information written in a language different from the language of the user's query. The term "cross-language information retrieval" has many synonyms, of which the following are perhaps the most frequent: cross-lingual information retrieval, translingual information retrieval, multilingual information retrieval. The term "multilingual information retrieval" refers more generally both to technology for retrieval of multilingual collections and to technology which has been moved to handle material in one language to another. The term Multilingual Information Retrieval (MLIR) involves the study of systems that accept queries for information in various languages and return objects (text, and other media) of various languages, translated into the user's language. Cross-language information retrieval refers more specifically to the use case where users formulate their information need in one language and the system retrieves relevant documents in another. To do so, most CLIR systems use various translation techniques. CLIR techniques can be classified into different categories based on different translation resources: Dictionary-based CLIR techniques Parallel corpora based CLIR techniques Comparable corpora based CLIR techniques Machine translator based CLIR techniques CLIR systems have improved so much that the most accurate multi-lingual and cross-lingual adhoc information retrieval systems today are nearly as effective as monolingual systems. Other related information access tasks, such as media monitoring, information filtering and routing, sentiment analysis, and information extraction require more sophisticated models and typically more processing and analysis of the information items of interest. Much of that processing needs to be aware of the specifics of the target languages it is deployed in. Mostly, the various mechanisms of variation in human language pose coverage challenges for information retrieval systems: texts in a collection may treat a topic of interest but use terms or expressions which do not match the expression of information need given by the user. This can be true even in a mono-lingual case, but this is especially true in cross-lingual information retrieval, where users may know the target language only to some extent. The benefits of CLIR technology for users with poor to moderate competence in the target language has been found to be greater than for those who are fluent. Specific technologies in place for CLIR services include morphological analysis to handle inflection, decompounding or compound splitting to handle compound terms, and translations mechanisms to translate a query from one language to another. The first workshop on CLIR was held in Zürich during the SIGIR-96 conference. Workshops have been held yearly since 2000 at the meetings of the Cross Language Evaluation Forum (CLEF). Researchers also convene at the annual Text Retrieval Conference (TREC) to discuss their findings regarding different systems and methods of information retrieval, and the conference has served as a point of reference for the CLIR subfield. Early CLIR experiments were conducted at TREC-6, held at the National Institute of Standards and Technology (NIST) on November 19–21, 1997. Google Search had a cross-language search feature that was removed in 2013.

Learning automaton

A learning automaton is one type of machine learning algorithm studied since 1970s. Learning automata select their current action based on past experiences from the environment. It will fall into the range of reinforcement learning if the environment is stochastic and a Markov decision process (MDP) is used. == History == Research in learning automata can be traced back to the work of Michael Lvovitch Tsetlin in the early 1960s in the Soviet Union. Together with some colleagues, he published a collection of papers on how to use matrices to describe automata functions. Additionally, Tsetlin worked on reasonable and collective automata behaviour, and on automata games. Learning automata were also investigated by researches in the United States in the 1960s. However, the term learning automaton was not used until Narendra and Thathachar introduced it in a survey paper in 1974. == Definition == A learning automaton is an adaptive decision-making unit situated in a random environment that learns the optimal action through repeated interactions with its environment. The actions are chosen according to a specific probability distribution which is updated based on the environment response the automaton obtains by performing a particular action. With respect to the field of reinforcement learning, learning automata are characterized as policy iterators. In contrast to other reinforcement learners, policy iterators directly manipulate the policy π. Another example for policy iterators are evolutionary algorithms. Formally, Narendra and Thathachar define a stochastic automaton to consist of: a set X of possible inputs, a set Φ = { Φ1, ..., Φs } of possible internal states, a set α = { α1, ..., αr } of possible outputs, or actions, with r ≤ s, an initial state probability vector p(0) = ≪ p1(0), ..., ps(0) ≫, a computable function A which after each time step t generates p(t+1) from p(t), the current input, and the current state, and a function G: Φ → α which generates the output at each time step. In their paper, they investigate only stochastic automata with r = s and G being bijective, allowing them to confuse actions and states. The states of such an automaton correspond to the states of a "discrete-state discrete-parameter Markov process". At each time step t=0,1,2,3,..., the automaton reads an input from its environment, updates p(t) to p(t+1) by A, randomly chooses a successor state according to the probabilities p(t+1) and outputs the corresponding action. The automaton's environment, in turn, reads the action and sends the next input to the automaton. Frequently, the input set X = { 0,1 } is used, with 0 and 1 corresponding to a nonpenalty and a penalty response of the environment, respectively; in this case, the automaton should learn to minimize the number of penalty responses, and the feedback loop of automaton and environment is called a "P-model". More generally, a "Q-model" allows an arbitrary finite input set X, and an "S-model" uses the interval [0,1] of real numbers as X. A visualised demo/ Art Work of a single Learning Automaton had been developed by μSystems (microSystems) Research Group at Newcastle University. == Finite action-set learning automata == Finite action-set learning automata (FALA) are a class of learning automata for which the number of possible actions is finite or, in more mathematical terms, for which the size of the action-set is finite.

Content-oriented workflow models

In data management, a content-oriented workflow model seeks to articulate workflow progression by the presence of content units (like data-records/objects/documents). Most content-oriented workflow approaches provide a life-cycle model for content units, such that workflow progression can be qualified by conditions on the state of the units. Most approaches are research and work in progress and the content models and life-cycle models are more or less formalized. The term content-oriented workflows is an umbrella term for several scientific workflow approaches, namely "data-driven", "resource-driven", "artifact-centric", "object-aware", and "document-oriented". Thus, the meaning of "content" ranges from simple data attributes to self-contained documents; the term "content-oriented workflows" appeared at first in as an umbrella term. Such a general term, independent from a specific approach, is necessary to contrast the content-oriented modelling principle with traditional activity-oriented workflow models (like Petri nets or BPMN) where a workflow is driven by a control flow and where the content production perspective is neglected or even missing. The term "content" was chosen to subsume the different levels in granularity of the content units in the respective workflow models; it was also chosen to make associations with content management. Both terms "artifact-centric" and "data-driven" would also be good candidates for an umbrella term, but each is closely related to a specific approach of a single working group. The "artifact-centric" group itself (i.e. IBM Research) has generalized the characteristics of their approach and has used "information-centric" as an umbrella term in. Yet, the term information is too unspecific in the context of computer science, thus, "content-orientated workflows" is considered as good compromise. == Workflow Model Approaches == === Data-driven === The data-driven process structures provides a sophisticated workflow model being specialized on hierarchical write-and-review-processes. The approach provides interleaved synchronization of sub-processes and extends activity diagrams. Unfortunately, the COREPRO prototype implementation is not publicly available. Research on the project had been ceased. The general idea has been continued by Reichert in form of the #Object-aware approach. Synonyms data-driven process structures / data-driven modeling and coordination Protagonists Dr. Dominic Müller (University of Twente), Joachim Herbst (DaimlerChrysler Research), and Manfred Reichert (at this time Assoc. Prof. at Univ. of Twente, currently Prof. at Ulm Univ.) Organization(s) University of Twente, DaimlerChrysler Period 2005 - 2007 Selected publications Implementation COREPRO === Resource-driven === The resource-driven workflow system is an early approach that considered workflows from a content-oriented perspective and emphasizes on the missing support for plain document-driven processes by traditional activity-oriented workflow engines. The resource-driven approach demonstrated the application of database triggers for handling workflow events. Still the system implementation is centralized and the workflow schema is statically defined. The project appeared in 2005 but many aspects are considered future work by the authors. Research did not continue on the project. Wang completed his PhD thesis in 2009, yet, his thesis does not mention the resource-driven approach to workflow modelling but is about discrete event simulation. Synonyms Resource-based Workflows / Document-Driven Workflow Systems Protagonists Jianrui Wang and Prof. Akhil Kumar Organization Pennsylvania State University Period 2005 - today Selected publications Implementation N/A === Artifact-centric === The artifact-centric approach provides a framework for content-oriented workflows. In this model, the enterprise application landscape includes distributed business services, while the workflow engine is centralized. Process enactment is integrated with database management system infrastructure, and the project is funded by IBM. Synonyms artifact-centric business process models / artifact-based business process (ACP) / artifact-centric workflows Protagonists Richard Hull and Dr. Kamal Bhattacharya as well as Cagdas E. Gerede and Jianwen Su Organization IBM (T.J. Watson Research Center, NY) Period 2007 - today Selected publications Implementation ArtiFact === Object-aware === The object-aware approach manages a set of object types and generates forms for creating object instances. The form completion flow is controlled by transitions between object configurations each describing a progressing set of mandatory attributes. Each object configuration is named by an object state. The data production flow is user-shifting and it is discrete by defining a sequence of object states. The discussion is currently limited to a centralized system, without any workflows across different organizations. However, the approach is of great relevance to many domains like concurrent engineering. Finally, the object-aware approach and its PHILharmonicFlows system are going to provide general-purpose workflow systems for generic enactment of data production processes. Synonyms object-aware process management / datenorientiertes Prozess-Management-System Protagonists Vera Künzle and Prof. Manfred Reichert Organization Ulm University Period 2009 - today Selected publications Implementation PHILharmonicFlows === Distributed Document-oriented === Distributed document-oriented process management (dDPM) enables distributed case handling in heterogeneous system environments and it is based on document-oriented integration. The workflow model reflects the paper-based working practice in inter-institutional healthcare scenarios. It targets distributed knowledge-driven ad hoc workflows, wherein distributed information systems are required to coordinate work with initially unknown sets of actors and activities. The distributed workflow engine supports process planning & process history as well as participant management and process template creation with import/export. The workflow engine embeds a functional fusion of 1) group-based instant messaging 2) with a shared work list editor 3) with version control. The software implementation of dDPM is α-Flow which is available as open source. dDPM and α-Flow provide a content-oriented approach to schema-less workflows. The complete distributed case handling application is provided in form of a single active Document ("α-Doc"). The α-Doc is a case file (as information carrier) with an embedded workflow engine (in form of active properties). Inviting process participants is equivalent to providing them with a copy of an α-Doc, copying it like an ordinary desktop file. All α-Docs that belong to the same case can synchronize each other, based on the participant management, electronic postboxes, store-and-forward messaging, and an offline-capable synchronization protocol. Synonyms distributed document-oriented process management (dDPM), distributed case handling via active documents Protagonists Christoph P. Neumann and Prof. Richard Lenz Organization Friedrich-Alexander-Universität Erlangen-Nürnberg Period 2009 - 2012 Selected Publications and a PhD thesis Implementation α-Flow (open source) == Related Concepts == === Content Management === The bandwidth of Content management systems (CMS) reaches from Web content management systems (WCMS) and Document management system (DMS) to Enterprise Content Management (ECM). Mature DMS products support document production workflows in a basic form, primarily focusing on review cycle workflows concerning a single document. === Groupware and Computer-Supported Cooperative Work === Groupware focuses on messaging (like E-Mail, Chat, and Instant Messaging), shared calendars (e.g. Lotus Notes, Microsoft Outlook with Exchange Server), and conferencing (e.g. Skype). Groupware overlaps with Computer-supported cooperative work (CSCW), that originated from shared multimedia editors (for live drawing/sketching) and synchronous multi-user applications like desktop sharing. The extensive conceptual claim of CSWC must be put into perspective by its actual solution scope, that is available as the CSCW Matrix. === Case Handling === The case handling paradigm stems from Prof. van der Aalst and gained momentum in 2005. The core features are: (a) provide all information available, i.e. present the case as a whole rather than showing bits and pieces, (b) decide about activities on the basis of the information available rather than the activities already executed, (c) separate work distribution from authorization and allow for additional types of roles, not just the execute role, and (d) allow workers to view and add/modify data before or after the corresponding activities have been executed. In healthcare, the flow of a patient between healthcare professionals is considered as a workflow - with activities that inc

Social recruiting

Social recruiting (social hiring or social media recruitment) is recruiting candidates by using social platforms as talent databases or for advertising. Social recruiting uses social media profiles, blogs, and other Internet sites to find information on candidates. It also uses social media to advertise jobs either through HR vendors or through crowdsourcing where job seekers and others share job openings within their online social networks. Social recruiting's effectiveness and return on investment have been difficult to determine, since applicants do not usually apply through the social channels which first attracted them. In May 2013, Maximum Employment Marketing Group released the Social Recruitment Monitor, which ranks the reach, engagement, and interactivity of employers' social recruiting efforts around the world. == Social recruitment software == The social recruitment software market (a form of e-recruitment) is often included in the wider talent management software sector. Bersin & Associates valued the wider talent management market at over $2bn in 2007. Social recruitment increasingly sits at an intersection of a number of fast-moving areas including social networking, recruitment and now cloud computing. Additionally, mobile recruiting has become another hot topic, especially with the rise in tablet and smartphone usage. In 2012, there was a rise of tech companies using social recruiting applications to find and screen applicants. As more companies saw value in filling jobs by putting them on the social platforms where millions of people spend at least 37 minutes daily, there developed a much larger focus on social recruiting among the talent acquisition community. By mid-2013, many major enterprise companies such as Pepsi, Gap, AIG, and Oracle had begun effectively utilizing social recruiting software, making it clear that large corporations were open to automating or streamlining (and ultimately investing in) their social recruiting processes.

Protecting Kids From Social Media Act

Protecting Kids on Social Media Act or HB 1891 is an American law that was introduced by William Lamberth of Sumner County, Tennessee and was signed into law by Tennessee's governor on May 2, 2024. The bill requires social media websites such as X, YouTube, TikTok, Facebook and others to verify the age of users and if those users are under 18, they must have parental consent. == Progress == The law passed the Tennessee State Legislature with little opposition: the bill had only two no votes in the House from Aftyn Behn and Vincent B. Dixie, and it had zero no votes in the Senate. == Bill summary == Every social media company must verify the age of new users after the law takes effect, and if the user had created an account before the law took effect, they must verify the age of the person attempting to access the account within 14 days. If the new user or the user who originally owned an account is under 18 years of age, they must get parental consent and the third party or social media company must not retain the data from the age verification process or obtaining parental consent. Parents who are account holders of those under 18 can view the privacy settings, set daily time restrictions, and implement breaks during which the minor cannot access the account. The law is enforced by the Attorney General of Tennessee and went into effect on January 1, 2025. == Lawsuit == On October 3, 2024, the trade association NetChoice filed a lawsuit against Tennessee Attorney General Jonathan Skrmetti in the Middle District Court of Tennessee, claiming that the law violates the First Amendment. The Judge for the case is William L. Campbell Jr. An initial case management conference was originally scheduled for December 4, 2024, however it was delayed because of the Supreme Court case United States v. Skrmetti, recommending that the conference be delayed after January 20, 2025. On February 14, 2025, Judge Eli Richardson denied NetChoice's motion for a temporary restraining order because it would disrupt the status quo of the case.

The Triple Revolution

"The Triple Revolution" was an open memorandum sent to U.S. President Lyndon B. Johnson and other government figures on March 22, 1964. It concerned three megatrends of the time: increasing use of automation, the nuclear arms race, and advancements in human rights. Drafted under the auspices of the Center for the Study of Democratic Institutions, it was signed by an array of noted social activists, professors, and technologists who identified themselves as the Ad Hoc Committee on the Triple Revolution. The chief initiator of the proposal was W. H. "Ping" Ferry, at that time a vice-president of CSDI, basing it in large part on the ideas of the futurist Robert Theobald. == Overview == The statement identified three revolutions underway in the world: the cybernation revolution of increasing automation; the weaponry revolution of mutually assured destruction; and the human rights revolution. It discussed primarily the cybernation revolution. The committee claimed that machines would usher in "a system of almost unlimited productive capacity" while continually reducing the number of manual laborers needed, and increasing the skill needed to work, thereby producing increasing levels of unemployment. It proposed that the government should ease this transformation through large-scale public works, low-cost housing, public transit, electrical power development, income redistribution, union representation for the unemployed, and government restraint on technology deployment. == Legacy == Martin Luther King Jr.'s final Sunday sermon, delivered six days before his April 1968 assassination, explicitly references the thesis of "The Triple Revolution": There can be no gainsaying of the fact that a great revolution is taking place in the world today. In a sense it is a triple revolution: that is, a technological revolution, with the impact of automation and cybernation; then there is a revolution in weaponry, with the emergence of atomic and nuclear weapons of warfare; then there is a human rights revolution, with the freedom explosion that is taking place all over the world. Yes, we do live in a period where changes are taking place. And there is still the voice crying through the vista of time saying, "Behold, I make all things new; former things are passed away." In Harlan Ellison's 1967 anthology Dangerous Visions, Philip José Farmer's story "Riders of the Purple Wage" uses the Triple Revolution document as the premise of a future society, in which the "purple wage" of the title is a guaranteed income dole on which most of the population lives. At the 1968 World Science Fiction Convention in San Francisco, Farmer delivered a lengthy Guest of Honor speech in which he called for the founding of a grassroots activist organization called REAP which would work for implementation of the Ad Hoc Committee's recommendations. Looking back on the proposal in his 2008 book, Daniel Bell wrote: "the cybernetic revolution quickly proved to be illusory. There were no spectacular jumps in productivity. ... Cybernation had proved to be one more instance of the penchant for overdramatizing a momentary innovation and blowing it up far out of proportion to its actuality. ... The image of a completely automated production economy—with an endless capacity to turn out goods—was simply a social-science fiction of the early 1960s. Paradoxically, the vision of Utopia was suddenly replaced by the spectre of Doomsday. In place of the early-sixties theme of endless plenty, the picture by the end of the decade was one of a fragile planet of limited resources whose finite stocks were being rapidly depleted, and whose wastes from soaring industrial production were polluting the air and waters." In his 2015 book Rise of the Robots, Martin Ford claims The Triple Revolution's predictions of steady decline in future employment were not wrong, but rather premature. He cites "Seven Deadly Trends" that began in the 1970s-1980s and by the mid-2010s appeared set to continue: Stagnation in real wages Decline in labor's share of national income in many countries (breakdown of Bowley's law), while corporate profits increased Declining labor force participation Diminishing job creation, lengthening jobless recoveries, and soaring long-term unemployment Rising inequality Declining incomes, and underemployment for recent college graduates Polarization and part-time jobs (middle-class jobs are disappearing, to be replaced by a small number of high-paying jobs and large number of low-paying jobs) According to Ford, the 1960s were part of what in retrospect seems like a golden age for labor in the United States, when productivity and wages rose together in near lockstep, and unemployment was low. But after about 1980, wages began stagnating while productivity continued to rise. Labor's share of the economic output began to decline. Ford describes the role that automation and information technology play in these trends, and how new technologies including narrow AI threaten to destroy jobs faster than displaced workers can be retrained for new jobs, before automation takes the new jobs as well. This includes many job categories, such as in transportation, that were never threatened by automation before. According to a 2013 study, about 47% of US jobs are susceptible to automation. == Signatories ==

BitFunnel

BitFunnel is the search engine indexing algorithm and a set of components used in the Bing search engine, which were made open source in 2016. BitFunnel uses bit-sliced signatures instead of an inverted index in an attempt to reduce operations cost. == History == Progress on the implementation of BitFunnel was made public in early 2016, with the expectation that there would be a usable implementation later that year. In September 2016, the source code was made available via GitHub. A paper discussing the BitFunnel algorithm and implementation was released as through the Special Interest Group on Information Retrieval of the Association for Computing Machinery in 2017 and won the Best Paper Award. == Components == BitFunnel consists of three major components: BitFunnel – the text search/retrieval system itself WorkBench – a tool for preparing text for use in BitFunnel NativeJIT – a software component that takes expressions that use C data structures and transforms them into highly optimized assembly code == Algorithm == === Initial problem and solution overview === The BitFunnel paper describes the "matching problem", which occurs when an algorithm must identify documents through the usage of keywords. The goal of the problem is to identify a set of matches given a corpus to search and a query of keyword terms to match against. This problem is commonly solved through inverted indexes, where each searchable item is maintained with a map of keywords. In contrast, BitFunnel represents each searchable item through a signature. A signature is a sequence of bits which describe a Bloom filter of the searchable terms in a given searchable item. The bloom filter is constructed through hashing through several bit positions. === Theoretical implementation of bit-string signatures === The signature of a document (D) can be described as the logical-or of its term signatures: S D → = ⋃ t ∈ D S t → {\displaystyle {\overrightarrow {S_{D}}}=\bigcup _{t\in D}{\overrightarrow {S_{t}}}} Similarly, a query for a document (Q) can be defined as a union: S Q → = ⋃ t ∈ Q S t → {\displaystyle {\overrightarrow {S_{Q}}}=\bigcup _{t\in Q}{\overrightarrow {S_{t}}}} Additionally, a document D is a member of the set M' when the following condition is satisfied: S Q → ∩ S D → = S Q → {\displaystyle {\overrightarrow {S_{Q}}}\cap {\overrightarrow {S_{D}}}={\overrightarrow {S_{Q}}}} This knowledge is then combined to produce a formula where M' is identified by documents which match the query signature: M ′ = { D ∈ C ∣ S Q → ∩ S D → = S Q → } {\displaystyle M'=\left\{D\in C\mid {\overrightarrow {S_{Q}}}\cap {\overrightarrow {S_{D}}}={\overrightarrow {S_{Q}}}\right\}} These steps and their proofs are discussed in the 2017 paper. === Pseudocode for bit-string signatures === This algorithm is described in the 2017 paper. M ′ = ∅ foreach D ∈ C do if S D → ∩ S Q → = S Q → then M ′ = M ′ ∪ { D } endif endfor {\displaystyle {\begin{array}{l}M'=\emptyset \\{\texttt {foreach}}\ D\in C\ {\texttt {do}}\\\qquad {\texttt {if}}\ {\overrightarrow {S_{D}}}\cap {\overrightarrow {S_{Q}}}={\overrightarrow {S_{Q}}}\ {\texttt {then}}\\\qquad \qquad M'=M'\cup \{D\}\\\qquad {\texttt {endif}}\\{\texttt {endfor}}\end{array}}}