AI Assistant For Writing — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.
Cloud-to-Cloud Integration ( C2I ) allows users to connect disparate cloud computing platforms. While Paas (Platform as a service) and Saas (Software as a service) continue to gain momentum, different vendors have different implementations for cloud computing, e.g. Database, REST, SOAP API. Another name for Cloud-to-Cloud Integration is Cloud-Surfing. See also Cloud-based integration
Read more →
Pyramid, or pyramid representation, is a type of multi-scale signal representation developed by the computer vision, image processing and signal processing communities, in which a signal or an image is subject to repeated smoothing and subsampling. Pyramid representation is a predecessor to scale-space representation and multiresolution analysis. == Pyramid generation == There are two main types of pyramids: lowpass and bandpass. A lowpass pyramid is made by smoothing the image with an appropriate smoothing filter and then subsampling the smoothed image, usually by a factor of 2 along each coordinate direction. The resulting image is then subjected to the same procedure, and the cycle is repeated multiple times. Each cycle of this process results in a smaller image with increased smoothing, but with decreased spatial sampling density (that is, decreased image resolution). If illustrated graphically, the entire multi-scale representation will look like a pyramid, with the original image on the bottom and each cycle's resulting smaller image stacked one atop the other. A bandpass pyramid is made by forming the difference between images at adjacent levels in the pyramid and performing image interpolation between adjacent levels of resolution, to enable computation of pixelwise differences. == Pyramid generation kernels == A variety of different smoothing kernels have been proposed for generating pyramids. Among the suggestions that have been given, the binomial kernels arising from the binomial coefficients stand out as a particularly useful and theoretically well-founded class. Thus, given a two-dimensional image, we may apply the (normalized) binomial filter (1/4, 1/2, 1/4) typically twice or more along each spatial dimension and then subsample the image by a factor of two. This operation may then proceed as many times as desired, leading to a compact and efficient multi-scale representation. If motivated by specific requirements, intermediate scale levels may also be generated where the subsampling stage is sometimes left out, leading to an oversampled or hybrid pyramid. With the increasing computational efficiency of CPUs available today, it is in some situations also feasible to use wider supported Gaussian filters as smoothing kernels in the pyramid generation steps. === Gaussian pyramid === In a Gaussian pyramid, subsequent images are weighted down using a Gaussian average (Gaussian blur) and scaled down. Each pixel containing a local average corresponds to a neighborhood pixel on a lower level of the pyramid. This technique is used especially in texture synthesis. === Laplacian pyramid === A Laplacian pyramid is very similar to a Gaussian pyramid but saves the difference image of the blurred versions between each levels. Only the smallest level is not a difference image to enable reconstruction of the high resolution image using the difference images on higher levels. This technique can be used in image compression. === Steerable pyramid === A steerable pyramid, developed by Simoncelli and others, is an implementation of a multi-scale, multi-orientation band-pass filter bank used for applications including image compression, texture synthesis, and object recognition. It can be thought of as an orientation selective version of a Laplacian pyramid, in which a bank of steerable filters are used at each level of the pyramid instead of a single Laplacian or Gaussian filter. == Applications of pyramids == === Alternative representation === In the early days of computer vision, pyramids were used as the main type of multi-scale representation for computing multi-scale image features from real-world image data. More recent techniques include scale-space representation, which has been popular among some researchers due to its theoretical foundation, the ability to decouple the subsampling stage from the multi-scale representation, the more powerful tools for theoretical analysis as well as the ability to compute a representation at any desired scale, thus avoiding the algorithmic problems of relating image representations at different resolution. Nevertheless, pyramids are still frequently used for expressing computationally efficient approximations to scale-space representation. === Detail manipulation === Levels of a Laplacian pyramid can be added to or removed from the original image to amplify or reduce detail at different scales. However, detail manipulation of this form is known to produce halo artifacts in many cases, leading to the development of alternatives such as the bilateral filter. Some image compression file formats use the Adam7 algorithm or some other interlacing technique. These can be seen as a kind of image pyramid. Because those file format store the "large-scale" features first, and fine-grain details later in the file, a particular viewer displaying a small "thumbnail" or on a small screen can quickly download just enough of the image to display it in the available pixels—so one file can support many viewer resolutions, rather than having to store or generate a different file for each resolution.
Read more →
Automatic acquisition of lexicon is a computerized process used for the development of a complex morphological lexicon of a language. The lexicon is essential for the NLP (Natural language processing), as well as a prerequisite to any wide-coverage parser. The two main requirements represent raw corpus and the morphological description of the language. The aim is to provide lemmas that will serve to the explanation of all the words that occur within the corpus. For the achievement of a quality lexicon it is necessary to manually validate the generated lemmas and iterate the whole process several times. The process is focused on the open word classes (e.g. nouns, adjectives, verbs). Closed classes (e.g. prepositions, pronouns, numerals) are excluded. This method is applicable to the languages with a rich morphology, such as Slovak, Russian or Croatian. Applied to Slovak, being an inflectional language, the automatic acquisition focuses on the inflectional morphology as well as on the derivational morphology. This fact enables the users to find out the information about derivational relations (e.g. adjectivizations, prefixes) in the lexicon. For example, Slovak word korpusový is an adjectivization of korpus (eng. corpus). == Three-step loop == Conformably to Benoît Sagot, there are three stages involved in the acquisition of lemmas: Generation and inflection Ranking Manual validation The more iteration will be performed, the more accurate lexicon will be obtained. For each iteration are essential the information given by a manual validator. === Generation and inflection === Firstly, all words which represent the closed word classes (pronouns, prepositions, numerals) are manually excluded from the given corpus. Number of their occurrences in the corpus is provided. Then the automatic generation comes, when the hypothetical lemmas according to the morphological description of a language are created. Generated lemmas are consequently being inflected, so that all of their inflected forms are built. Obtained forms are associated with the corresponding lemma and a morphological tag. === Ranking === There was created a probabilistic model, represented by a fix-point algorithm, to rank the hypothetical lemmas generated in the first step. Best ranked lemmas are expected to be ideally all correct, whereas the least ranked tend to be incorrect. === Manual validation === Correctness of the best- ranked lemmas created in the previous step are checked by the manual validator, who should be a native speaker. Lemmas are at this stage divided into three categories: valid lemmas, appended to lexicon erroneous lemmas generated by valid forms (later associated to another lemmas) erroneous lemmas generated by invalid forms (these need to be excluded) == Future development == Automatic acquisition, in comparison to a purely manual development of the lexicons, seems to be promising, considering the future development, because of the short validation time needed and the relatively small amount of human labor involved.
Read more →
Talking Angela is a mobile game (formerly a chatbot), developed by Slovenian studio Outfit7 as part of the Talking Tom & Friends series. It was released on 13 November 2012 and December 2012 for iPhone, iPod and iPad, January 2013 for Android, and January 2014 for Google Play. The game's successor, the My Talking Angela game, was released in December 2014. The game takes place in a café in Paris and allows players to interact with Angela, an anthropomorphic white cat in different ways. Players can use coins to purchase makeup, accessories and items, as well as drinks that will trigger different visual effects. The fortune cookie button causes Angela to read out a fortune cookie, while the bird icon will prompt birds to fly around the screen, or have Angela feed them. Players can also pet or poke Angela, as well the café's sign. Prior to their removal, the game featured a chat system and a camera button. Users can engage in conversations with Angela, ask for quizzes or initiate a short snippet of the song "That's Falling In Love". If the player was to type in "Who is an idiot?", Angela would respond with a random swear word. Additionally, inquiring Angela about sexual topics would cause her to reply with "Do you want to talk about sex?", though she will quickly change the topic regardless of what the player writes next. A hoax claiming that Angela's eyes were hidden cameras that enabled hackers or paedophiles to watch children was spread. Despite the claims, Snopes and The Guardian found no evidence. Due to the hoax, Angela received a blue dress, as well as an altered eye asset with a different reflection, and later the chat and camera functions were removed altogether. == Hoaxes == In February 2014, Talking Angela was the subject of an Internet hoax alleging that the application was a front for child predators to exploit children. The rumor, which was widely circulated on Facebook and various websites claiming to be dedicated to parenting, claims that a sinister sexual predator or hacker, asked children for private personal information using the game's text-chat feature. Other versions of the rumour even attributed the disappearance of a child to the game; one news report claimed that a seven year old boy disappeared after downloading the app. Another variation included that it was run by a paedophile ring, citing a man that could be seen in Angela's eyes. The app's developers, Outfit7, later gave a statement refuting the hoaxes. The hoax was eventually debunked by Snopes, a fact-checking website. The site's owners, Barbara and David Mikkelson, reported that they had tried to "prompt" it to give responses asking for private information, but were unsuccessful, even when asking it explicitly sexual questions. While it is true that, in the game with child mode off, Angela does ask for the user's name, age and personal preferences to determine conversation topics, Outfit7 has said that this information is all "anonymized" and all personal information is removed from it. It is also impossible for a person to take control of what Angela says in the game, since the game is based on chatbot software. When the mode was turned on, the chat feature was disabled, meaning no personal questions could be asked. In 2015, the hoax was revived on Facebook, which prompted online security company Sophos and The Guardian to debunk it again. Sophos employee Paul Ducklin wrote that the message being posted on Facebook promoting the hoax was "close to 600 rambling, repetitious words, despite claiming at the start that it didn't have words to describe the situation. It's ill-written, and borders on being illiterate and incomprehensible." Bruce Wilcox, one of the game's programmers, attributed the hoax's popularity to the fact that the chatbot program in Talking Angela aimed to sound realistic. Concern was raised that the game's child mode may have been too easy for children to turn off. It allowed them to purchase "coins", premium currency in the game, via iTunes, and enabled the chat feature. While not "connecting your children to paedophiles", this still raised concerns according to The Guardian. === Impact === The scare significantly boosted the game's popularity, and was credited with helping the app enter the top 10 free iPhone apps soon after the hoax became widely known in February 2015,In the truth the reason there is a man in Angela’s eyes is because of pareidoila, the ability to see through diamonds and other minerals and water bodies and shiny objects,which is the reason why players notice a man in her eyes,The truth is that being Angela’s eyes simply serve as a reflective surface,Because of the low quality of this reflection the reflection was mistaken for a humanoid figure. oref>Smith, Josh (19 February 2014). "Talking Angela App Scare Skyrockets App to Top of Charts". GottaBeMobile.com. Archived from the original on 2 April 2016. Retrieved 10 May 2014. and third most popular for all iPhone apps at the start of the following month. In 2016, Outfit7 removed the chat feature along with the camera function from the app due to this controversy, though this decision was met with criticism.
Read more →
iubenda (stylized in lowercase; Italian pronunciation: [juˈbɛnda]) is an Italian software company that develops tools intended to support website and application compliance with data protection and privacy regulations, including consent management platforms. The company was founded in 2011 in Milan by Andrea Giannangelo. In February 2022, the company was acquired by team.blue. == History == iubenda was founded in 2011 in Milan, Italy, initially focusing on automated privacy policy generation. In 2015, the company expanded its services to include cookie compliance tools following the implementation of ePrivacy regulations in Italy. In 2018, following the introduction of the General Data Protection Regulation (GDPR) in the European Union, iubenda expanded its products to include consent management and compliance documentation services. In February 2022, iubenda was acquired by team.blue, which obtained a majority stake in the company. Italian media described the acquisition as one of the largest Italian technology startup exits in recent years. In October 2022, iubenda acquired consentmanager, a Sweden-based consent management provider. In 2025, the company acquired CookieFirst, a Netherlands-based consent management platform. In 2025, iubenda partnered with AccessiWay, a digital accessibility company owned by team.blue. == Activities == iubenda develops software tools intended to support compliance with data protection and privacy regulations. Its products include generators for privacy policies, cookie banners, terms and conditions documents, and consent management platforms. The company’s consent management platform integrates with frameworks used for online advertising and privacy compliance, including Google's Consent Mode. The platform is designed to support compliance with regulatory frameworks including the GDPR in the European Union, the UK GDPR, Brazil’s LGPD, Switzerland’s FADP and privacy laws in the United States. Its tools can be integrated with content management systems, web applications, and other digital platforms, including WordPress. The company operates internationally, with a customer base of more than 150,000 organisations, primarily in Europe and the Americas.
Read more →
Luminoso is a Cambridge, MA-based text analytics and artificial intelligence company. It spun out of the MIT Media Lab and its crowd-sourced Open Mind Common Sense (OMCS) project. The company has raised $20.6 million in financing, and its clients include Sony, Autodesk, Scotts Miracle-Gro, and GlaxoSmithKline. == History == Luminoso was co-founded in 2010 by Dennis Clark, Jason Alonso, Robyn Speer, and Catherine Havasi, a research scientist at MIT in artificial intelligence and computational linguistics. The company builds on the knowledge base of MIT’s Open Mind Common Sense (OMCS) project, co-founded in 1999 by Havasi, who continues to serve as its director. The OCMS knowledge base has since been combined with knowledge from other crowdsourced resources to become ConceptNet. ConceptNet consists of approximately 28 million statements in 304 languages, with full support for 10 languages and moderate support for 77 languages. ConceptNet is a resource for making an AI that understands the meanings of the words people use. During the World Cup in June 2014, the company provided a widely reported real-time sentiment analysis of the U.S. vs. Germany match, analyzing 900,000 posts on Twitter, Facebook and Google+. == Applications == The company uses artificial intelligence, natural language processing, and machine learning to derive insights from unstructured data such as contact center interactions, chatbot and live chat transcripts, product reviews, open-ended survey responses, and email. Luminoso's software identifies and quantifies patterns and relationships in text-based data, including domain-specific or creative language. Rather than human-powered keyword searches of data, the software automates taxonomy creation around concepts, allowing related words and phrases to be dynamically generated and tracked. Commercial applications include analyzing, prioritizing, and routing contact center interactions; identifying consumer complaints before they begin to trend; and tracking sentiment during product launches. The software natively analyzes text in fourteen languages, as well as emoji. == Products == Luminoso's technology can be accessed via two products: Luminoso Daylight and Luminoso Compass. Luminoso Daylight enables a deep-dive analysis into batch or real-time data, whereas Luminoso Compass automates the categorization of real-time data. Both products offer a user interface as well as an API. Luminoso's products can be implemented through either a cloud-based or an on-premise solution. == Research == Luminoso continues to actively conduct research in natural language processing and word embeddings and regularly participates in evaluations such as SemEval. At SemEval 2017, Luminoso participated in Task 2, measuring the semantic similarity of word pairs within and across five languages. Its solution outperformed all competing systems in every language pair tested, with the exception of Persian. == Recognition == Luminoso has been listed as a "Cool Vendor in AI for Marketing" by Gartner, and has also been named a "Boston Artificial Intelligence Startup to Watch" by BostInno. In May 2017, Luminoso was recognized as having the Best Application for AI in the Enterprise by AI Business, and was also shortlisted as the Best AI Breakthrough and Best Innovation in NLP. == Competitors == Major competitors include Clarabridge and Lexalytics. == Investors == The company raised $1.5 million from angel investors led by Basis Technology in 2012. Its first institutional funding round of $6.5 was completed in July 2014, led by Acadia Woods with participation from Japan’s Digital Garage. The company followed that with a $10M series B funding round in December 2018, led by DVI Equity Partners, with participation from Liberty Global Ventures, DF Enterprises, Raptor Holdco, Acadia Woods Partners, and Accord Ventures, among others.
Read more →
Automatic pronunciation assessment uses computer speech recognition to determine how accurately speech has been pronounced, instead of relying on a human instructor or proctor. It is also called speech verification, pronunciation evaluation, and pronunciation scoring. This technology is used to grade speech quality, for language testing, for computer-aided pronunciation teaching (CAPT) in computer-assisted language learning (CALL), for speaking skill remediation, and for accent reduction. Pronunciation assessment is different from dictation or automatic transcription, because instead of determining unknown speech, it verifies learners' pronunciation of known word(s), often from prior transcription of the same utterance; ideally scoring the intelligibility of the learners' speech. Sometimes pronunciation assessment evaluates the prosody of the learners' speech, such as intonation, pitch, tempo, rhythm, and syllable and word stress, although those are usually not essential for being understood in most languages. Pronunciation assessment is also used in reading tutoring, for example in products from Google, Microsoft, and Amira Learning. Automatic pronunciation assessment can also be used to help diagnose and treat speech disorders such as apraxia. == Intelligibility == Intelligibility refers to how well a learner's utterance is understood by a listener, rather than how much it sounds like a native speaker. This is separate from measures of fluency, such as so-called "Goodness of Pronunciation" (GoP) scores, which estimate how closely an utterance aligns with those of native speakers. Intelligibility is widely regarded as the most important communicative goal in pronunciation teaching and assessment. For example, in the Common European Framework of Reference for Languages (CEFR) assessment criteria for "overall phonological control", intelligibility outweighs formally correct pronunciation at all levels. Studies in applied linguistics have shown that accent reduction does not always increase intelligibility because listeners can often comprehend heavily accented speech without difficulty. Pronunciation assessment systems often rely on acoustic methods such as GoP which compare learner speech to reference models to produce phoneme-level scores, which are in turn aggregated to produce word and phrase scores. While these methods are effective for identifying deviations from native speakers' utterances, they do not effectively measure how understandable speech is to human listeners. Intelligibility is influenced by broader linguistic and contextual factors such as stress placement, speech rate, and coarticulation, which are not represented in purely segmental scores. The earliest work on pronunciation assessment avoided measuring genuine listener intelligibility, a shortcoming corrected in 2011 at the Toyohashi University of Technology, and included in the Versant high-stakes English fluency assessment from Pearson and mobile apps from 17zuoye Education & Technology, but still missing in 2023 products from Google Search, Microsoft, Educational Testing Service, Speechace, and ELSA. Assessing authentic listener intelligibility is essential for avoiding inaccuracies from accent bias, especially in high-stakes assessments; from words with multiple correct pronunciations; and from phoneme coding errors in machine-readable pronunciation dictionaries. In 2022, researchers found that some newer speech-to-text systems, based on end-to-end reinforcement learning to map audio signals directly into words, produce word and phrase confidence scores (from 10-25ms audio frame logit aggregation) closely correlated with genuine listener intelligibility. Others have been able to assess intelligibility using Levenshtein or dynamic time warping distance measures from Wav2Vec2 representation of good speech. Further work through 2025 has focused specifically on measuring intelligibility. A 2025 study of 42 pronunciation and speech coaching apps (32 mobile and 10 web) found that none offered intelligibility assessment. Instead, most provided only segmental and accent-focused scoring. About two-thirds of the apps provided some form of specific pronunciation feedback, usually with phonetic transcriptions, but accompanied by visual cues (such as animations of the vocal tract or the lips and tongue from the front) in only about 5% of the apps. Less than a third provided feedback on learner perception of exemplar speech. == Evaluation == Although there are as yet no industry-standard benchmarks for evaluating pronunciation assessment accuracy, researchers occasionally release evaluation speech corpuses for others to use for improving assessment quality. Such evaluation databases often emphasize formally unaccented pronunciation to the exclusion of genuine intelligibility evident from blinded listener transcriptions. As of mid-2025, state of the art approaches for automatically transcribing phonemes typically achieve an error rate of about 10% from known good speech. The International Speech Communication Association (ISCA) 2025 Workshop on Speech and Language Technology in Education (SLaTE) administered a Speak & Improve Challenge: Spoken Language Assessment and Feedback, introducing benchmarks for evaluating pronunciation assessment and remediation systems across languages, accents, and learner populations. The challenge emphasized cross-lingual generalization and alignment with human intelligibility judgments, for more robust and interpretable assessment systems. Ethical issues in pronunciation assessment are present in both human and automatic methods. Authentic validity, fairness, and mitigating bias in evaluation are all crucial. Diverse speech data should be included in automatic pronunciation assessment models. Combining human judgments, especially blinded transcriptions from a wide diversity of listeners, with automated feedback can improve accuracy and fairness. Second language learners benefit substantially from their use of widely available speech recognition systems for dictation, virtual assistants, and AI chatbots. In such systems, users naturally try to correct their own errors evident in speech recognition results that they notice. Such use improves their grammar and vocabulary development along with their pronunciation skills. The extent to which explicit pronunciation assessment and remediation approaches improve on such self-directed interactions remains an open question. Similarly, automatic dictation results have been shown to reflect intelligibility about as well as human scorers. == Recent developments == During 2021–22, a smartphone-based CAPT system was used to sense articulation through both audible and inaudible signals, providing feedback at the phoneme level. Some promising areas for improvement which were being developed in 2024 include articulatory feature extraction and transfer learning to suppress unnecessary corrections. Other interesting advances under development include "augmented reality" interfaces for mobile devices using optical character recognition to provide pronunciation training on text found in user environments. In 2024, audio multimodal large language models were first described as assessing pronunciation. That work has been carried forward by other researchers in 2025 who report positive results. Subsequently, researchers demonstrated pronunciation scoring by providing a language model with textual descriptions of speech, including the speech-to-text transcript, phoneme sequences, pauses, and phoneme sequence matching; this approach can achieve performance similar to multimodal LLMs that analyze raw audio while avoiding their higher computational cost. In 2025, the Duolingo English Test authors published a description of their pronunciation assessment method, purportedly built to measure intelligibility rather than accent imitation. While achieving a correlation of 0.82 with expert human ratings, very close to inter-rater agreement and outperforming alternative methods, the method is nonetheless based on experts' scores along the six-point CEFR common reference levels scale, instead of actual blinded listener transcriptions. Further promising work in 2025 includes assessment feedback aligning learner speech to synthetic utterances using interpretable features, identifying continuous spans of words for remediation feedback; synthesizing corrected speech matching learners' self-perceived voices, which they prefer and imitate more accurately as corrections; and streaming such interactions. On January 21, 2026, Educational Testing Service's TOEFL iBT high-stakes English language test, required by US university admissions and employers from English as a foreign language applicants more often than all other internet-based tests combined, changed its speaking assessments. While official rubrics claim that the new scoring will be based primarily on intelligibility, the new test's technical description indicates that it ju
Read more →
Brave Leo is a large language model-based chatbot developed by Brave Software and included with the Brave browser. == History == In November 2023, the company said versions for iOS and Android would be available "in the coming months". == Features == Since January 2024, Leo has used the open-source Mixtral 8x7B from Mistral AI as its default large language model, in addition to LLaMA 2 from Meta Platforms and Claude from Anthropic, both of which have been used previously. Leo can suggest follow-up questions, and summarize webpages, PDFs, and videos. Leo has a $15 (US) per month premium version that enables more requests and uses larger LLMs. == Privacy == The answers given by Leo are not saved. Brave uses the slogan Love Privacy to emphasize its focus on user privacy and data protection. The phrase has been featured in Brave's official marketing campaigns and has been cited in media coverage of the browser's privacy-first approach. == Controversies == In 2023, PC World reported that Leo evades questions about US elections.
Read more →
VACUUM is a set of normative guidance principles for achieving training and test dataset quality for structured datasets in data science and machine learning. The garbage-in, garbage out principle motivates a solution to the problem of data quality but does not offer a specific solution. Unlike the majority of the ad-hoc data quality assessment metrics often used by practitioners VACUUM specifies qualitative principles for data quality management and serves as a basis for defining more detailed quantitative metrics of data quality. VACUUM is an acronym that stands for: valid accurate consistent uniform unified model
Read more →
Natural language processing (NLP) is the processing of natural language information by a computer. NLP is a subfield of computer science and is closely associated with artificial intelligence. NLP is also related to information retrieval, knowledge representation, computational linguistics, and linguistics more broadly. Major processing tasks in an NLP system include: speech recognition, text classification, natural language understanding, and natural language generation. == History == Natural language processing has its roots in the 1950s. Already in 1950, Alan Turing published an article titled "Computing Machinery and Intelligence," which proposed what is now called the Turing test as a criterion of intelligence, though at the time that was not articulated as a problem separate from artificial intelligence. The proposed test includes a task that involves the automated interpretation and generation of natural language. === Symbolic NLP (1950s – early 1990s) === The premise of symbolic NLP is often illustrated using John Searle's Chinese room thought experiment: Given a collection of rules (e.g., a Chinese phrasebook, with questions and matching answers), the computer emulates natural language understanding (or other NLP tasks) by applying those rules to the data it confronts. 1950s: The Georgetown experiment in 1954 involved fully automatic translation of more than sixty Russian sentences into English. The authors claimed that within three or five years, machine translation would be a solved problem. However, real progress was much slower, and after the ALPAC report in 1966, which found that ten years of research had failed to fulfill the expectations, funding for machine translation was dramatically reduced. Little further research in machine translation was conducted in America (though some research continued elsewhere, such as Japan and Europe) until the late 1980s when the first statistical machine translation systems were developed. 1960s: Some notably successful natural language processing systems developed in the 1960s were SHRDLU, a natural language system working in restricted "blocks worlds" with restricted vocabularies, and ELIZA, a simulation of Rogerian psychotherapy, written by Joseph Weizenbaum between 1964 and 1966. Despite using minimal information about human thought or emotion, ELIZA was able to produce interactions that appeared human-like. When the "patient" exceeded the very small knowledge base, ELIZA might provide a generic response, for example, responding to "My head hurts" with "Why do you say your head hurts?". Ross Quillian's successful work on natural language was demonstrated with a vocabulary of only twenty words, because that was all that would fit in a computer memory at the time. 1970s: During the 1970s, many programmers began to write "conceptual ontologies", which structured real-world information into computer-understandable data. Examples are MARGIE (Schank, 1975), SAM (Cullingford, 1978), PAM (Wilensky, 1978), TaleSpin (Meehan, 1976), QUALM (Lehnert, 1977), Politics (Carbonell, 1979), and Plot Units (Lehnert 1981). During this time, the first chatterbots were written (e.g., PARRY). 1980s: The 1980s and early 1990s mark the heyday of symbolic methods in NLP. Focus areas of the time included research on rule-based parsing (e.g., the development of HPSG as a computational operationalization of generative grammar), morphology (e.g., two-level morphology), semantics (e.g., Lesk algorithm), reference (e.g., within Centering Theory) and other areas of natural language understanding (e.g., in the Rhetorical Structure Theory). Other lines of research were continued, e.g., the development of chatterbots with Racter and Jabberwacky. An important development (that eventually led to the statistical turn in the 1990s) was the rising importance of quantitative evaluation in this period. === Statistical NLP (1990s–present) === Up until the 1980s, most natural language processing systems were based on complex sets of hand-written rules. Starting in the late 1980s, however, there was a revolution in natural language processing with the introduction of machine learning algorithms for language processing. This shift was influenced by increasing computational power (see Moore's law) and a decline in the dominance of Chomskyan linguistic theories (e.g. transformational grammar), whose theoretical underpinnings discouraged the sort of corpus linguistics that underlies the machine-learning approach to language processing. 1990s: Many of the notable early successes in statistical methods in NLP occurred in the field of machine translation, due especially to work at IBM Research, such as IBM alignment models. These systems were able to take advantage of existing multilingual textual corpora that had been produced by the Parliament of Canada and the European Union as a result of laws calling for the translation of all governmental proceedings into all official languages of the corresponding systems of government. However, many systems relied on corpora that were specifically developed for the tasks they were designed to perform. This reliance has been a major limitation to their broader effectiveness and continues to affect similar systems. Consequently, significant research has focused on methods for learning effectively from limited amounts of data. 2000s: With the growth of the web, increasing amounts of raw (unannotated) language data have become available since the mid-1990s. Research has thus increasingly focused on unsupervised and semi-supervised learning algorithms. Such algorithms can learn from data that has not been hand-annotated with the desired answers or using a combination of annotated and non-annotated data. Generally, this task is much more difficult than supervised learning, and typically produces less accurate results for a given amount of input data. However, large quantities of non-annotated data are available (including, among other things, the entire content of the World Wide Web), which can often make up for the worse efficiency if the algorithm used has a low enough time complexity to be practical. 2003: word n-gram model, at the time the best statistical algorithm, is outperformed by a multi-layer perceptron (with a single hidden layer and context length of several words, trained on up to 14 million words, by Bengio et al.) 2010: Tomáš Mikolov (then a PhD student at Brno University of Technology) with co-authors applied a simple recurrent neural network with a single hidden layer to language modeling, and in the following years he went on to develop Word2vec. In the 2010s, representation learning and deep neural network-style (featuring many hidden layers) machine learning methods became widespread in natural language processing. This shift gained momentum due to results showing that such techniques can achieve state-of-the-art results in many natural language tasks, e.g., in language modeling and parsing. This is increasingly important in medicine and healthcare, where NLP helps analyze notes and text in electronic health records that would otherwise be inaccessible for study when seeking to improve care or protect patient privacy. == Approaches: Symbolic, statistical, neural networks == Symbolic approach, i.e., the hand-coding of a set of rules for manipulating symbols, coupled with a dictionary lookup, was historically the first approach used both by AI in general and by NLP in particular: such as by writing grammars or devising heuristic rules for stemming. Machine learning approaches, which include both statistical and neural networks, on the other hand, have many advantages over the symbolic approach: both statistical and neural network methods tend to focus more on the most common cases extracted from a corpus of texts, whereas the rule-based approach needs to provide rules for both rare and common cases equally. language models, produced by either statistical or neural networks methods, are more robust to both unfamiliar (e.g. containing words or structures that have not been seen before) and erroneous input (e.g. with misspelled words or words accidentally omitted) in comparison to the rule-based systems, which are also more costly to produce. the larger such a (probabilistic) language model is, the more accurate it becomes, in contrast to rule-based systems that can gain accuracy only by increasing the amount and complexity of the rules leading to intractability problems. Rule-based systems are commonly used: when the amount of training data is insufficient to successfully apply machine learning methods, e.g., for the machine translation of low-resource languages such as provided by the Apertium system, for preprocessing in NLP pipelines, e.g., tokenization, or for post-processing and transforming the output of NLP pipelines, e.g., for knowledge extraction from syntactic parses. === Statistical approach === In the late 1980s and mid-1990s, the statistical approach ended a peri
Read more →
A recursive transition network ("RTN") is a graph theoretical schematic used to represent the rules of a context-free grammar. RTNs have application to programming languages, natural language and lexical analysis. Any sentence that is constructed according to the rules of an RTN is said to be "well-formed". The structural elements of a well-formed sentence may also be well-formed sentences by themselves, or they may be simpler structures. This is why RTNs are described as recursive. == Notes and references ==
Read more →
Augmented Analytics is an approach of data analytics that employs the use of machine learning and natural language processing to automate analysis processes normally done by a specialist or data scientist. The term was introduced in 2017 by Rita Sallam, Cindi Howson, and Carlie Idoine in a Gartner research paper. Augmented analytics is based on business intelligence and analytics. In the graph extraction step, data from different sources are investigated. == Defining Augmented Analytics == Machine Learning – a systematic computing method that uses algorithms to sift through data to identify relationships, trends, and patterns. It is a process that allows algorithms to dynamically learn from data instead of having a set base of programmed rules. Natural language generation (NLG) – a software capability that takes unstructured data and translates it into plain-English, readable, language. Automating Insights – using machine learning algorithms to automate data analysis processes. Natural Language Query – enabling users to query data using business terms that are either typed onto a search box or spoken. == Data Democratization == Data Democratization is the democratizing data access in order to relieve data congestion and get rid of any sense of data "gatekeepers". This process must be implemented alongside a method for users to make sense of the data. This process is used in hopes of speeding up company decision making and uncovering opportunities hidden in data. There are three aspects to democratising data: Data Parameterisation and Characterisation. Data Decentralisation using an OS of blockchain and DLT technologies, as well as an independently governed secure data exchange to enable trust. Consent Market-driven Data Monetisation. When it comes to connecting assets, there are two features that will accelerate the adoption and usage of data democratisation: decentralized identity management and business data object monetization of data ownership. It enables multiple individuals and organizations to identify, authenticate, and authorize participants and organizations, enabling them to access services, data or systems across multiple networks, organizations, environments, and use cases. It empowers users and enables a personalized, self-service digital onboarding system so that users can self-authenticate without relying on a central administration function to process their information. Simultaneously, decentralized identity management ensures the user is authorized to perform actions subject to the system’s policies based on their attributes (role, department, organization, etc.) and/ or physical location. == Use cases == Agriculture – Farmers collect data on water use, soil temperature, moisture content and crop growth, augmented analytics can be used to make sense of this data and possibly identify insights that the user can then use to make business decisions. Smart Cities – Many cities across the United States, known as Smart Cities collect large amounts of data on a daily basis. Augmented analytics can be used to simplify this data in order to increase effectiveness in city management (transportation, natural disasters, etc.). Analytic Dashboards – Augmented analytics has the ability to take large data sets and create highly interactive and informative analytical dashboards that assist in many organizational decisions. Augmented Data Discovery – Using an augmented analytics process can assist organizations in automatically finding, visualizing and narrating potentially important data correlations and trends. Data Preparation – Augmented analytics platforms have the ability to take large amounts of data and organize and "clean" the data in order for it to be usable for future analyses. Business – Businesses collect large amounts of data, daily. Some examples of types of data collected in business operations include; sales data, consumer behavior data, distribution data. An augmented analytics platform provides access to analysis of this data, which could be used in making business decisions.
Read more →
Knowledge as a service (KaaS) is a computing service that delivers information to users, backed by a knowledge model, which might be drawn from a number of possible models based on decision trees, association rules, or neural networks. A knowledge as a service provider responds to knowledge requests from users through a centralised knowledge server, and provides an interface between users and data owners. KaaS is one of several cloud computing-dependent business models in which computer resources are sold on an on-demand and pay-as-you-use basis. == Overview == At the International Semantic Web Conference 2019, it was described how knowledge can be made live and evolve on the web allowing users to learn directly from elaborated knowledge, now appearing in the form of knowledge graphs. KaaS appear when knowledge graphs are accessed via services This is opposed to DaaS which might "compute large volumes of data; integrate and analyzes that data; and publish it in real-time, using Web service APIs" (from Data as a Service) where the KaaS is able to exploit context - both the context of the user in relation to their information requests of the KaaS (where and when they make the request) and also the context of the information in relation to some objective or purpose of the users either understood by the KaaS automatically or indicated to it by the user. == Differentiating knowledge from data == Conceptual models that make such a differentiation such as the so-called DIKW pyramid have existed for perhaps more than 40 years (see a 1974 journal article about this) however definitions are not stable and universally accepted (see the discussion about the conceptualizations of DIKW within the DIKW Wikipedia article that question value of wisdom). The knowledge component of DIKW is generally agreed to be an elusive concept which is difficult to define, however Rowley 2007, in a well known student textbook differentiated knowledge from data by stating that knowledge is "defined with reference to information" and that it contains more than just facts but also "beliefs and expectations". In relation to knowledge graphs, knowledge may be additional content they provide over and above pure data which is the definition of the categories, properties and relations between the concepts, data and entities that substantiate one, many or all domains of discourse (see the definition of Ontology). The ability to represent "beliefs and expectations", or other forms of not so straightforwardly explicit knowledge is an on-going area of improvement in information sciences (see Tacit knowledge) and, with relation to KaaS, the establishment of recent informatics mechanics to do so it critical to the legitimacy of KaaS as it is differentiated from just value-added DaaS. Knowledge graphs' ability to represent context via the definition of the categories, properties and relations between the concepts, data and entities that substantiate one, many or all domains of discourse that they provide (see the definition of Ontology) has led to the idea that supplying access to KNs might be a required competency of a KaaS. == Delivery of knowledge == Much service-delivered content is dependent on a session to provide much of the context that the user (client) needs to understand answers to questions. For example, using current HTTP internet protocols, a GET request to retrieve information identified by a URI, such as a web page, a client (a human or a machine) may have access information supplied automatically to enable that client to bypass paywalls or other content access controls. Such context, in this case about the client's information access allowances, can alter the information provided. In a logical extension to this internet protocols example, a server would receive from the client, either manually or automatically, a full context which would be information about the situation the client is in and this would allow the server to best interpret the client's request. Current internet protocols allow for formats, languages and related preferences to be expressed by clients but make no mention of what a client already knows and what they may understand. The recent Content Negotiation by Profile proposes additions to both the HTTP internet protocols and related services that allow clients to also request information - a response from the server - that accords with an identified information model. This then allows clients to indicate not just formats and languages that they understand (technically that they prefer) but also domains of discourse that that do, which is a step towards comprehensive client context provision.
Read more →
In deep learning, weight initialization or parameter initialization describes the initial step in creating a neural network. A neural network contains trainable parameters that are modified during training: weight initialization is the pre-training step of assigning initial values to these parameters. The choice of weight initialization method affects the speed of convergence, the scale of neural activation within the network, the scale of gradient signals during backpropagation, and the quality of the final model. Proper initialization is necessary for avoiding issues such as vanishing and exploding gradients and activation function saturation. Note that even though this article is titled "weight initialization", both weights and biases are used in a neural network as trainable parameters, so this article describes how both of these are initialized. Similarly, trainable parameters in convolutional neural networks (CNNs) are called kernels and biases, and this article also describes these. == Constant initialization == We discuss the main methods of initialization in the context of a multilayer perceptron (MLP). Specific strategies for initializing other network architectures are discussed in later sections. For an MLP, there are only two kinds of trainable parameters, called weights and biases. Each layer l {\displaystyle l} contains a weight matrix W ( l ) ∈ R n l − 1 × n l {\displaystyle W^{(l)}\in \mathbb {R} ^{n_{l-1}\times n_{l}}} and a bias vector b ( l ) ∈ R n l {\displaystyle b^{(l)}\in \mathbb {R} ^{n_{l}}} , where n l {\displaystyle n_{l}} is the number of neurons in that layer. A weight initialization method is an algorithm for setting the initial values for W ( l ) , b ( l ) {\displaystyle W^{(l)},b^{(l)}} for each layer l {\displaystyle l} . The simplest form is zero initialization: W ( l ) = 0 , b ( l ) = 0 {\displaystyle W^{(l)}=0,b^{(l)}=0} Zero initialization is usually used for initializing biases, but it is not used for initializing weights, as it leads to symmetry in the network, causing all neurons to learn the same features. In this page, we assume b = 0 {\displaystyle b=0} unless otherwise stated. Recurrent neural networks typically use activation functions with bounded range, such as sigmoid and tanh, since unbounded activation may cause exploding values. (Le, Jaitly, Hinton, 2015) suggested initializing weights in the recurrent parts of the network to identity and zero bias, similar to the idea of residual connections and LSTM with no forget gate. In most cases, the biases are initialized to zero, though some situations can use a nonzero initialization. For example, in multiplicative units, such as the forget gate of LSTM, the bias can be initialized to 1 to allow good gradient signal through the gate. For neurons with ReLU activation, one can initialize the bias to a small positive value like 0.1, so that the gradient is likely nonzero at initialization, avoiding the dying ReLU problem. == Random initialization == Random initialization means sampling the weights from a normal distribution or a uniform distribution, usually independently. === LeCun initialization === LeCun initialization, popularized in (LeCun et al., 1998), is designed to preserve the variance of neural activations during the forward pass. It samples each entry in W ( l ) {\displaystyle W^{(l)}} independently from a distribution with mean 0 and variance 1 / n l − 1 {\displaystyle 1/n_{l-1}} . For example, if the distribution is a continuous uniform distribution, then the distribution is U ( ± 3 / n l − 1 ) {\displaystyle {\mathcal {U}}(\pm {\sqrt {3/n_{l-1}}})} . === Glorot initialization === Glorot initialization (or Xavier initialization) was proposed by Xavier Glorot and Yoshua Bengio. It was designed as a compromise between two goals: to preserve activation variance during the forward pass and to preserve gradient variance during the backward pass. For uniform initialization, it samples each entry in W ( l ) {\displaystyle W^{(l)}} independently and identically from U ( ± 6 / ( n l + 1 + n l − 1 ) ) {\displaystyle {\mathcal {U}}(\pm {\sqrt {6/(n_{l+1}+n_{l-1})}})} . In the context, n l − 1 {\displaystyle n_{l-1}} is also called the "fan-in", and n l + 1 {\displaystyle n_{l+1}} the "fan-out". When the fan-in and fan-out are equal, then Glorot initialization is the same as LeCun initialization. === He initialization === As Glorot initialization performs poorly for ReLU activation, He initialization (or Kaiming initialization) was proposed by Kaiming He et al. for networks with ReLU activation. It samples each entry in W ( l ) {\displaystyle W^{(l)}} from N ( 0 , 2 / n l − 1 ) {\displaystyle {\mathcal {N}}(0,2/n_{l-1})} . === Orthogonal initialization === (Saxe et al. 2013) proposed orthogonal initialization: initializing weight matrices as uniformly random (according to the Haar measure) semi-orthogonal matrices, multiplied by a factor that depends on the activation function of the layer. It was designed so that if one initializes a deep linear network this way, then its training time until convergence is independent of depth. Sampling a uniformly random semi-orthogonal matrix can be done by initializing X {\displaystyle X} by IID sampling its entries from a standard normal distribution, then calculate ( X X ⊤ ) − 1 / 2 X {\displaystyle \left(XX^{\top }\right)^{-1/2}X} or its transpose, depending on whether X {\displaystyle X} is tall or wide. For CNN kernels with odd widths and heights, orthogonal initialization is done this way: initialize the central point by a semi-orthogonal matrix, and fill the other entries with zero. As an illustration, a kernel K {\displaystyle K} of shape 3 × 3 × c × c ′ {\displaystyle 3\times 3\times c\times c'} is initialized by filling K [ 2 , 2 , : , : ] {\displaystyle K[2,2,:,:]} with the entries of a random semi-orthogonal matrix of shape c × c ′ {\displaystyle c\times c'} , and the other entries with zero. (Balduzzi et al., 2017) used it with stride 1 and zero-padding. This is sometimes called the Orthogonal Delta initialization. Related to this approach, unitary initialization proposes to parameterize the weight matrices to be unitary matrices, with the result that at initialization they are random unitary matrices (and throughout training, they remain unitary). This is found to improve long-sequence modelling in LSTM. Orthogonal initialization has been generalized to layer-sequential unit-variance (LSUV) initialization. It is a data-dependent initialization method, and can be used in convolutional neural networks. It first initializes weights of each convolution or fully connected layer with orthonormal matrices. Then, proceeding from the first to the last layer, it runs a forward pass on a random minibatch, and divides the layer's weights by the standard deviation of its output, so that its output has variance approximately 1. === Fixup initialization === In 2015, the introduction of residual connections allowed very deep neural networks to be trained, much deeper than the ~20 layers of the previous state of the art (such as the VGG-19). Residual connections gave rise to their own weight initialization problems and strategies. These are sometimes called "normalization-free" methods, since using residual connection could stabilize the training of a deep neural network so much that normalizations become unnecessary. Fixup initialization is designed specifically for networks with residual connections and without batch normalization, as follows: Initialize the classification layer and the last layer of each residual branch to 0. Initialize every other layer using a standard method (such as He initialization), and scale only the weight layers inside residual branches by L − 1 2 m − 2 {\displaystyle L^{-{\frac {1}{2m-2}}}} . Add a scalar multiplier (initialized at 1) in every branch and a scalar bias (initialized at 0) before each convolution, linear, and element-wise activation layer. Similarly, T-Fixup initialization is designed for Transformers without layer normalization. === Others === Instead of initializing all weights with random values on the order of O ( 1 / n ) {\displaystyle O(1/{\sqrt {n}})} , sparse initialization initialized only a small subset of the weights with larger random values, and the other weights zero, so that the total variance is still on the order of O ( 1 ) {\displaystyle O(1)} . Random walk initialization was designed for MLP so that during backpropagation, the L2 norm of gradient at each layer performs an unbiased random walk as one moves from the last layer to the first. Looks linear initialization was designed to allow the neural network to behave like a deep linear network at initialization, since W R e L U ( x ) − W R e L U ( − x ) = W x {\displaystyle W\;\mathrm {ReLU} (x)-W\;\mathrm {ReLU} (-x)=Wx} . It initializes a matrix W {\displaystyle W} of shape R n 2 × m {\displaystyle \mathbb {R} ^{{\frac {n}{2}}\times m}} by any method, such as orthogonal initialization, t
Read more →
Absher (Arabic: أبشر ‘Absher, roughly meaning "good tidings" or "yes, done") is a smartphone application and web portal which allows citizens and residents of Saudi Arabia to use a variety of governmental services. Amongst several other services with the Absher app, it can be used to apply for jobs and Hajj permits, passport info can be updated, and electronic crimes can be reported. The application provides around 280 services for residents of Saudi Arabia including but not limited to making appointments, renewing passports, residents' cards, IDs, driver's licenses and others, and, controversially, enables Saudi men to track the whereabouts of women they control as part of the country's male guardianship system. The app can be downloaded from the Google Play Store and Apple App Store and is supplied by the Saudi Interior Ministry. According to the Ministry of the Interior, Absher has more than 20 million users. As of February 2019, Absher has been downloaded 4.2 million times from the App Store. Some services provided through Absher can also be accessed through the website absher.sa. In March 2021, Saudi Arabia launched the digital version of the Absher for individuals app through which the users can download a copy of their digital ID. Then, new services were added to the platform such as online birth and death registration services, requesting amendments to academic credentials, correcting names in English and marital status and requesting civil records of children. == Impact on women's rights == The app has been criticized by various human rights activists, human rights organisations and international communities. The US and European countries have also condemned the app and urged the kingdom to end its male guardianship system. Absher gained media attention in 2019 for its functions supporting the Saudi policy of male guardianship following an investigation by Business Insider. The app allows for designated guardians to receive notifications if a woman under their guardianship passes through an airport and subsequently gives them the option to withdraw her right to travel. In a few cases, this system has been circumvented by women who have been able to gain control over its settings and use it to allow themselves to travel. US Senator Ron Wyden of Oregon wrote a letter to the CEO's of Apple and Google, criticizing the app and demanding for its removal immediately. Wyden said "American companies should not enable or facilitate the Saudi government's patriarchy," and called the Saudi system of control over women "abhorrent". According to the EU lawmakers, current rules imposed over the women by the Saudi government make women “second-class citizens”. The lawmakers also asked the EU states to continue to build pressure on Riyadh so as to improve the conditions of women and human rights. Amnesty International and Human Rights Watch accused Apple and Google of helping "enforce gender apartheid" by hosting the app. US congresswomen Rep. Katherine Clark and Rep. Carolyn B. Maloney condemned the kingdom's male guardianship system that reflected from the app, calling Absher a "patriarchal weapon" and asking for its removal. In response to the criticism received by Absher, Apple chief executive officer Tim Cook stated in February 2019 that he intended to investigate the situation. Similarly, Google announced that it would also review the application. After a prompt review, Google declined to remove the app from Google Play, citing that it did not violate the agreed upon terms and conditions of the store. Saudi doctor Khawla Al-Kuraya supported this app an editorial in Bloomberg News. Kuraya wrote that Absher helped Saudi women avoid governmental bureaucracy as it allows their male guardians to process their travel permits anywhere and anytime through Absher. Although she believes that the guardianship system needs to be reconsidered, she thinks that Absher is an important step towards facilitating women-guardians related issues in Saudi Arabia. Absher manager Atiyah Al-Anazy announced in 2019 that two million women were using the application in Saudi Arabia to facilitate their transactions. Some female users stated that the application has made their movement and travel-related issues easier. New measures were introduced that year to allow Saudi women above the age of 18 to travel without their male guardians, which ultimately released male authoritative rights on women. A law was subsequently passed allowing women over the age of 21 to receive a passport and travel without prior male permission.
Read more →