AI Content Used In Pragmata

AI Content Used In Pragmata — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

XRX (web application architecture)

In software development XRX is a web application architecture based on XForms, REST and XQuery. XRX applications store data on both the web client and on the web server in XML format and do not require a translation between data formats. XRX is considered a simple and elegant application architecture due to the minimal number of translations needed to transport data between client and server systems. The XRX architecture is also tightly coupled to W3C standards (CSS, XHTML 2.0, XPath, XML Schema) to ensure XRX applications will be robust in the future. Because XRX applications leverage modern declarative languages on the client and functional languages on the server they are designed to empower non-developers who are not familiar with traditional imperative languages such as JavaScript, Java or .Net. == Overview of XRX == XRX is a zero translation application architecture that uses XML to store data in the client web browser, on the application server and in the database server. It is because each of these layers uses XML as the same structural data model that XRX applications do not have to translate data structures to and from both object and relational data structures. Because of the lack of need for translation, XRX is considered to have a clean and elegant design. The XRX web application architecture allows developers to focus on the business problem and not the translation problem. XRX benefits from several advances in software technology: === Client Architectural Features === A model–view–controller (MVC) architecture that separates the data from its presentation and business logic. A single element (xf:submission) for all server submissions. This replaces much of the JavaScript code required in most AJAX applications. An advanced event model (XML Events) consistent with W3C standards that frees applications from having to deal with vendor-specific and browser-specific event handling. A Dependency graph that is used to store the dependency structure of the client controllers. This frees the developer from having to manually update either the model or the views when data changes in an application. This allows spreadsheet-like applications to be created on the client with very little effort. A declarative programming style that allows most client XForms applications to be created using a small set of approximately 20 elements. This allows rich client applications to be created without knowledge of JavaScript or other procedural scripting languages. An easy-to-extend system for creating new user interface controls using the EXtensible Bindings Language. This allows developers to add new controls at any time without fear of incompatibilities with W3C standards. === Server Architecture Features === Many native XML databases have built-in REST interfaces making each XQuery inherently a RESTful web service. A functional programming model that promotes side-effect free systems that are easier to debug and easier to run on multiple processors. An easy-to-extend system using XQuery function and modules. === Both Client and Server === Both XRX client and server components support a wide range of XML related standards such as XPath, XML Schema and XML Namespaces. Consistent use of REST interfaces to exchange data between the client and server for all transfers of data including as-you-type data checking and suggest functions. Consistent integration of W3C standards including use of XPath and XML Schema data types. A large library of standard of functions used on both the client and server. == Overall Benefits of XRX == One of the principal benefits of the XRX architecture is that it avoids the requirement to "shred" complex data structures into relational structures and then reconstitute the data back into structures when a record is edited on the client. Another benefits of the XRX Web application architecture is that it avoids most of the problems around the object-relational impedance mismatch. Another advantage is that the client developer does not have to learn JavaScript on the client. == Comparison with Traditional Object/Relational Web Application Architectures == Many traditional web application architectures created in the late 1990 were based on middle object tiers and persistence layers that used tabular data streams and relational database systems. Because each of these layers used different structures to store the models the systems required much additional complexity to translate between layers. == History of XRX == Early examples of using a zero-translation architecture in multi-tier systems can be traced back to the rise of object-oriented databases in the 1990s. See OODBMS History Mark Birbeck suggested that the combination of XForms, XQuery with REST interfaces between the two had many advantages in a meeting to the UK XML User Group in September 2006 . His presentation was one of the first to specifically suggest that the combination of three technologies: XForms and XQuery with REST interfaces would have surprisingly beneficial effects. Mark termed this process "Skimming" but that term did not seem to be contagious. Erik Bruchez of Orbeon spoke at the XML 2007 conference on Boston in December 2007. In his presentation "XForms and the eXist XML database: a perfect couple", Bruchez showed that many people were discovering synergistic benefits of XForms on the client and XQuery on the server. The label for XRX was suggested by a blog posting by Dan McCreary on December 14, 2007. It was in this article that Dan suggested the need for a contagious meme for the ideas behind the XRX architecture. == Generalizations of XRX == Although XRX was originally intended to connote the use of XForms on the client, REST as an interface and XQuery on the server, other proponents of the symmetrical use of XML on the client and server have generalized the term to encompass any XML-centric web client and any server that can store and query XML documents. This use of XRX is generally referred to as "shallow XRX". These generalizations do benefit from a simplified zero-translation architecture but many do not benefit from REST interfaces, XPath for consistent data selection, declarative systems in the client, and functional languages on the server (one of the key aspects of XRX). Use of all three technologies (XForms, REST and XQuery) is referred to as "deep XRX". Although XRX architecture is centred on XForms and XQuery, it does not preclude the use of other technologies that manipulate XML natively, such as XSLT, XProc, and XSL-FO.
Read more →
AI Paraphrasing Tools: Free vs Paid (2026)

In search of the best AI paraphrasing tool? An AI paraphrasing tool is software that uses machine learning to help you get more done — it turns a rough idea into a polished result in seconds. When choosing one, weigh output quality, pricing, export formats, and how well it fits the tools you already use. Whether you are a beginner or a pro, the right AI paraphrasing tool slots into your workflow and pays for itself fast. Below we compare features, pricing, and real output so you can choose with confidence.
Read more →
The Best Free AI Writing Assistant for Beginners

Shopping for the best AI writing assistant? An AI writing assistant is software that uses machine learning to help you get more done — it keeps getting smarter as the underlying models improve. Pricing, accuracy, and the size of the model behind the tool are the three factors that most affect daily usefulness. Whether you are a beginner or a pro, the right AI writing assistant slots into your workflow and pays for itself fast. Below we compare features, pricing, and real output so you can choose with confidence.
Read more →
Frederick Jelinek

Frederick Jelinek (18 November 1932 – 14 September 2010) was a Czech-American researcher in information theory, automatic speech recognition, and natural language processing. He is well known for his oft-quoted statement, "Every time I fire a linguist, the performance of the speech recognizer goes up". Jelinek was born in Czechoslovakia before World War II and emigrated with his family to the United States in the early years of the communist regime. He studied engineering at the Massachusetts Institute of Technology and taught for 10 years at Cornell University before accepting a job at IBM Research. In 1961, he married Czech screenwriter Milena Jelinek. At IBM, his team advanced approaches to computer speech recognition and machine translation. After IBM, he went to head the Center for Language and Speech Processing at Johns Hopkins University for 17 years, where he was still working on the day he died. == Personal life == Jelinek was born on November 18, 1932, as Bedřich Jelínek in Kladno to Vilém and Trude Jelínek. His father was Jewish; his mother was born in Switzerland to Czech Catholic parents and had converted to Judaism. Jelínek senior, a dentist, had planned early to escape Nazi occupation and flee to England; he arranged for a passport, visa, and the shipping of his dentistry materials. The couple planned to send their son to an English private school. However, Vilém decided to stay at the last minute and was eventually sent to the Theresienstadt concentration camp, where he died in 1945. The family was forced to move to Prague in 1941, but Frederick, his sister and mother—thanks to the latter's background—escaped the concentration camps. After the war, Jelinek entered in the gymnasium, despite having missed several years of schooling because education of Jewish children had been forbidden since 1942. His mother, anxious that her son should get a good education, made great efforts for their emigration, especially when it became clear he would not be allowed to even attempt the graduation examination. His mother hoped her son would become a physician, but Jelinek dreamed of being a lawyer. He studied engineering in evening classes at the City College of New York and received stipends from the National Committee for a Free Europe that allowed him to study at the Massachusetts Institute of Technology. About his choice of specialty, he said: "Fortunately, to electrical engineering there belonged a discipline whose aim was not the construction of physical systems: the theory of information". He obtained his Ph.D. in 1962, with Robert Fano as his adviser. In 1957, Jelinek paid an unexpected visit to Prague. He had been in Vienna and applied for a visa, hoping to see his former acquaintances again. He met with his old friend Miloš Forman, who introduced him to film student Milena Tobolová—whose screenplay had been the basis for the movie Easy Life (Snadný život). His flight back to the U.S. had a stopover in Munich, during which he called her to propose. Tobolová was considered a dissident and the authorities were not happy with her film. Jelinek asked for help from Jerome Wiesner and Cyrus Eaton, the latter who lobbied Nikita Khrushchev. Following the inauguration of John F. Kennedy, a group of Czech dissidents were allowed to emigrate in January 1961. Thanks to the lobbying, the future Milena Jelinek was one of them. After completing his graduate studies, Jelinek, who had developed an interest in linguistics, had plans to work with Charles F. Hockett at Cornell University. However these fell through and during the next ten years he continued to study information theory. Having previously worked at IBM during a sabbatical, he began full-time work there in 1972—at first on leave for Cornell, but permanently from 1974. He remained there for over twenty years. Although at first he had been offered a regular research job, upon his arrival he learned that Josef Raviv had recently been promoted to head of the newly opened IBM Haifa Research Laboratory, and became head of the Continuous Speech Recognition group at the Thomas J. Watson Research Center. Despite his team's successes in this area, Jelinek's work remained little known in his home country because Czech scientists were not allowed to participate in key conferences. After the 1989 fall of communism, Jelinek helped establish scientific relationships, regularly visiting to lecture and helping to persuade IBM to establish a computing centre at Charles University. In 1993, he retired from IBM and went to Johns Hopkins University's Center for Language and Speech Processing, where he was director and Julian Sinclair Smith Professor of Electrical and Computer Engineering. He was still working there at the time of his death; Jelinek died of a heart attack at the close of an otherwise normal workday in mid-September 2010. He was survived by his wife, daughter and son, sister, stepsister, and three grandchildren, including Sophie Gold Jelinek. == Research and legacy == Information theory was a fashionable scientific approach in the mid '50s. However, pioneer Claude Shannon wrote in 1956 that this trendiness was dangerous. He said, "Our fellow scientists in many different fields, attracted by the fanfare and by the new avenues opened to scientific analysis, are using these ideas in their own problems ... It will be all too easy for our somewhat artificial prosperity to collapse overnight when it is realized that the use of a few exciting words like information, entropy, redundancy, do not solve all our problems." During the next decade, a combination of factors shut down the application of information theory to natural language processing (NLP) problems—in particular machine translation. One factor was the 1957 publication of Noam Chomsky's Syntactic Structures, which stated, "probabilistic models give no insight into the basic problems of syntactic structure". This accorded well with the philosophy of the artificial intelligence research of the time, which promoted rule-based approaches. The other factor was the 1966 ALPAC report, which recommended that the government should stop funding research into machine translation. ALPAC chairman John Pierce later said that the field was filled with "mad inventors or untrustworthy engineers". He said that the underlying linguistic problems must be solved before attempts at NLP could be reasonably made. These elements essentially halted research in the field. Jelinek had begun to develop an interest in linguistics after the immigration of his wife, who initially enrolled in the MIT linguistics program with the help of Roman Jakobson. Jelinek often accompanied her to Chomsky's lectures, and even discussed the possibility of changing orientation with his adviser. Fano was "really upset", and after the failure of his project with Hockett at Cornell, he did not return to this field of research until starting work at IBM. The scope of research at IBM was considerably different from that of most other teams. According to Mark Liberman, "While [Jelinek] was leading IBM's effort to solve the general dictation problem during the decade or so following 1972, most other U.S. companies and academic researchers were working on very limited problems ... or were staying out of the field entirely". Jelinek regarded speech recognition as an information theory problem—a noisy channel, in this case the acoustic signal—which some observers considered a daring approach. The concept of perplexity was introduced in their first model, New Raleigh Grammar, which was published in 1976 as the paper "Continuous Speech Recognition by Statistical Methods" in the journal Proceedings of the IEEE. According to Young, the basic noisy channel approach "reduced the speech recognition problem to one of producing two statistical models". Whereas New Raleigh Grammar was a hidden Markov model, their next model, called Tangora, was broader and involved n-grams, specifically trigrams. Even though "it was obvious to everyone that this model was hopelessly impoverished", it was not improved upon until Jelinek presented another paper in 1999. The same trigram approach was applied to phones in single words. Although the identification of parts of speech turned out not to be very useful for speech recognition, tagging methods developed during these projects are now used in various NLP applications. The incremental research techniques developed at IBM eventually became dominant in the field after DARPA, in the mid-80s, returned to NLP research and imposed that methodology to participating teams, shared common goals, data, and precise evaluation metrics. The Continuous Speech Recognition Group's research, which required large amounts of data to train the algorithms, eventually led to the creation of the Linguistic Data Consortium. In the 1980s, although the broader problem of speech recognition remained unsolved, they sought to apply the methods developed to other problems; machine translat
Read more →
Sprite multiplexing

Sprite multiplexing is a computer graphics technique where additional sprites (moving images) can be drawn on the screen, beyond the nominal maximum. It is largely historical, applicable principally to older hardware, where limited resources (such as CPU speed and memory) meant only a relatively small number of sprites were supported. On the other hand, it is also true that without multiplexing, the sprite circuitry would be idle much of the time, and limited resources were wasted. == Description == The sprite multiplexing technique is based on the idea that while the hardware may only support a finite number of sprites, it is sometimes possible to re-use the same sprite "slots" more than once per frame or scan line. The program will first use the hardware to draw one or more sprite(s), as normal. Before the next frame (or next scanline) needs to be drawn, the software reprograms the hardware to display additional sprites, in other positions. For example, the Nintendo Entertainment System explicitly supports hardware sprite multiplexing, where it has 64 hardware sprites, but is only capable of rendering 8 of them per scanline. On the older Atari 2600, sprite multiplexing was not intentionally designed in, but programmers discovered they could reset the TIA graphics chip to draw additional sprites on the same scanline. The sprite multiplexing technique relies on the program being able to identify what part of the video screen is being drawn at the moment, or being triggered by the video hardware to run a subroutine at the crucial moment. The programmer must carefully consider the layout of the screen. If the video graphics hardware is not reprogrammed in time for the extra sprites to be displayed, they will not appear, or will be drawn incorrectly. Modern video graphics hardware typically does not use hardware sprites, since modern computer systems do not have the kind of limitations that sprite hardware is designed to circumvent. == Implementations == Systems that allow the programmer to employ the sprite multiplexing technique include: Atari 2600 Atari 8-bit computers Amiga Commodore 64 MSX Nintendo Entertainment System Super Nintendo Entertainment System Master System Sega Genesis/Mega Drive
Read more →
How to Choose an AI Art Generator

Looking for the best AI art generator? An AI art generator is software that uses machine learning to help you get more done — it can save you hours every week by automating repetitive work. Most options offer a generous free tier, with paid plans unlocking higher limits, faster processing, and team features. Whether you are a beginner or a pro, the right AI art generator slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.
Read more →
AI Voice Assistants Reviews: What Actually Works in 2026

In search of the best AI voice assistant? An AI voice assistant is software that uses machine learning to help you get more done — it turns a rough idea into a polished result in seconds. When choosing one, weigh output quality, pricing, export formats, and how well it fits the tools you already use. Whether you are a beginner or a pro, the right AI voice assistant slots into your workflow and pays for itself fast. Below we compare features, pricing, and real output so you can choose with confidence.
Read more →
Nicholas Carlini

Nicholas Carlini is an American researcher affiliated with Anthropic and previously with Google DeepMind who has published research in the fields of computer security and machine learning. He is known for his work on adversarial machine learning, particularly his work on the Carlini & Wagner attack in 2016. This attack was particularly useful in defeating defensive distillation, a method used to increase model robustness, and has since been effective against other defenses against adversarial input. In 2018, Carlini demonstrated an attack on Mozilla's DeepSpeech model, showing that hidden commands could be embedded in speech inputs, which the model would execute even if they were inaudible to humans. He also led a team at UC Berkeley that successfully broke seven out of nine defenses against adversarial attacks presented at the 2018 International Conference on Learning Representations. In addition to his work on adversarial attacks, Carlini has made significant contributions to understanding the privacy risks of machine learning models. In 2020, he revealed that large language models, like GPT-2, could memorize and output personally identifiable information. His research demonstrated that this issue worsened with larger models, and he later showed similar vulnerabilities in generative image models, such as Stable Diffusion. == Life and career == Nicholas Carlini obtained his Bachelor of Arts in Computer Science and Mathematics from the University of California, Berkeley, in 2013. He then continued his studies at the same university, where he pursued a PhD under the supervision of David Wagner, completing it in 2018. Carlini became known for his work on adversarial machine learning. In 2016, he worked alongside Wagner to develop the Carlini & Wagner attack, a method of generating adversarial examples against machine learning models. The attack was proved to be useful against defensive distillation, a popular mechanism where a student model is trained based on the features of a parent model to increase the robustness and generalizability of student models. The attack gained popularity when it was shown that the methodology was also effective against most other defenses, rendering them ineffective. In 2018, Carlini demonstrated an attack against Mozilla Foundation's DeepSpeech model where he showed that by hiding malicious commands inside normal speech input the speech model would respond to the hidden commands even when the commands were not discernible by humans. In the same year, Carlini and his team at UC Berkeley showed that out of the 11 papers presenting defenses to adversarial attacks accepted in that year's ICLR conference, seven of the defenses could be broken. Since 2021, he and his team have been working on large language models, creating a questionnaire where humans typically scored 35% whereas AI models scored in the 40%, with GPT-3 getting 38% which could be improved to 40% through few shot prompting. The best performer in the test was UnifiedQA, a model developed by Google specifically for answer questions and answer sets. Carlini has also developed methods to cause large language models like ChatGPT to answer harmful questions like how to construct bombs. He is also known for his work studying the privacy of machine learning models. In 2020, he showed for the first time that large language models would memorize some of the text data that they were trained on. For example, he found that GPT-2 could output personally identifiable information. He then led an analysis of larger models and studied how memorization increased with model size. Then, in 2022 he showed the same vulnerability in generative image models, and specifically diffusion models, by showing that Stable Diffusion could output images of people's faces that it was trained on. Following on this, Carlini then showed that ChatGPT would also sometimes output exact copies of webpages it was trained on, including personally identifiable information. Some of these studies have since been referenced by the courts in debating the copyright status of AI models. == Other work == Carlini received the Best of Show award at the 2020 IOCCC for implementing a tic-tac-toe game entirely with calls to printf, expanding on work from a research paper of his from 2015. The judges commented on his submission "This year's Best of Show (carlini) is such a novel way of obfuscation that it would be worth of a special mention in the (future) Best of IOCCC list!". [sic] == Awards == Best Student Paper Award, IEEE S&P 2017 ("Towards Evaluating the Robustness of Neural Networks") Best Paper Award, ICML 2018 ("Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples") Distinguished Paper Award, USENIX 2021 ("Poisoning the Unlabeled Dataset of Semi-Supervised Learning") Distinguished Paper Award, USENIX 2023 ("Tight Auditing of Differentially Private Machine Learning") Best Paper Award, ICML 2024 ("Stealing Part of a Production Language Model") Best Paper Award, ICML 2024 ("Considerations for Differentially Private Learning with Large-Scale Public Pretraining")
Read more →
IEEE Transactions on Visualization and Computer Graphics

IEEE Transactions on Visualization and Computer Graphics is a peer-reviewed scientific journal published by the IEEE Computer Society. It covers subjects related to computer graphics and visualization techniques, systems, software, hardware, and user interface issues. TVCG has been considered the top journal in the field of visualization. Since 2011, TVCG has allowed authors to present recently accepted papers at partner conferences. These include: IEEE Visualization (VIS), including VAST, InfoVis, and SciVis. IEEE Virtual Reality Conference (IEEE VR) IEEE International Symposium on Mixed and Augmented Reality (ISMAR) ACM Symposium on Interactive 3D Graphics and Games (I3D) IEEE Pacific Visualization Conference (IEEE PacificVis) ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) Eurographics Symposium on Geometry Processing (SGP) Pacific Graphics Conference (PG) Eurovis - The EG and VGTC Conference on Visualization Graphics Interfaces (GI)
Read more →
Mark Steedman

Mark Jerome Steedman (born 18 September 1946) is a British computational linguist and cognitive scientist. == Biography == Steedman graduated from the University of Sussex in 1968, with a B.Sc. in Experimental Psychology, and from the University of Edinburgh in 1973, with a Ph.D. in Artificial Intelligence (Dissertation: The Formal Description of Musical Perception gained in 1972. Advisor: Prof. H.C. Longuet-Higgins FRS). He has held posts as Lecturer in Psychology, University of Warwick (1977–83); Lecturer and Reader in Computational Linguistics, University of Edinburgh (1983–8); Associate and full Professor in Computer and Information Sciences, University of Pennsylvania (1988–98). He has held visiting positions at the University of Texas at Austin, the Max Planck Institute for Psycholinguistics, Radboud University Nijmegen, and the University of Pennsylvania, Philadelphia. Steedman currently holds the Chair of Cognitive Science in the School of Informatics at the University of Edinburgh (1998– ). He works in computational linguistics, artificial intelligence, and cognitive science, on Generation of Meaningful Intonation for Speech by Artificial Agents, Animated Conversation, The Communicative Use of Gesture, Tense and Aspect, and combinatory categorial grammar (CCG). He is also interested in Computational Musical Analysis and combinatory logic. == Distinctions == Member of the Academia Europæa (2006) Fellow of the British Academy (2002). Fellow of the Royal Society of Edinburgh (2002) AAAI Fellow (1993) President elect for 2008 of the Association for Computational Linguistics Fellow of the Association for Computational Linguistics (2012) == Principal publications == Steedman, Mark (1996). Surface structure and interpretation. Linguistic Inquiry Monograph. Vol. 30. Cambridge, MA: MIT Press. p. 123. ISBN 978-0-262-19379-5. Steedman, Mark (2000). The Syntactic Process. Language, Speech, and Communication. Cambridge, MA: MIT Press. p. 344. ISBN 978-0-262-69268-7. Steedman, Mark (Fall 2000). "Information Structure and the Syntax-Phonology Interface". Linguistic Inquiry. 31 (4): 649–689. doi:10.1162/002438900554505. ISSN 0024-3892. S2CID 9084597.
Read more →
EDLUT

EDLUT (Event-Driven LookUp Table) is a computer application for simulating networks of spiking neurons. It was developed in the University of Granada and source code was released under GNU GPL version 3. EDLUT uses event-driven simulation scheme and lookup tables to efficiently simulate medium or large spiking neural networks. This allows this application to simulate detailed biological neuron models and to interface with experimental setups (such as a robotic arm) in real time.
Read more →
Abeba Birhane

Abeba Birhane is an Ethiopian-born cognitive scientist who works at the intersection of complex adaptive systems, machine learning, algorithmic bias, and critical race studies. Birhane's work with Vinay Prabhu uncovered that large-scale image datasets commonly used to develop AI systems, including ImageNet and 80 Million Tiny Images, carried racist and misogynistic labels and offensive images. She has been recognized by VentureBeat as a top innovator in computer vision and named as one of the 100 most influential persons in AI 2023 by TIME magazine. == Early life and education == Birhane was born in Ethiopia. She received her Bachelors of Science in Psychology and a Bachelors of Arts in Philosophy from The Open University. In 2015, she completed her Master of Science in Cognitive Science and, in 2021, her Ph.D. at the Complex Software Lab in the School of Computer Science at University College Dublin. == Career and research == Birhane studied the impacts of emerging AI technologies and how they shape individuals and local communities. She found that AI algorithms tend to disproportionately impact vulnerable groups such as older workers, trans people, immigrants, and children. Her research on relational ethics won the best paper award at NeurIPS’s Black in AI workshop in 2019. She has also studied and written about algorithmic colonization driven by corporate agendas. Her work in decolonizing computational sciences addressed the inherited oppressions in current systems especially towards women of color. In 2020, Birhane and Vinay Prabhu, principal machine learning scientist at UnifyID, published a paper examining the problematic data collection, labelling, classification, and consequences of large image datasets. These datasets, including ImageNet and MIT's 80 Million Tiny Images, have been used to develop thousands of AI algorithms and systems. Birhane and Prabhu found that they contained many racist and misogynistic labels and slurs as well as offensive images. This resulted in MIT voluntarily and formally taking down the 80 Million Tiny Images dataset. More recently, Birhane has worked with Rediet Abebe, George Obaido, and Sekou Remy on researching the barriers to data sharing in Africa. They found that power imbalances are significant in the data sharing process, even when the data comes from Africa. Their research was published at the ACM Conference on Fairness, Accountability, and Transparency. In 2024, Birhane established the AI Accountability Lab research group at Trinity College Dublin. == Selected awards == 2019 NeurIPS Black in AI Workshop Best Paper Award 2020 Venture Beat AI Innovations Award in the category Computer Vision Innovation (received with Vinay Prabhu) 2021 100 Brilliant Women in AI Ethics Hall of Fame Honoree 2022 Lero Director’s Prize for PhD/PostDoctoral Contribution. 2023 100 Most Influential People in AI by TIME magazine
Read more →
Multimodal representation learning

Multimodal representation learning is a subfield of representation learning focused on integrating and interpreting information from different modalities, such as text, images, audio, or video, by projecting them into a shared latent space. This allows for semantically similar content across modalities to be mapped to nearby points within that space, facilitating a unified understanding of diverse data types. By automatically learning meaningful features from each modality and capturing their inter-modal relationships, multimodal representation learning enables a unified representation that enhances performance in cross-media analysis tasks such as video classification, event detection, and sentiment analysis. It also supports cross-modal retrieval and translation, including image captioning, video description, and text-to-image synthesis. == Motivation == The primary motivations for multimodal representation learning arise from the inherent nature of real-world data and the limitations of unimodal approaches. Since multimodal data offers complementary and supplementary information about an object or event from different perspectives, it is more informative than relying on a single modality. A key motivation is to narrow the heterogeneity gap that exists between different modalities by projecting their features into a shared semantic subspace. This allows semantically similar content across modalities to be represented by similar vectors, facilitating the understanding of relationships and correlations between them. Multimodal representation learning aims to leverage the unique information provided by each modality to achieve a more comprehensive and accurate understanding of concepts. These unified representations are crucial for improving performance in various cross-media analysis tasks such as video classification, event detection, and sentiment analysis. They also enable cross-modal retrieval, allowing users to search and retrieve content across different modalities. Additionally, it facilitates cross-modal translation, where information can be converted from one modality to another, as seen in applications like image captioning and text-to-image synthesis. The abundance of ubiquitous multimodal data in real-world applications, including understudied areas like healthcare, finance, and human-computer interaction (HCI), further motivates the development of effective multimodal representation learning techniques. == Approaches and methods == === Canonical-correlation analysis based methods === Canonical-correlation analysis (CCA) was first introduced in 1936 by Harold Hotelling and is a fundamental approach for multimodal learning. CCA aims to find linear relationships between two sets of variables. Given two data matrices X ∈ R n × p {\displaystyle X\in \mathbb {R} ^{n\times p}} and Y ∈ R n × q {\displaystyle Y\in \mathbb {R} ^{n\times q}} representing different modalities, CCA finds projection vectors w x ∈ R p {\displaystyle w_{x}\in \mathbb {R} ^{p}} and w y ∈ R q {\displaystyle w_{y}\in \mathbb {R} ^{q}} that maximizes the correlation between the projected variables: ρ = max w x , w y w x ⊤ Σ x y w y w x ⊤ Σ x x w x w y ⊤ Σ y y w y {\displaystyle \rho =\max _{w_{x},w_{y}}{\frac {w_{x}^{\top }\Sigma _{xy}w_{y}}{{\sqrt {w_{x}^{\top }\Sigma _{xx}w_{x}}}{\sqrt {w_{y}^{\top }\Sigma _{yy}w_{y}}}}}} such that Σ x x {\displaystyle \Sigma _{xx}} and Σ y y {\displaystyle \Sigma _{yy}} are the within-modality covariance matrices, and Σ x y {\displaystyle \Sigma _{xy}} is the between-modality covariance matrix. However, standard CCA is limited by its linearity, which led to the development of nonlinear extensions, such as kernel CCA and deep CCA. ==== Kernel CCA ==== Kernel canonical correlation analysis (KCCA) extends traditional CCA to capture nonlinear relationships between modalities by implicitly mapping the data into high dimensional feature spaces using kernel functions. Given kernel functions K x {\displaystyle K_{x}} and K y {\displaystyle K_{y}} with corresponding Gram matrices K x ∈ R n × n {\displaystyle K_{x}\in \mathbb {R} ^{n\times n}} and K y ∈ R n × n {\displaystyle K_{y}\in \mathbb {R} ^{n\times n}} , KCCA seeks coefficients α {\displaystyle \alpha } and β {\displaystyle \beta } that maximize: ρ = max α , β α ⊤ K x K y β α ⊤ K x 2 α β ⊤ K y 2 β {\displaystyle \rho =\max _{\alpha ,\beta }{\frac {\alpha ^{\top }K_{x}Ky\beta }{{\sqrt {\alpha ^{\top }K_{x}^{2}\alpha }}{\sqrt {\beta ^{\top }K_{y}^{2}\beta }}}}} To prevent overfitting, regularization terms are typically added, resulting in: ρ = max α , β α T K x K y β α T ( K x 2 + λ x K x ) α β T ( K y 2 + λ y K y ) β {\displaystyle \rho =\max _{\alpha ,\beta }{\frac {\alpha ^{T}K_{x}K_{y}\beta }{{\sqrt {\alpha ^{T}\left(K_{x}^{2}+\lambda _{x}K_{x}\right)\alpha }}{\sqrt {\;\beta ^{T}\left(K_{y}^{2}+\lambda _{y}K_{y}\right)\beta }}}}} where λ x {\displaystyle \lambda _{x}} and λ y {\displaystyle \lambda _{y}} are regularization parameters. KCCA has proven effective for tasks such as cross-modal retrieval and semantic analysis, though it faces computational challenges with large datasets due to its O ( n 2 ) {\displaystyle O(n^{2})} memory requirement for sorting kernel matrices. KCCA was proposed independently by several researchers. ==== Deep CCA ==== Deep canonical correlation analysis (DCCA), introduced in 2013, employs neural networks to learn nonlinear transformations for maximizing the correlation between modalities. DCCA uses separate neural networks f x {\displaystyle f_{x}} and f y {\displaystyle f_{y}} for each modality to transform the original data before applying CCA: max W x , W y , θ x , θ y corr ⁡ ( f x ( X ; θ x ) , f y ( Y ; θ y ) ) {\displaystyle \max _{W_{x},W_{y},\theta _{x},\theta _{y}}\operatorname {corr} \left(f_{x}(X;\theta _{x}),f_{y}(Y;\theta _{y})\right)} where θ x {\displaystyle \theta _{x}} and θ y {\displaystyle \theta _{y}} represent the parameters of the neural networks, and W x {\displaystyle W_{x}} and W y {\displaystyle W_{y}} are the CCA projection matrices. The correlation objective is computed as: corr ⁡ ( H x , H y ) = tr ⁡ ( T − 1 / 2 H x T H y S − 1 / 2 ) {\displaystyle \operatorname {corr} (H_{x},H_{y})=\operatorname {tr} \left(T^{-1/2}H_{x}^{T}H_{y}S^{-1/2}\right)} where H x = f x ( X ) {\displaystyle H_{x}=f_{x}(X)} and H y = f y ( Y ) {\displaystyle H_{y}=f_{y}(Y)} are the network outputs, T = H x T H x + r x I {\displaystyle T=H_{x}^{T}H_{x}+r_{x}I} , S = H y T H y + r y I {\displaystyle S=H_{y}^{T}H_{y}+r_{y}I} and r x , r y {\displaystyle r_{x},r_{y}} are the regularization parameters. DCCA overcomes the limitations of linear CCA and kernel CCA by learning complex nonlinear relationships while maintaining computational efficiency for large datasets through mini-batch optimization. === Graph-based methods === Graph-based approaches for multimodal representation learning leverage graph structure to model relationships between entities across different modalities. These methods typically represent each modality as a graph and then learn embedding that preserve cross-modal similarities, enabling more effective joint representation of heterogeneous data. One such method is cross-modal graph neural networks (CMGNNs) that extend traditional graph neural networks (GNNs) to handle data from multiple modalities by constructing graphs that capture both intra-modal and inter-modal relationships. These networks model interactions across modalities by representing them as nodes and their relationships as edges. Other graph-based methods include Probabilistic Graphical Models (PGMs) such as deep belief networks (DBN) and deep Boltzmann machines (DBM). These models can learn a joint representation across modalities, for instance, a multimodal DBN achieves this by adding a shared restricted Boltzmann Machine (RBM) hidden layer on top of modality-specific DBNs. Additionally, the structure of data in some domains like Human-Computer Interaction (HCI), such as the view hierarchy of app screens, can potentially be modeled using graph-like structures. The field of graph representation learning is also relevant, with ongoing progress in developing evaluation benchmarks. === Diffusion maps === Another set of methods relevant to multimodal representation learning are based on diffusion maps and their extensions to handle multiple modalities. ==== Multi-view diffusion maps ==== Multi-view diffusion maps address the challenge of achieving multi-view dimensionality reduction by effectively utilizing the availability of multiple views to extract a coherent low-dimensional representation of the data. The core idea is to exploit both the intrinsic relations within each view and the mutual relations between the different views, defining a cross-view model where a random walk process implicitly hops between objects in different views. A multi-view kernel matrix is constructed by combining these relations, defining a cross-view diffusion process and associ
Read more →
Top 10 AI Video Generators Compared (2026)

Shopping for the best AI video generator? An AI video generator is software that uses machine learning to help you get more done — it keeps getting smarter as the underlying models improve. Pricing, accuracy, and the size of the model behind the tool are the three factors that most affect daily usefulness. Whether you are a beginner or a pro, the right AI video generator slots into your workflow and pays for itself fast. We tested the leading options and ranked them by quality, value, and ease of use.
Read more →
AI Avatar Generators: Free vs Paid (2026)

Comparing the best AI avatar generator? An AI avatar generator is software that uses machine learning to help you get more done — it lowers the barrier so anyone can produce professional output. Privacy matters too: check whether your data trains the model and whether a no-log or enterprise tier is available. Whether you are a beginner or a pro, the right AI avatar generator slots into your workflow and pays for itself fast. We tested the leading options and ranked them by quality, value, and ease of use.
Read more →