Honeywell's JetWave is a piece of satellite communications hardware produced by Honeywell that enables global in-flight internet connectivity. Its connectivity is provided using Inmarsat’s GX Aviation network. The JetWave platform is used in business and general aviation, as well as defense and commercial airline users. == History == In 2012, Honeywell announced it would provide Inmarsat with the hardware for its GX Ka-band in-flight connectivity network. The Ka-band (pronounced either "kay-ay band" or "ka band") is a portion of the microwave part of the electromagnetic spectrum defined as frequencies in the range 27.5 to 31 gigahertz (GHz). In satellite communications, the Ka-band allows higher bandwidth communication. In 2017, after five years and more than 180 flight hours and testing, JetWave was launched as part of GX Aviation with Lufthansa Group. Honeywell’s JetWave was the exclusive terminal hardware option for the Inmarsat GX Aviation network; however, the exclusivity clause in that contract has expired. In July 2019, the United States Air Force selected Honeywell’s JetWave satcom system for 70 of its C-17 Globemaster III cargo planes. In December 2019, it was reported that six AirAsia aircraft had been fitted with Inmarsat’s GX Aviation Ka-band connectivity system and is slated to be implemented fleetwide across AirAsia’s Airbus A320 and A330 models in 2020, requiring installation of JetWave atop AirAsia’s fuselages. Today, Honeywell’s JetWave hardware is installed on over 1,000 aircraft worldwide. In August 2021, the Civil Aviation Administration of China approved a validation of Honeywell’s MCS-8420 JetWave satellite connectivity system for Airbus 320 aircraft. In December 2021, Honeywell, SES, and Hughes Network Systems demonstrated multi-orbit high-speed airborne connectivity for military customers using Honeywell’s JetWave MCX terminal with a Hughes HM-series modem, and SES satellites in both medium Earth orbit (MEO) and geostationary orbit (GEO). The tests achieved full duplex data rates of more than 40 megabits per second via a number of SES' (GEO) satellites including GovSat-1, and the high-throughput, low-latency O3b MEO satellite constellation, with connections moving between GEO/MEO links in under 30 sec. == Uses == === Commercial aviation === Honeywell’s JetWave enables air transport and regional aircraft to connect to Inmarsat’s GX Aviation network. The multichannel satellite (MSC) JetWave terminals share the same antenna controller, modem and router hardware with the business market, but have an MCS-8200 fuselage-mounted antenna. === Business aviation === Honeywell’s JetWave hardware allows users to connect to Inmarsat’s Jet ConneX, a business aviation broadband connectivity offering to provide Wi-Fi for connected devices. JetWave offers a tail-mount antenna for business jets. === Defense === Honeywell’s JetWave satellite communications system for defense allows users to connect to the Inmarsat GX network, offering global coverage for military airborne operators, including over water, over nontraditional flight paths and in remote areas. JetWave and the Inmarsat GX network enable mission-critical applications like real-time weather; videoconferencing; large file transfers; encryption capabilities; in-flight briefings; intelligence, surveillance, and reconnaissance video; and secure communications. JetWave is configurable for a variety of military platforms and offers antennas for large and small airframes.
ImageNet
The ImageNet project is a large visual database designed for use in visual object recognition software research. More than 14 million images have been hand-annotated by the project to indicate what objects are pictured and in at least one million of the images, bounding boxes are also provided. ImageNet contains more than 20,000 categories, with a typical category, such as "balloon" or "strawberry", consisting of several hundred images. The database of annotations of third-party image URLs is freely available directly from ImageNet, though the actual images are not owned by ImageNet. Since 2010, the ImageNet project runs an annual software contest, the ImageNet Large Scale Visual Recognition Challenge (ILSVRC), where software programs compete to correctly classify and detect objects and scenes. The challenge uses a "trimmed" list of one thousand non-overlapping classes. == History == AI researcher Fei-Fei Li began working on the idea for ImageNet in 2006. At a time when most AI research focused on models and algorithms, Li wanted to expand and improve the data available to train AI algorithms. In 2007, Li met with Princeton professor Christiane Fellbaum, one of the creators of WordNet, to discuss the project. As a result of this meeting, Li went on to build ImageNet starting from the roughly 22,000 nouns of WordNet and using many of its features. She was also inspired by a 1987 estimate that the average person recognizes roughly 30,000 different kinds of objects. As an assistant professor at Princeton, Li assembled a team of researchers to work on the ImageNet project. They used Amazon Mechanical Turk to help with the classification of images. Labeling started in July 2008 and ended in April 2010. It took 49K workers from 167 countries filtering and labeling over 160M candidate images. They had enough budget to have each of the 14 million images labelled three times. The original plan called for 10,000 images per category, for 40,000 categories at 400 million images, each verified 3 times. They found that humans can classify at most 2 images/sec. At this rate, it was estimated to take 19 human-years of labor (without rest). They presented their database for the first time as a poster at the 2009 Conference on Computer Vision and Pattern Recognition (CVPR) in Florida, titled "ImageNet: A Preview of a Large-scale Hierarchical Dataset". The poster was reused at Vision Sciences Society 2009. In 2009, Alex Berg suggested adding object localization as a task. Li approached PASCAL Visual Object Classes contest in 2009 for a collaboration. It resulted in the subsequent ImageNet Large Scale Visual Recognition Challenge starting in 2010, which has 1000 classes and object localization, as compared to PASCAL VOC which had just 20 classes and 19,737 images (in 2010). === Significance for deep learning === On 30 September 2012, a convolutional neural network (CNN) called AlexNet achieved a top-5 error of 15.3% in the ImageNet 2012 Challenge, more than 10.8 percentage points lower than that of the runner-up. Using convolutional neural networks was feasible due to the use of graphics processing units (GPUs) during training, an essential ingredient of the deep learning revolution. According to The Economist, "Suddenly people started to pay attention, not just within the AI community but across the technology industry as a whole." In 2015, AlexNet was outperformed by Microsoft's very deep CNN with over 100 layers, which won the ImageNet 2015 contest, having 3.57% error on the test set. Andrej Karpathy estimated in 2014 that with concentrated effort, he could reach 5.1% error rate, and ~10 people from his lab reached ~12-13% with less effort. It was estimated that with maximal effort, a human could reach 2.4%. == Dataset == ImageNet crowdsources its annotation process. Image-level annotations indicate the presence or absence of an object class in an image, such as "there are tigers in this image" or "there are no tigers in this image". Object-level annotations provide a bounding box around the (visible part of the) indicated object. ImageNet uses a variant of the broad WordNet schema to categorize objects, augmented with 120 categories of dog breeds to showcase fine-grained classification. In 2012, ImageNet was the world's largest academic user of Mechanical Turk. The average worker identified 50 images per minute. The original plan of the full ImageNet would have roughly 50M clean, diverse and full resolution images spread over approximately 50K synsets. This was not achieved. The summary statistics given on April 30, 2010: Total number of non-empty synsets: 21841 Total number of images: 14,197,122 Number of images with bounding box annotations: 1,034,908 Number of synsets with SIFT features: 1000 Number of images with SIFT features: 1.2 million === Categories === The categories of ImageNet were filtered from the WordNet concepts. Each concept, since it can contain multiple synonyms (for example, "kitty" and "young cat"), so each concept is called a "synonym set" or "synset". There were more than 100,000 synsets in WordNet 3.0, majority of them are nouns (80,000+). The ImageNet dataset filtered these to 21,841 synsets that are countable nouns that can be visually illustrated. Each synset in WordNet 3.0 has a "WordNet ID" (wnid), which is a concatenation of part of speech and an "offset" (a unique identifying number). Every wnid starts with "n" because ImageNet only includes nouns. For example, the wnid of synset "dog, domestic dog, Canis familiaris" is "n02084071". The categories in ImageNet fall into 9 levels, from level 1 (such as "mammal") to level 9 (such as "German shepherd"). === Image format === The images were scraped from online image search (Google, Picsearch, MSN, Yahoo, Flickr, etc) using synonyms in multiple languages. For example: German shepherd, German police dog, German shepherd dog, Alsatian, ovejero alemán, pastore tedesco, 德国牧羊犬. ImageNet consists of images in RGB format with varying resolutions. For example, in ImageNet 2012, "fish" category, the resolution ranges from 4288 x 2848 to 75 x 56. In machine learning, these are typically preprocessed into a standard constant resolution, and whitened, before further processing by neural networks. For example, in PyTorch, ImageNet images are by default normalized by dividing the pixel values so that they fall between 0 and 1, then subtracting by [0.485, 0.456, 0.406], then dividing by [0.229, 0.224, 0.225]. These are the mean and standard deviations for ImageNet, so this whitens the input data. === Labels and annotations === Each image is labelled with exactly one wnid. Dense SIFT features (raw SIFT descriptors, quantized codewords, and coordinates of each descriptor/codeword) for ImageNet-1K were available for download, designed for bag of visual words. The bounding boxes of objects were available for about 3000 popular synsets with on average 150 images in each synset. Furthermore, some images have attributes. They released 25 attributes for ~400 popular synsets: Color: black, blue, brown, gray, green, orange, pink, red, violet, white, yellow Pattern: spotted, striped Shape: long, round, rectangular, square Texture: furry, smooth, rough, shiny, metallic, vegetation, wooden, wet === ImageNet-21K === The full original dataset is referred to as ImageNet-21K. ImageNet-21k contains 14,197,122 images divided into 21,841 classes. Some papers round this up and name it ImageNet-22k. The full ImageNet-21k was released in Fall of 2011, as fall11_whole.tar. There is no official train-validation-test split for ImageNet-21k. Some classes contain only 1-10 samples, while others contain thousands. === ImageNet-1K === There are various subsets of the ImageNet dataset used in various context, sometimes referred to as "versions". One of the most highly used subsets of ImageNet is the "ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2012–2017 image classification and localization dataset". This is also referred to in the research literature as ImageNet-1K or ILSVRC2017, reflecting the original ILSVRC challenge that involved 1,000 classes. ImageNet-1K contains 1,281,167 training images, 50,000 validation images and 100,000 test images. Each category in ImageNet-1K is a leaf category, meaning that there are no child nodes below it, unlike ImageNet-21K. For example, in ImageNet-21K, there are some images categorized as simply "mammal", whereas in ImageNet-1K, there are only images categorized as things like "German shepherd", since there are no child-words below "German shepherd". === Later developments === In the WordNet they built ImageNet on, there were 2832 synsets in the "person" subtree. During 2018--2020 period, they removed the download of the ImageNet-21k as they went through extensive filtering in these person synsets. Out of these 2832 synsets, 1593 were deemed "potentially offensive". Out of the remaining 1239, 1081 were deemed not really "visual". The result was that only 158 syn
Knuth–Eve algorithm
In computer science, the Knuth–Eve algorithm is an algorithm for polynomial evaluation. It preprocesses the coefficients of the polynomial to reduce the number of multiplications required at runtime. Ideas used in the algorithm were originally proposed by Donald Knuth in 1962. His procedure opportunistically exploits structure in the polynomial being evaluated. In 1964, James Eve determined for which polynomials this structure exists, and gave a simple method of "preconditioning" polynomials (explained below) to endow them with that structure. == Algorithm == === Preliminaries === Consider an arbitrary polynomial p ∈ R [ x ] {\displaystyle p\in \mathbb {R} [x]} of degree n {\displaystyle n} . Assume that n ≥ 3 {\displaystyle n\geq 3} . Define m {\displaystyle m} such that: if n {\displaystyle n} is odd then n = 2 m + 1 {\displaystyle n=2m+1} , and if n {\displaystyle n} is even then n = 2 m + 2 {\displaystyle n=2m+2} . Unless otherwise stated, all variables in this article represent either real numbers or univariate polynomials with real coefficients. All operations in this article are done over R {\displaystyle \mathbb {R} } . Again, the goal is to create an algorithm that returns p ( x ) {\displaystyle p(x)} given any x {\displaystyle x} . The algorithm is allowed to depend on the polynomial p {\displaystyle p} itself, since its coefficients are known in advance. === Overview === ==== Key idea ==== Using polynomial long division, we can write p ( x ) = q ( x ) ⋅ ( x 2 − α ) + ( β x + γ ) , {\displaystyle p(x)=q(x)\cdot (x^{2}-\alpha )+(\beta x+\gamma ),} where x 2 − α {\displaystyle x^{2}-\alpha } is the divisor. Picking a value for α {\displaystyle \alpha } fixes both the quotient q {\displaystyle q} and the coefficients in the remainder β {\displaystyle \beta } and γ {\displaystyle \gamma } . The key idea is to cleverly choose α {\displaystyle \alpha } such that β = 0 {\displaystyle \beta =0} , so that p ( x ) = q ( x ) ⋅ ( x 2 − α ) + γ . {\displaystyle p(x)=q(x)\cdot (x^{2}-\alpha )+\gamma .} This way, no operations are needed to compute the remainder polynomial, since it's just a constant. We apply this procedure recursively to q {\displaystyle q} , expressing p ( x ) = ( ( q ( x ) ⋅ ( x 2 − α m ) + γ m ) ⋯ ) ⋅ ( x 2 − α 1 ) + γ 1 . {\displaystyle p(x)=\left(\left(q(x)\cdot (x^{2}-\alpha _{m})+\gamma _{m}\right)\cdots \right)\cdot (x^{2}-\alpha _{1})+\gamma _{1}.} After m {\displaystyle m} recursive calls, the quotient q {\displaystyle q} is either a linear or a quadratic polynomial. In this base case, the polynomial can be evaluated with (say) Horner's method. ==== "Preconditioning" ==== For arbitrary p {\displaystyle p} , it may not be possible to force β = 0 {\displaystyle \beta =0} at every step of the recursion. Consider the polynomials p e {\displaystyle p^{e}} and p o {\displaystyle p^{o}} with coefficients taken from the even and odd terms of p {\displaystyle p} respectively, so that p ( x ) = p e ( x 2 ) + x ⋅ p o ( x 2 ) . {\displaystyle p(x)=p^{e}(x^{2})+x\cdot p^{o}(x^{2}).} If every root of p o {\displaystyle p^{o}} is real, then it is possible to write p {\displaystyle p} in the form given above. Each α i {\displaystyle \alpha _{i}} is a different root of p o {\displaystyle p^{o}} , counting multiple roots as distinct. Furthermore, if at least n − 1 {\displaystyle n-1} roots of p {\displaystyle p} lie in one half of the complex plane, then every root of p o {\displaystyle p^{o}} is real. Ultimately, it may be necessary to "precondition" p {\displaystyle p} by shifting it — by setting p ( x ) ← p ( x + t ) {\displaystyle p(x)\gets p(x+t)} for some t {\displaystyle t} — to endow it with the structure that most of its roots lie in one half of the complex plane. At runtime, this shift has to be "undone" by first setting x ← x − t {\displaystyle x\gets x-t} . === Preprocessing step === The following algorithm is run once for a given polynomial p {\displaystyle p} . At this point, the values of x {\displaystyle x} that p {\displaystyle p} will be evaluated on are not known. ==== Better choice of t ==== While any t ≥ Re ( r 2 ) {\displaystyle t\geq {\text{Re}}(r_{2})} can work, it is possible to remove one addition during evaluation if t {\displaystyle t} is also chosen such that two roots of p ( x + t ) {\displaystyle p(x+t)} are symmetric about the origin. In that case, α 1 {\displaystyle \alpha _{1}} can be chosen such that the shifted polynomial has a factor of x 2 − α 1 {\displaystyle x^{2}-\alpha _{1}} , so γ 1 = 0 {\displaystyle \gamma _{1}=0} . It is always possible to find such a t {\displaystyle t} . One possible algorithm for choosing t {\displaystyle t} is: === Evaluation step === The following algorithm evaluates p {\displaystyle p} at some, now known, point x {\displaystyle x} . Assuming t {\displaystyle t} is chosen optimally, γ 1 = 0 {\displaystyle \gamma _{1}=0} . So, the final iteration of the loop can instead run y ← y ⋅ ( s − α i ) , {\displaystyle y\gets y\cdot (s-\alpha _{i}),} saving an addition. == Analysis == In total, evaluation using the Knuth–Eve algorithm for a polynomial of degree n {\displaystyle n} requires n {\displaystyle n} additions and ⌊ n / 2 ⌋ + 2 {\displaystyle \lfloor n/2\rfloor +2} multiplications, assuming t {\displaystyle t} is chosen optimally. No algorithm to evaluate a given polynomial of degree n {\displaystyle n} can use fewer than n {\displaystyle n} additions or fewer than ⌈ n / 2 ⌉ {\displaystyle \lceil n/2\rceil } multiplications during evaluation. This result assumes only addition and multiplication are allowed during both preprocessing and evaluation. The Knuth–Eve algorithm is not well-conditioned.
Information and media literacy
Information and media literacy (IML) is a combination of information literacy and media literacy. It enables people to show and make informed judgments as users of information and media, as well as to become skillful creators and producers of information and media messages. The transformative nature of IML includes creative works and creating new knowledge; to publish and collaborate responsibly requires ethical, cultural and social understanding. IML is also known as media and information literacy (MIL). UNESCO first adopted the term MIL in 2008 as a "composite concept" combining the competencies of information literacy and media literacy. UNESCO emphasizes the importance of global education in media and information literacy, and in 2013 defined Media and Information Literacy (MIL) as the ability to access, evaluate, use, and create information and media content in critical and ethical ways. Prior to the 1990s, the primary focus of information literacy was research skills. Media literacy, a study that emerged around the 1970s, traditionally focuses on the analysis and the delivery of information through various forms of media. Information literacy, as a skill proposed as early as 1974, centers on an individual's ability to recognize information needs and effectively locate, evaluate, and use information. These days, the study of information literacy has been extended to include the study of media literacy in many countries like the UK, Australia and New Zealand. It is also referred to as information and communication technologies (ICT) in the United States. Educators such as Gregory Ulmer have also defined the field as electracy.Media literacy is the ability to actively inquire into and think critically about information. It includes the ability to understand, evaluate, and create media content, and is an essential skill in today's information society. Livingstone, Van Couvering, and Thumim (2008) described the distinction between media literacy and information literacy: "Media literacy views media as lenses or windows for observing the world and expressing the self, whereas information literacy sees information as a tool for taking action in the world." == Integration of media and information literacy == Historically, the fields of information and media literacy have been separate, but over the course of the 21st century there have been calls to integrate both fields. Most definitions of information and media literacy include not only the abilities to locate, access, and analyze information but also the ability to create information. Only by integrating media literacy with information literacy can students better understand the sources of information and how it is used. Media education has primarily taken place in educational institutions, while information education has primarily occurred in libraries. Discussions surrounding the overlap of information literacy and media literacy came to fruition in the mid-to-late 2000s and 2010s as noted by Marcus Leaning. == In the digital age == The definition of literacy is "the ability to read and write". In practice many more skills are needed to locate, critically assess and make effective use of information. By extension, literacy now also includes the ability to manage and interact with digital information and media, in personal, shared and public domains. Historically, "information literacy" has largely been seen from the relatively top-down, organisational viewpoint of library and information sciences. However the same term is also used to describe a generic "information literacy" skill. The modern digital age has led to the proliferation of information spread across the Internet. Individuals must be able to recognize whether information is true or false and better yet know how to locate, evaluate, use, and communicate information in various formats; this is called information literacy. Towards the end of the 20th century, literacy was redefined to include "new literacies" relating to the new skills needed in everyday experience. "Multiliteracies" recognised the multiplicity of literacies, which were often used in combination. "21st century skills" frameworks link new literacies to wider life skills such as creativity, critical thinking, accountability. What these approaches have in common is a focus on the multiple skills needed by individuals to navigate changing personal, professional and public "information landscapes". As the conventional definition of literacy itself continues to evolve among practitioners, so too has the definition of information literacies. Noteworthy definitions include: Zurkowski defined information literacy as "the ability to find known or knowable content on any subject." CILIP, the Chartered Institute of Library and Information Practitioners, defines information literacy as "the ability to think critically and make balanced judgements about any information we find and use". In the United States, the definition proposed by the Association of College and Research Libraries (ACRL) is the most widely recognized. It defines information literacy as "a set of abilities requiring individuals to recognize when information is needed and to locate, evaluate, and use the needed information effectively." JISC, the Joint Information Systems Committee, refers to information literacy as one of six "digital capabilities", seen as an interconnected group of elements centered on "ICT literacy". Mozilla groups digital and other literacies as "21st century skills", a "broad set of knowledge, skills, habits and traits that are important to succeed in today's world". UNESCO, the United Nations Educational, Scientific and Cultural Organization, recognizing the necessity of teaching and learning both traditional and new types of information, the global importance of education was emphasized in 2008 through the "Teacher Media and Information Literacy (MIL) Curriculum". It defines MIL as a set of competencies that enable citizens to access, retrieve, understand, evaluate, use, create, and share information and media content in all formats through various tools in a critical, ethical, and effective manner, so as to participate in and carry out personal, professional, and social activities. Besides this, UNESCO also asserts information literacy as a "universal human right". == 21st-century students == In modern society, although the overall level of education has improved, the channels for knowledge production and dissemination have become increasingly diverse and commercialized, and traditional authoritative institutions no longer hold a monopoly over knowledge validation. While digital platforms have broadened access to information, they have also weakened trust mechanisms and evaluation standards, making epistemological skepticism a norm. Moreover, with the rise and spread of social media, misinformation and disinformation can be just as easily accessed in both densely and sparsely populated areas. These factors further underscore the importance of information literacy education. The IML learning capacities prepare students to be 21st century literate. According to Jeff Wilhelm (2000), "technology has everything to do with literacy. And being able to use the latest electronic technologies has everything to do with being literate." He supports his argument with J. David Bolter's statement that "if our students are not reading and composing with various electronic technologies, then they are illiterate. They are not just unprepared for the future; they are illiterate right now, in our current time and context". In a broader sense, developing this advanced competency of media and information literacy is essential, as it is crucial for students to exercise their freedom of expression in the 21st century. Wilhelm's statement is supported by the 2005 Wired World Phase II (YCWW II) survey conducted by the Media Awareness Network of Canada on 5000 Grade 4 – 11 students. The key findings of the survey were: 62% of Grade 4 students prefer the Internet. 38% of Grade 4 students prefer the library. 91% of Grade 11 students prefer the Internet. 9% of Grade 11 students prefer the library. Marc Prensky (2001) uses the term "digital native" to describe people who have been brought up in a digital world. The Internet has been a pervasive element of young people's home lives. 94% of kids reported that they had Internet access at home, and a significant majority (61%) had a high-speed connection. By the time kids reach Grade 11, half of them (51 percent) have their own Internet-connected computer, separate and apart from the family computer. The survey also showed that young Canadians are now among the most wired in the world. Contrary to the earlier stereotype of the isolated and awkward computer nerd, today's wired kid is a social kid. In general, many students are better networked through the use of technology than most teachers and parents, who may not understand the abilities of technology.
Golden record (informatics)
In informatics, a golden record is the valid version of a data element (record) in a single source of truth system. It may refer to a database, specific table or data field, or any unit of information used. A golden copy is a consolidated data set, and is supposed to provide a single source of truth and a "well-defined version of all the data entities in an organizational ecosystem". Other names sometimes used include master source or master version. The term has been used in conjunction with data quality, master data management, and similar topics. (Different technical solutions exist, see master data management). == Master data == In master data management (MDM), the golden copy refers to the master data (master version) of the reference data which works as an authoritative source for the "truth" for all applications in a given IT landscape.
Comparison of machine learning software
The following tables are a comparison of machine learning software such as software frameworks, libraries, and computer programs used for machine learning. == Machine learning software == == Other comparisons == == Machine learning helper libraries and platforms == Apache OpenNLP — natural language processing toolkit CUDA — GPU computing platform used to accelerate machine learning and deep learning workloads Horovod — distributed training framework for deep learning Hugging Face Transformers — library of pretrained transformer models built on other machine learning frameworks Kubeflow — machine learning platform for Kubernetes Mallet — toolkit for natural language processing and text analysis NumPy — numerical computing library used in machine learning OpenCV — computer vision library with machine learning functions ONNX — open format for representing machine learning models pandas — data analysis and data preparation library used in machine learning PlaidML — tensor compiler and backend for machine learning frameworks Polars — Dataframe library used for machine learning data preparation and analysis PyArrow — columnar data library used in machine learning data processing ROOT (TMVA) — data analysis framework with machine learning tools SciPy — scientific computing and optimization library used in machine learning == Online development environments for machine learning == Google Colab — hosted Jupyter Notebook environment commonly used for machine learning and deep learning JupyterLab — notebook-based development environment for machine learning and data science Jupyter Notebook — interactive notebook environment used for machine learning and data science Kaggle — online data science and machine learning platform
MarkLogic Server
MarkLogic Server is a document-oriented database developed by MarkLogic. It is a NoSQL multi-model database that evolved from an XML database to natively store JSON documents and RDF triples, the data model for semantics. MarkLogic is designed to be a data hub for operational and analytical data. == History == MarkLogic Server was built to address shortcomings with existing search and data products. The product first focused on using XML as the document markup standard and XQuery as the query standard for accessing collections of documents up to hundreds of terabytes in size. Currently the MarkLogic platform is widely used in publishing, government, finance and other sectors. MarkLogic's customers are mostly Global 2000 companies. == Technology == MarkLogic uses documents without upfront schemas to maintain a flexible data model. In addition to having a flexible data model, MarkLogic uses a distributed, scale-out architecture that can handle hundreds of billions of documents and hundreds of terabytes of data. It has received Common Criteria certification, and has high availability and disaster recovery. MarkLogic is designed to run on-premises and within public or private cloud environments like Amazon Web Services. == Features == Indexing MarkLogic indexes the content and structure of documents including words, phrases, relationships, and values in over 200 languages with tokenization, collation, and stemming for core languages. Functionality includes the ability to toggle range indexes, geospatial indexes, the RDF triple index, and reverse indexes on or off based on your data, the kinds of queries that you will run, and your desired performance. Full-text search MarkLogic supports search across its data and metadata using a word or phrase and incorporates Boolean logic, stemming, wildcards, case sensitivity, punctuation sensitivity, diacritic sensitivity, and search term weighting. Data can be searched using JavaScript, XQuery, SPARQL, and SQL. Semantics MarkLogic uses RDF triples to provide semantics for ease of storing metadata and querying. ACID Unlike other NoSQL databases, MarkLogic maintains ACID consistency for transactions. Replication MarkLogic provides high availability with replica sets. Scalability MarkLogic scales horizontally using sharding. MarkLogic can run over multiple servers, balancing the load or replicating data to keep the system up and running in the event of hardware failure. Security MarkLogic has built in security features such as element-level permissions and data redaction. Optic API for Relational Operations An API that lets developers view their data as documents, graphs or rows. Security MarkLogic provides redaction, encryption, and element-level security (allowing for control on read and write rights on parts of a document). == Applications == Banking Big Data Fraud prevention Insurance Claims Management and Underwriting Master data management Recommendation engines == Licensing == MarkLogic is available under various licensing and delivery models, namely a free Developer or an Essential Enterprise license.[3] Licenses are available from MarkLogic or directly from cloud marketplaces such as Amazon Web Services and Microsoft Azure. == Releases == 2001 – Cerisent XQE 1: ACID transactions, Full-text search, XML Storage, XQuery, Role-based security 2004 – Cerisent XQE 2: Scale-out architecture, Enhanced search (stemming, thesaurus, wildcard), Backup and restore 2005 – MarkLogic Server 3: Continuing search improvements, Content Processing Framework (including PDF, Word, Excel, PPT), Failover 2008 – MarkLogic Server 4: Geospatial search, entity extraction, advanced XQuery, performance, scalability enhancements, auditing 2011 – MarkLogic Server 5: Flexible replication / DDIL, real-time indexing, advanced search, improved analytics, concurrency enhancements 2012 – MarkLogic Server 6: REST and Java APIs, App Builder, enhanced UI, improved search 2013 – MarkLogic Server 7: Semantic graph, bitemporal data, tiered storage, improved search, better management 2015 – MarkLogic Server 8: A Native JSON storage, Server-side JavaScript, Bitemporal, Node.js client API, Incremental backup, Flexible replication[16] 2017 – MarkLogic Server 9: Data integration across Relational and Non-Relational data, Advanced Encryption, Element Level Security, Redaction 2019 – MarkLogic Server 10: Enhanced Data Hub, improved SQL, security, analytics performance, cloud support 2022 – MarkLogic Server 11: MarkLogic Ops Director (Monitoring and Administration Improvements), expanded PKI 2025 – MarkLogic Server 12: Generative AI and Native Vector Search, Graph Algorithm Support, Virtual TDEs (relational views on the fly)