AI Chat Microsoft Copilot

AI Chat Microsoft Copilot — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  Agent-assisted automation

  Agent-assisted automation is a type of call center technology that automates elements of what the call center agent 1) does with his/her desktop tools and/or 2) says to customers during the call using pre-recorded audio. It is a relatively new category of call center technology that shows promise in improving call center productivity and compliance. == Types of agent-assisted automation == === Pre-recorded audio === Pre-recorded audio (sometimes referred to as soundboard (computer program) or as soundboard technology) is another form of agent-assisted automation. The purpose of using pre-recorded messages is to increase the probability (and in some cases error-proof the process so) that the right information is provided to customers at the right time. The required disclosures are pre-recorded to ensure accuracy and understandability. By integrating the recordings with the customer relationship management software, the right combination of disclosures can be played based on the combination of goods and services the customer purchased. The integration with the customer relationship management software also ensures that the order cannot be submitted until the disclosures are played, essentially error-proofing (poka-yoke) the process of ensuring the customer gets all the required consumer protection information. Phone surveys are ideal applications of this technology. Whether surveying market preferences or political views, the pre-recorded audio with an agent listening allows the questions to be asked in the same way every time, uninfluenced by the agents' fatigue levels, accents, or their own views. === Fraud prevention === Fraud prevention is a specialized type of agent-assisted automation focused on reducing ID theft and credit card fraud. ID theft and credit card fraud are huge threats for call centers and their customers and few good solutions exist, but new agent-assisted automation solutions are producing promising results. The technology allows the agents to remain on the phone while the customers use their phone key pads to enter the information. The tones are masked and the information passes directly into the customer relationship management system or payment gateway in the case of credit card transactions. The automation essentially makes it impossible for call center agents and also call center personnel that might be monitoring the calls to steal the credit card number, social security number, or other personally identifiable information. === Outbound telemarketing === Another specialized application space of agent-assisted automation is in outbound telemarketing, which goes under numerous headings including outbound prospecting, cold calling, solicitation, fund-raising, etc. Turnover is high among agents engaged in this kind of work because the task is tedious and emotionally difficult. It is tedious because the agent spends the bulk of their day, not talking to qualified leads, but in getting wrong numbers and answering machines. == Benefits == Just as automation has benefited manufacturing by reducing the mental and physical effort required of workers while simultaneously improving throughput, quality, and safety, agent-assisted automation is improving call center results while reducing the tiring aspects of the job for agents. In some cases, the agent-assisted automation streamlines the process and allows calls to be handled more quickly. By eliminating cutting and pasting from one application to another, by auto-navigating applications, and by providing a single view of the customer, agent-assisted automation can reduce call handle time and increase agent productivity. Second, in theory, the more steps that can be automated and the more logic that can be built into the call flow (e.g., if the customer buys items 2 and 9, then disclosures a, c, and f are read by the pre-recorded audio), then companies may be able to reduce the amount of training that is required of the agents while at the same time ensuring more consistency and accuracy. However, no published studies have reported this result yet. But an even larger problem in call centers is between-agent variation in behavior and results. Agents differ in the amount of training and coaching they receive, they differ in the amount of experience they have, their jobs are repetitious and tiring, and the process and procedures the agents are supposed to follow constantly change. Moreover, there are significant individual differences between agents in their intelligence, personality, motivations, etc. which all affect performance. Despite the large amount of money call centers have spent over decades trying to reduce between-agent variation, the problem is still so prevalent that one large study of customer interactions with call centers found that a customer's experience was completely a function of the quality of the agent who happened to answer the phone. Therefore, the most significant benefit of agent-assisted automation may prove to be in how the automation error-proofs or poka-yoke the process and ensures that something that needs to be done or said happens every time. Properly implemented, the between-agent variation for whatever step of the process the automation is applied to may be able to be reduced to near zero. This is especially important in a collection agency whose processes and procedures are closely regulated by the Fair Debt Collection Practices Act.

  Read more →
• 

  Hardware-based encryption

  Hardware-based encryption is the use of computer hardware to assist software, or sometimes replace software, in the process of data encryption. Typically, this is implemented as part of the processor's instruction set. For example, the AES encryption algorithm (a modern cipher) can be implemented using the AES instruction set on the ubiquitous x86 architecture. Such instructions also exist on the ARM architecture. However, more unusual systems exist where the cryptography module is separate from the central processor, instead being implemented as a coprocessor, in particular a secure cryptoprocessor or cryptographic accelerator, of which an example is the IBM 4758, or its successor, the IBM 4764. Hardware implementations can be faster and less prone to exploitation than traditional software implementations, and furthermore can be protected against tampering. == History == Prior to the use of computer hardware, cryptography could be performed through various mechanical or electro-mechanical means. An early example is the Scytale used by the Spartans. The Enigma machine was an electro-mechanical system cipher machine notably used by the Germans in World War II. After World War II, purely electronic systems were developed. In 1987 the ABYSS (A Basic Yorktown Security System) project was initiated. The aim of this project was to protect against software piracy. However, the application of computers to cryptography in general dates back to the 1940s and Bletchley Park, where the Colossus computer was used to break the encryption used by German High Command during World War II. The use of computers to encrypt, however, came later. In particular, until the development of the integrated circuit, of which the first was produced in 1960, computers were impractical for encryption, since, in comparison to the portable form factor of the Enigma machine, computers of the era took the space of an entire building. It was only with the development of the microcomputer that computer encryption became feasible, outside of niche applications. The development of the World Wide Web lead to the need for consumers to have access to encryption, as online shopping became prevalent. The key concerns for consumers were security and speed. This led to the eventual inclusion of the key algorithms into processors as a way of both increasing speed and security. == Implementations == === In the instruction set === ==== x86 ==== The X86 architecture, as a CISC (Complex Instruction Set Computer) Architecture, typically implements complex algorithms in hardware. Cryptographic algorithms are no exception. The x86 architecture implements significant components of the AES (Advanced Encryption Standard) algorithm, which can be used by the NSA for Top Secret information. The architecture also includes support for the SHA Hashing Algorithms through the Intel SHA extensions. Whereas AES is a cipher, which is useful for encrypting documents, hashing is used for verification, such as of passwords (see PBKDF2). ==== ARM ==== ARM processors can optionally support Security Extensions. Although ARM is a RISC (Reduced Instruction Set Computer) architecture, there are several optional extensions specified by ARM Holdings. === As a coprocessor === IBM 4758 – The predecessor to the IBM 4764. This includes its own specialised processor, memory and a Random Number Generator. IBM 4764 and IBM 4765, identical except for the connection used. The former uses PCI-X, while the latter uses PCI-e. Both are peripheral devices that plug into the motherboard. === Proliferation === Advanced Micro Devices (AMD) processors are also x86 devices, and have supported the AES instructions since the 2011 Bulldozer processor iteration. Due to the existence of encryption instructions on modern processors provided by both Intel and AMD, the instructions are present on most modern computers. They also exist on many tablets and smartphones due to their implementation in ARM processors. == Advantages == Implementing cryptography in hardware means that part of the processor is dedicated to the task. This can lead to a large increase in speed. In particular, modern processor architectures that support pipelining can often perform other instructions concurrently with the execution of the encryption instruction. Furthermore, hardware can have methods of protecting data from software. Consequently, even if the operating system is compromised, the data may still be secure (see Software Guard Extensions). == Disadvantages == If, however, the hardware implementation is compromised, major issues arise. Malicious software can retrieve the data from the (supposedly) secure hardware – a large class of method used is the timing attack. This is far more problematic to solve than a software bug, even within the operating system. Microsoft regularly deals with security issues through Windows Update. Similarly, regular security updates are released for Mac OS X and Linux, as well as mobile operating systems like iOS, Android, and Windows Phone. However, hardware is a different issue. Sometimes, the issue will be fixable through updates to the processor's microcode (a low level type of software). However, other issues may only be resolvable through replacing the hardware, or a workaround in the operating system which mitigates the performance benefit of the hardware implementation, such as in the Spectre exploit.

  Read more →
• 

  The Dodo (website)

  The Dodo is an American online publisher focused on animals. The website was launched in January 2014 by Izzie Lerer, the daughter of media executive Kenneth Lerer, and journalist Kerry Lauerman. The Dodo has become one of the most popular Facebook publishers, garnering 1 billion video views from the social network in November 2015. The Dodo is headquartered in New York, New York. == History == The company—named after the first recorded species that humans drove to extinction—was founded by Lerer out of "a personal passion for the subject manner". Lerer has a PhD in animal studies with a focus on animal ethics and human relationships from Columbia University, launching the website after noticing the viral success of animal videos online but seeing no one "really owned the space." The Dodo's editorial and video production staff unionized with the Writers Guild of America, East in April 2018.

  Read more →
• 

  Outline of telecommunication

  The following outline is provided as an overview of and topical guide to telecommunication: Telecommunication – the transmission of signals over a distance for the purpose of communication. In modern times, this process almost always involves the use of electromagnetic waves by transmitters and receivers, but in earlier years it also involved the use of drums and visual signals such as smoke, fire, beacons, semaphore lines and other optical communications. == Modes of telecommunication == E-mail Fax Instant messaging Radio Satellite SMS Telegraphy Telephony Television Television broadcasting mobile telephony Videoconferencing VoIP Voicemail == Types of telecommunication networks == Telecommunications network Computer networks ARPANET Ethernet Internet Wireless networks Public switched telephone networks (PSTN) Packet switched networks Radio network Broadband Wireless Broadband == Aspects of telecommunication transmission == Telecommunication Analog Digital Functional profile Optics === Telecommunication technology === Modulation Amplitude modulation Frequency modulation Quadrature amplitude modulation Nyquist rate Nyquist ISI criterion Pulse shaping Intersymbol interference === Communications media types === Physical media for Telecommunication Twisted pair Coaxial cable Optical fiber Telecommunication through Free Space Broadcast radio frequency including television and radio Line-of-sight Communications satellite Terrestrial Microwave Wireless LAN === Relationship between media and transmitters === Physical access to media Simplex Duplex (telecommunications) Logical relationships Return channel Two-way alternating Two-way simultaneous === Multiple access to media === Multiplexing Analog Frequency division multiplexing Space division multiplexing Digital Time-division multiplexing Statistical multiplexing and Packet switching Media Access Control Contention Token-based Centralized token control Distributed token control == History of telecommunication == History of telecommunication History of telegraphy History of the telephone Invention of the telephone Timeline of the telephone History of radio History of television History of videophones History of mobile phones History of computing hardware History of the Internet == Major telecommunications equipment manufacturers == Alcatel-Lucent – French global telecommunications equipment company Aricent – Former company AT&T – American telecommunications company Avaya – American technology company Ciena – American telecommunications company Cisco Systems – American multinational technology companyPages displaying short descriptions of redirect targets Ericsson – Swedish multinational networking and telecommunications company Fujitsu – Japanese multinational technology company HCL Technologies – Indian multinational technology companyPages displaying short descriptions of redirect targets Huawei – Chinese multinational technology company NEC – Japanese technology corporation Nokia – Multinational data networking and telecommunications equipment company ShoreTel – US telecommunications company Verizon – American telecommunications company ZTE – Chinese telecommunications company == Major telecommunications service providers == List of mobile network operators List of telephone operating companies == Telecommunication organizations == Alliance for Telecommunications Industry Solutions Telecommunications Industry Association == Telecommunication publications == Magazines Billing and OSS World Cabling Installation & Maintenance Call Center Communications News Communications System Design Lightwave Mobile Radio Technology (MRT) New Telephony Phone+ RCR Wireless News Telecom Asia Telecommunications Magazine Telephony WhatSatphone Magazine Wireless Systems Design Wireless Week Xchange == Persons influential in telecommunication == Edwin Howard Armstrong – American radio-frequency engineer and inventor (1890–1954) John Logie Baird – Scottish inventor (1888–1946) Paul Baran – American-Jewish engineer (1926–2011) Alexander Graham Bell – Inventor of the telephone (1847–1922) Tim Berners-Lee – English computer scientist (born 1955) Jagadish Chandra Bose – Physicist, biologist and botanist (1857–1937) Vint Cerf – American computer scientist and Internet pioneer (born 1943) Claude Chappe – Late 18th-century French inventor Donald Davies – British computer scientist (1924–2000) Louis Pouzin – French computer scientist and Internet pioneer (born 1931) Lee de Forest – American inventor (1873–1961) Philo Farnsworth – American inventor (1906–1971) Reginald Fessenden – Canadian-American electrical engineer and inventor (1866–1932) Elisha Gray – American electrical engineer (1835–1901) Innocenzo Manzetti – Italian inventor (1826–1877) Guglielmo Marconi – Italian radio-frequency engineer and inventor (1874–1937) Antonio Meucci – Italian inventor (1808–1889) Alexander Stepanovich Popov – Russian physicist (1859–1906)Pages displaying short descriptions of redirect targets Johann Philipp Reis – German scientist and inventor Almon Brown Strowger – American inventor of the telephone exchange (1839–1902) Nikola Tesla – Serbian-American engineer and inventor (1856–1943) Camille Tissot – French physicist (1868–1917) Alfred Vail – 19th-century American machinist and inventor Charles Wheatstone – English physicist and inventor (1802–1875) Vladimir K. Zworykin – Russian-American engineer (1888–1982)

  Read more →
• 

  BevQ

  BevQ is a queue management mobile application developed by Faircode Technologies of Kochi, Kerala. It is provided by the Kerala State Beverages Corporation under Government of Kerala. == History == This app was released together by the Government of Kerala and the Kerala State Beverages Corporation in order to implement social distancing in the liquor stores Kerala in the case of the COVID-19 pandemic in Kerala and to reduce the congestion of people. The BevQ App was released by Faircode Technologies on 27 May 2020 on the Google Play Store. In January 2021, the app was withdrawn as bars had opened. In June 2021, there was a commitment from the Kerala CM that the App will be relaunched again. It has been reported that over 132,000 new users downloaded the app in the 48 hours after the announcement. == Achievements == The BEVQ app, which works only in the state of Kerala, beat all other Indian food and drink apps in 2020 to see the highest growth in year-on-year sessions, according to the State of Mobile 2021 report by App Annie. The app even beat the likes of Domino’s, which is used all across India. Around 300 government Liquor shops and 900 private liquor shops were enlisted in the platform. More than 200 million unique users registered in the platform. About 250,000 tokens were given out a day.

  Read more →
• 

  Spintronics

  Spintronics (a portmanteau of spin transport electronics), also known as spin electronics, is the study of the intrinsic spin of the electron and its associated magnetic moment, in addition to its fundamental electronic charge, in solid-state devices. The field of spintronics concerns spin-charge coupling in metallic systems. The analogous effects in insulators fall into the field of multiferroics. Spintronics fundamentally differs from traditional electronics in that, in addition to charge state, electron spins are used as a further degree of freedom, with implications in the efficiency of data storage and transfer. Spintronic systems are most often realised in dilute magnetic semiconductors (DMS) and Heusler alloys and are of particular interest in the field of quantum computing, such as atomtronics computation. == History == Spintronics emerged from discoveries in the 1980s concerning spin-dependent electron transport phenomena in solid-state devices. This includes the observation of spin-polarized electron injection from a ferromagnetic metal to a normal metal by Johnson and Silsbee (1985) and the discovery of giant magnetoresistance independently by Albert Fert et al. and Peter Grünberg et al. (1988). The origin of spintronics can be traced to the ferromagnet/superconductor tunneling experiments pioneered by Meservey and Tedrow and initial experiments on magnetic tunnel junctions by Julliere in the 1970s. The use of semiconductors for spintronics began with the theoretical proposal of a spin field-effect-transistor by Datta and Das in 1990 and of the electric dipole spin resonance by Rashba in 1960. In 2012, persistent spin helices of synchronized electrons were made to persist for more than a nanosecond, a 30-fold increase over earlier efforts, and longer than the duration of a modern processor clock cycle. In 2025, at 60 K (−213.2 °C; −351.7 °F) crystalline nickel(II) iodide (NiI2) was reported to exhibit p-wave magnetism, in which the spins of nickel atoms became arranged in a spiral pattern in two orientations. The orientations can be switched via a small electrical current. Applied in digital devices, this spintronics behavior requires far less current than the conventional charge-based electronics that powers devices such as computers and phones. == Theory == The spin of the electron is an intrinsic angular momentum that is separate from the angular momentum due to its orbital motion. The magnitude of the projection of the electron's spin along an arbitrary axis is 1 2 ℏ {\displaystyle {\tfrac {1}{2}}\hbar } , implying that the electron acts as a fermion by the spin-statistics theorem. Like orbital angular momentum, the spin has an associated magnetic moment, the magnitude of which is expressed as μ = 3 2 q m e ℏ {\displaystyle \mu ={\tfrac {\sqrt {3}}{2}}{\frac {q}{m_{e}}}\hbar } . In a solid, the spins of many electrons can act together to affect the magnetic and electronic properties of a material, for example endowing it with a permanent magnetic moment as in a ferromagnet. In many materials, electron spins are equally present in both the up and the down state, and no transport properties are dependent on spin. A spintronic device requires generation or manipulation of a spin-polarized population of electrons, resulting in an excess of spin up or spin down electrons. The polarization of any spin dependent property X can be written as P X = X ↑ − X ↓ X ↑ + X ↓ {\displaystyle P_{X}={\frac {X_{\uparrow }-X_{\downarrow }}{X_{\uparrow }+X_{\downarrow }}}} . A net spin polarization can be achieved either through creating an equilibrium energy split between spin up and spin down. Methods include putting a material in a large magnetic field (Zeeman effect), the exchange energy present in a ferromagnet or forcing the system out of equilibrium. The period of time that such a non-equilibrium population can be maintained is known as the spin lifetime, τ {\displaystyle \tau } . In a diffusive conductor, a spin diffusion length λ {\displaystyle \lambda } can be defined as the distance over which a non-equilibrium spin population can propagate. Spin lifetimes of conduction electrons in metals are relatively short (typically less than 1 nanosecond). An important research area is devoted to extending this lifetime to technologically relevant timescales. The mechanisms of decay for a spin polarized population can be broadly classified as spin-flip scattering and spin dephasing. Spin-flip scattering is a process inside a solid that does not conserve spin, and can therefore switch an incoming spin up state into an outgoing spin down state. Spin dephasing is the process wherein a population of electrons with a common spin state becomes less polarized over time due to different rates of electron spin precession. In confined structures, spin dephasing can be suppressed, leading to spin lifetimes of milliseconds in semiconductor quantum dots at low temperatures. Superconductors can enhance central effects in spintronics such as magnetoresistance effects, spin lifetimes and dissipationless spin-currents. The simplest method of generating a spin-polarised current in a metal is to pass the current through a ferromagnetic material. The most common applications of this effect involve giant magnetoresistance (GMR) devices. A typical GMR device consists of at least two layers of ferromagnetic materials separated by a spacer layer. When the two magnetization vectors of the ferromagnetic layers are aligned, the electrical resistance will be lower (so a higher current flows at constant voltage) than if the ferromagnetic layers are anti-aligned. This constitutes a magnetic field sensor. Two variants of GMR have been applied in devices: Current-in-plane (CIP), where the electric current flows parallel to the layers and, Current-perpendicular-to-plane (CPP), where the electric current flows in a direction perpendicular to the layers. Other metal-based spintronics devices: Tunnel magnetoresistance (TMR), where CPP transport is achieved by using quantum-mechanical tunneling of electrons through a thin insulator separating ferromagnetic layers. Spin-transfer torque, where a current of spin-polarized electrons is used to control the magnetization direction of ferromagnetic electrodes in the device. Spin-wave logic devices carry information in the phase. Interference and spin-wave scattering can perform logic operations. == Device types == === Spintronic-logic === Non-volatile spin-logic devices to enable scaling are being extensively studied. Spin-transfer, torque-based logic devices that use spins and magnets for information processing have been proposed. These devices are part of the ITRS exploratory road map. Logic-in memory applications are already in the development stage. A 2017 review article can be found in Materials Today. A generalized circuit theory for spintronic integrated circuits has been proposed so that the physics of spin transport can be utilized by SPICE developers and subsequently by circuit and system designers for the exploration of spintronics for "beyond CMOS computing". === Semiconductor === Doped semiconductor materials display dilute ferromagnetism. In recent years, dilute magnetic oxides (DMOs) including ZnO based DMOs and TiO2-based DMOs have been the subject of numerous experimental and computational investigations. N`0 sources (like manganese-doped gallium arsenide (Ga,Mn)As), increase the interface resistance with a tunnel barrier, or using hot-electron injection. Spin detection in semiconductors has been addressed with multiple techniques: Faraday/Kerr rotation of transmitted/reflected photons Circular polarization analysis of electroluminescence Nonlocal spin valve (adapted from Johnson and Silsbee's work with metals) Ballistic spin filtering The latter technique was used to overcome the lack of spin-orbit interaction and materials issues to achieve spin transport in silicon. Because external magnetic fields (and stray fields from magnetic contacts) can cause large Hall effects and magnetoresistance in semiconductors (which mimic spin-valve effects), the only conclusive evidence of spin transport in semiconductors is demonstration of spin precession and dephasing in a magnetic field non-collinear to the injected spin orientation, called the Hanle effect. === Storage media === Antiferromagnetic storage media have been studied as an alternative to ferromagnetism, especially since with antiferromagnetic material the bits can be stored as well as with ferromagnetic material. Instead of the usual definition 0 ↔ 'magnetisation upwards', 1 ↔ 'magnetisation downwards', the states can be, e.g., 0 ↔ 'vertically alternating spin configuration' and 1 ↔ 'horizontally-alternating spin configuration'.). The main advantages of antiferromagnetic material are: insensitivity to data-damaging perturbations by stray fields due to zero net external magnetization; no effect on near particles, implying that antiferromagnetic device elements wo

  Read more →
• 

  Digital Cinema Initiatives

  Digital Cinema Initiatives, LLC (DCI) is a consortium of major motion picture studios, formed to establish specifications for a common systems architecture for digital cinema systems. The organization was formed in March 2002 by Metro-Goldwyn-Mayer, Paramount Pictures, Sony Pictures, 20th Century Studios, Universal Studios, Walt Disney Studios and Warner Bros. Entertainment The primary purpose of DCI is to establish and document specifications for an open architecture for digital cinema that ensures a uniform and high level of technical performance, reliability and quality. By establishing a common set of content requirements, distributors, studios, exhibitors, d-cinema manufacturers and vendors can be assured of interoperability and compatibility. Because of the relationship of DCI to many of Hollywood's key studios, conformance to DCI's specifications is considered a requirement by software developers or equipment manufacturers targeting the digital cinema market. == Specification == On July 20, 2005, DCI released Version 1.0 of its "Digital Cinema System Specification", commonly referred to as the "DCI Specification". The document describes overall system requirements and specifications for digital cinema. Between March 28, 2006, and March 21, 2007, DCI issued 148 errata to Version 1.0. DCI released Version 1.1 of the DCI Specification on April 12, 2007, incorporating the previous 148 errata into the DCI Specification. On April 15, 2007, at the annual NAB Digital Cinema Summit, DCI announced the new version, as well as some future plans. They released the "Stereoscopic Digital Cinema Addendum" to begin to establish 3-D technical specifications in response to the popularity of 3-D stereoscopic films. It was also announced "which studios would take over the leadership roles in DCI after the current leadership term expires at the end of September." Subsequently, between August 27, 2007, and February 1, 2008, DCI issued 100 errata to Version 1.1. So, DCI released Version 1.2 of the DCI Specification on March 7, 2008, again incorporating the previous 100 errata into the specification document. An additional 96 errata were issued by August 30, 2012, so a revised Version 1.2 incorporating those additional errata was approved on October 10, 2012. DCI approved DCI Specification Version 1.3 on June 27, 2018, integrating the 45 errata issued to the previous version into a new document. On July 20, 2020, fifteen years to the day after Version 1.0, DCI issued a new DCI Specification Version 1.4 that assimilated 29 errata issued since Version 1.3. On October 13, 2021, DCI approved a new DCI Specification Version 1.4.1 that integrated the 23 errata that had been issued to DCI Specification Version 1.4. For the convenience of users, DCI also created an online HTML version of DCI Specification, Version 1.4.1. Due to the HTML conversion process, the footnotes in the DCSS now appear as endnotes. The PDF version contains pagination and page numbers whereas the HTML version does not. DCI Specification Version 1.4.2, dated June 15, 2022, includes revisions and refinements respecting Object-Based Audio Essence (OBAE), also known as Immersive Audio Bitstream (IAB). Version 1.4.2 also implements post-show log record collection utilizing SMPTE 430-17 SMS-OMB Communications Protocol Specification. Additionally, Version 1.4.2 incorporated two prior addenda: the Digital Cinema Object-Based Audio Addendum, dated October 1, 2018 and the Stereoscopic Digital Cinema Addendum, Version 1.0, dated July 11, 2007. Users using Version 1.4.2 no longer need to refer to the separate addenda. Previous DCSS versions are archived on the DCI web site. Based on many SMPTE and ISO standards, such as JPEG 2000-compressed image and "broadcast wave" PCM/WAV sound, the DCI Specification explains the route to create an entire Digital Cinema Package (DCP) from a raw collection of files known as the Digital Cinema Distribution Master (DCDM), as well as the specifics of its content protection, encryption, and forensic marking. The DCI Specification also establishes standards for the decoder requirements and the presentation environment itself, such as ambient light levels, pixel aspect and shape, image luminance, white point chromaticity, and those tolerances to be kept. Even though it specifies what kind of information is required, the DCI Specification does not include specific information about how data within a distribution package is to be formatted. Formatting of this information is defined by the Society of Motion Picture and Television Engineers (SMPTE) digital cinema standards and related documents. == Image and audio capability overview == === 2D image === 2048×1080 (2K) at 24 frame/s or 48 frame/s, or 4096×2160 (4K) at 24 frame/s In 2K, for Scope (2.39:1) presentation 2048×858 pixels of the imager is used In 2K, for Flat (1.85:1) presentation 1998×1080 pixels of the imager is used In 4K, for Scope (2.39:1) presentation 4096×1716 pixels of the imager is used In 4K, for Flat (1.85:1) presentation 3996×2160 pixels of the imager is used 12 bits per color component (36 bits per pixel) via dual HD-SDI (encrypted) 10 bits only permitted for 2K at 48 frame/s CIE XYZ color space, gamma-corrected TIFF 6.0 container format (one file per frame) JPEG 2000 compression From 0 to 5 or from 1 to 6 wavelet decomposition levels for 2K or 4K resolutions, respectively Compression rate of 4.71 bits/pixel (2K @ 24 frame/s), 2.35 bits/pixel (2K @ 48 frame/s), 1.17 bits/pixel (4K @ 24 frame/s) 250 Mbit/s maximum image bit rate === Stereoscopic 3D image === 2048×1080 (2K) at 48 frame/s - 24 frame/s per eye (4096×2160 4K not supported) In 2K, for Scope (2.39:1) presentation 2048×858 pixels of the imager is used In 2K, for Flat (1.85:1) presentation 1998×1080 pixels of the imager is used Optionally, in the HD-SDI link only: 12 bit color, YCxCz 4:2:2 (i.e. chroma subsampling in XYZ space), each eye in separate stream === Audio === 24 bits per sample, 48 kHz or 96 kHz Up to 16 channels WAV container, uncompressed PCM DCI has additionally published a document outlining recommended practice for High Frame Rate digital cinema. This document discloses the following proposed frame rates: 60, 96, and 120 frames per second for 2D at 2K resolution; 48 and 60 for stereoscopic 3D at 2K resolution; 48 and 60 for 2D at 4K resolution. The maximum compressed bit rate for support of all proposed frame rates should be 500 Mbit/s. == Related information == The idea for DCI was originally mooted in late 1999 by Tom McGrath, then COO of Paramount Pictures, who applied to the U.S. Department of Justice for anti-trust waivers to allow the joint cooperation of all seven major motion picture studios. Universal Pictures made one of the first feature-length DCPs created to DCI specifications, using their film Serenity. Although it was not distributed theatrically, it had one public screening on November 7, 2005, at the USC Entertainment Technology Center's Digital Cinema Laboratory in the Pacific Theatre, Hollywood. Inside Man (2006) was Universal's first DCP commercial release, and, in addition to 35mm film distribution, was delivered via hard drive to 20 theatres in the United States along with two trailers. The Academy Film Archive houses the Digital Cinema Initiatives, LLC Collection, which includes film and digital elements from DCI's Standard Evaluation Material (StEM), a 12-minute production shot on 35mm and 65mm film, created for vendors and standards organizations to test and evaluate image compression and digital projection technologies.

  Read more →
• 

  Atomtronics

  Atomtronics is an emerging field concerning the quantum technology of matter-wave circuits which coherently guide propagating ultra-cold atoms. The systems typically include components analogous to those found in electronics, quantum electronics or optical systems, such as beam splitters, transistors, and atomic counterparts of superconducting quantum interference devices (SQUIDs). Applications range from studies of fundamental physics to the development of practical devices such as quantum superfluids for the computation of large models for artificial general intelligence. == Etymology == Atomtronics is a portmanteau of "atom" and "electronics", in reference to the creation of atomic analogues of electronic components, such as transistors and diodes, and also electronic materials such as semiconductors. The field itself has considerable overlap with atom optics and quantum simulation, and is not strictly limited to the development of electronic-like components. However, this field develops into the research of ultra-cold atoms for the applied research implications of computations in quantum science. == Methodology == Three major elements are required for an atomtronic circuit. The first is a Bose-Einstein condensate, which is needed for its coherent and superfluid properties, although an ultracold Fermi gas may also be used for certain applications. The second is a tailored trapping potential, which can be generated optically, magnetically, or using a combination of both. The final element is a method to induce the movement of atoms within the potential, which can be achieved in several ways, for various research advancements around fields not limited to distributed computing, supercomputing, and quantum computing. For example, a transistor-like atomtronic circuit may be realized by a ring-shaped trap divided into two by two moveable weak barriers, with the two separate parts of the ring acting as the drain and the source and the barriers acting as the gate. As the barriers move, atoms flow from the source to the drain. It is now possible to coherently guide matterwaves over distances of up to 40 cm in ring-shaped atomtronic matterwave guide measurement. == Applications == The field of atomtronics is still very nascent and any schemes realized thus far are proof-of-concept. Applications include: gravimetry rotational sensing via the Sagnac effect quantum computing Obstacles to the development of practical sensing devices are largely due to the technical challenges of creating Bose-Einstein condensates. They require bulky lab-based setups not easily suitable for transportation. However, creating portable experimental setups is an active area of research.

  Read more →
• 

  Inpainting

  Inpainting is a conservation process where damaged, deteriorated, or missing parts of an artwork are filled in to present a complete image. This process is commonly used in image restoration. It can be applied to both physical and digital art mediums such as oil or acrylic paintings, chemical photographic prints, sculptures, or digital images and video. With its roots in physical artwork, such as painting and sculpture, traditional inpainting is performed by a trained art conservator who has carefully studied the artwork to determine the mediums and techniques used in the piece, potential risks of treatments, and ethical appropriateness of treatment. == History == The modern use of inpainting can be traced back to Pietro Edwards (1744–1821), Director of the Restoration of the Public Pictures in Venice, Italy. Using a scientific approach, Edwards focused his restoration efforts on the intentions of the artist. It was during the 1930 International Conference for the Study of Scientific Methods for the Examination and Preservation of Works of Art, that the modern approach to inpainting was established. Helmut Ruhemann (1891–1973), a German restorer and conservator, led the discussions on the use of inpainting in conservation. Helmut Ruhemann was a leading figure in modernizing restoration and conservation. His greatest contribution to the field of conservation "was his insistence on following the methods of the original painter exactly, and on understanding the painter's artistic intention". After his career of over 40 years as a conservator, Ruhemann published his treatise The Cleaning of Paintings: Problems & Potentialities in 1968. In describing his method, Ruhemann states that "The surface [of the fill] should be slightly lower than that of the surrounding paint to allow for the thickness of the inpainting...Inpainting medium should look and behave like the original medium, but must not darken with age." Cesare Brandi (1906–1988) developed the teoria del restauro, the inpainting approach combining aesthetics and psychology. However, this approach was used primarily by Italian restorers and conservators, with the terminology becoming widespread in the 1990s. Technological advancements led to new applications of inpainting. Widespread use of digital techniques range from entirely automatic computerized inpainting to tools used to simulate the process manually. Since the mid-1990s, the process of inpainting has evolved to include digital media. More commonly known as image or video interpolation, a form of estimation, digital inpainting includes the use of computer software that relies on sophisticated algorithms to replace lost or corrupted parts of the image data. == Ethics == In order to preserve the integrity of an original artwork, any inpainting technique or treatment applied to physical or digital work should be reversible or distinguishable from the original content of the artwork. Prior to any treatments, conservators proceed according to the American Institute of Conservation of Historical and Artistic Works. There are several ethic considerations before Inpainting can be justified. Various deliberation decisions over the ethical appropriateness of the amount and type of inpainting done, resides on many factors. As most conservation treatments, inpainting's ethical questions rest mainly with authenticity, reversibility and documentation.Any intervention to compensate for loss should be documented in treatment records and reports and should be detectable by common examination methods. Such compensation should be reversible and should not falsely modify the known aesthetic, conceptual, and physical characteristics of the cultural property, especially by removing or obscuring original material.New technologies and the aesthetic demand for perfect images without imperfections challenge conservators' ethical practices to protect the integrity of originals. == Methods == Inpainting methods and techniques depend on the desired goal and type of image being treated. Treatments to fill in the gaps are different between physical and digital art. In inpainting, detailed records of the initial state of the images can help with the treatment and replicate the original closer. === Physical inpainting === Inpainting is rooted in the conservation and restoration of paintings. Inpainting can aim to make a visual improvement to the artwork as a whole by repairing missing or damaged parts using methods and materials equivalent to the original artist's work. ==== Application techniques ==== By studying the painting methods of various artists and the composition of paints used historically, conservators are able to restore works very closely to their original visual appearance. The picture as a whole determines how to fill in the gap. Helmut Ruhemann's inpainting techniques by Jessell have procedures to "preserve" the quality of oil and tempera paintings. === Digital inpainting === Many programs are able to reconstruct missing or damaged areas of digital photographs and videos. Most widely known for use with digital images is Adobe Photoshop. Given the various abilities of the digital camera and the digitization of old photos, inpainting has become an automatic process that can be performed on digital images. The inpainting techniques can be applied to object removal, text removal, and other automatic modifications of images and videos. In video special effects, inpainting is usually performed after video matting. They can also be observed in applications like image compression and super-resolution. In photography and cinema, it is used for film restoration to reverse, repair, or mitigate deterioration (e.g., physical damage such as cracks in photographs, scratches and dust spots in film, or chemical damage resulting in image loss; performed infrared cleaning). It can also be used for removing red-eye, the stamped date from photographs, and objects for creative effect. This technique can be used to replace any lost blocks in the coding and transmission of images, for example, in a streaming video. It can also be used to remove logos or watermarks in videos. Deep learning neural network-based inpainting can be used for decensoring images. Deep image prior-based techniques can be used for digital image inpainting, where a trained deep learning model is either unavailable or infeasible. Deep models for visual content generation, like text-to-image or text-to-video, learn complex priors over the distribution of visual content, and can be used to inpaint missing parts. For example, videos can be separated into layers, using a technique called omnimatte, which either pretrain an omnimatte model or without any training using an omnimatte-zero model. Three main groups of 2D image-inpainting algorithms can be found in the literature. The first one to be noted is structural (or geometric) inpainting, the second one is texture inpainting, the last one is a combination of these two techniques. They use the information of the known or non-destroyed image areas in order to fill the gap, similar to how physical images are restored. ==== Structural ==== Structural or geometric inpainting is used for smooth images that have strong, defined borders. There are many different approaches to geometric inpainting, but they all come from the idea that geometry can be recovered from similar areas or domains. Bertalmio proposed a method of structural inpainting that mimics how conservators address painting restoration. Bertalmio proposed that by progressively transferring similar information from the borders of an inpainting domain inwards, the gap can be filled. ==== Textural ==== While structural/geometric inpainting works to repair smooth images, textural inpainting works best with images that are heavily textured. Texture has a repetitive pattern which means that a missing portion cannot be restored by continuing the level lines into the gap; level lines provide a complete, stable representation of an image. To repair texture in an image, one can combine frequency and spatial domain information to fill in a selected area with a desired texture. This method, while the most simple and very effective, works well when selecting a texture to be in-painted. For a texture that covers a wider area or a larger frame one would have to go through the image segmenting the areas to be in-painted and selecting the corresponding textures from throughout the image; there are programs that can help find the corresponding areas that work in a similar way as 'find and replace' works in a word processor. ==== Combined structural and textural ==== Combined structural and textural inpainting approaches simultaneously try to perform texture- and structure-filling in regions of missing image information. Most parts of an image consist of texture and structure and the boundaries between image regions contain a large amount of structural information. This is the result when blending differ

  Read more →
• 

  General time- and transfer constant analysis

  The general time- and transfer-constants (TTC) analysis is the generalized version of the Cochran-Grabel (CG) method, which itself is the generalized version of zero-value time-constants (ZVT), which in turn is the generalization of the open-circuit time constant method (OCT). While the other methods mentioned provide varying terms of only the denominator of an arbitrary transfer function, TTC can be used to determine every term both in the numerator and the denominator. Its denominator terms are the same as that of Cochran-Grabel method, when stated in terms of time constants (when expressed in Rosenstark notation). however, the numerator terms are determined using a combination of transfer constants and time constants, where the time constants are the same as those in CG method. Transfer constants are low-frequency ratios of the output variable to input variable under different open- and short-circuited active elements. In general, a transfer function (which can characterize gain, admittance, impedance, trans-impedance, etc., based on the choice of the input and output variables) can be written as: H ( s ) = a 0 + a 1 s + a 2 s 2 + … + a m s m 1 + b 1 s + b 2 s 2 + … + b n s n {\displaystyle H(s)={\frac {a_{0}+a_{1}s+a_{2}s^{2}+\ldots +a_{m}s^{m}}{1+b_{1}s+b_{2}s^{2}+\ldots +b_{n}s^{n}}}} == The denominator terms == The first denominator term b 1 {\textstyle b_{1}} can be expressed as the sum of zero value time constants (ZVTs): b 1 = ∑ i = 1 N τ i 0 {\displaystyle b_{1}=\sum _{i=1}^{N}\tau _{i}^{0}} where τ i 0 {\textstyle \tau _{i}^{0}} is the time constant associated with the reactive element i {\textstyle i} when all the other sources are zero-valued (hence the superscript '0'). Setting a capacitor value to zero corresponds to an open circuit, while a zero-valued inductor is a short circuit. So for calculation of the τ i 0 {\textstyle \tau _{i}^{0}} , all other capacitors are open-circuited and all other inductors are short-circuited. This is the essence of the ZVT method, which reduces to OCT when only capacitors are involved. All independent sources are also zero-valued during the time constant calculations (voltage sources short-circuited and current source open-circuited). In this case, if the element in question (element i {\textstyle i} ) is a capacitor, the time constant is given by τ i 0 = R i 0 C i {\displaystyle \tau _{i}^{0}=R_{i}^{0}C_{i}} and when element i {\textstyle i} is an inductor is it given by: τ i 0 = L i / R i 0 {\displaystyle \tau _{i}^{0}=L_{i}/R_{i}^{0}} . where in both cases, the resistance R i 0 {\textstyle R_{i}^{0}} , is the resistance seen by elements i {\textstyle i} (denoted by subscript), when all the other elements are zero-valued (denoted by the zero superscript). The second-order denominator term is equal to: b 2 = ∑ i = 1 N − 1 ∑ j = i + 1 N τ i 0 τ j i = ∑ i 1 ⩽ i ∑ j < j ⩽ N τ i 0 τ j i {\displaystyle b_{2}=\sum _{i=1}^{N-1}\sum _{j=i+1}^{N}\tau _{i}^{0}\tau _{j}^{i}=\sum _{i}^{1\leqslant i}\sum _{j}^{ Read more →

• 

  Global digital divide

  The global digital divide describes global disparities, primarily between developed and developing countries, in regards to access to computing and information resources such as the Internet and the opportunities derived from such access. The Internet is expanding very quickly, and not all countries—especially developing countries—can keep up with the constant changes. The term "digital divide" does not necessarily mean that someone does not have technology; it could mean that there is simply a difference in technology. These differences can refer to, for example, high-quality computers, fast Internet, technical assistance, or telephone services. == Statistics == There is a large inequality worldwide in terms of the distribution of installed telecommunication bandwidth. In 2014 only three countries (China, US, Japan) host 50% of the globally installed bandwidth potential (see pie-chart Figure on the right). This concentration is not new, as historically only ten countries have hosted 70–75% of the global telecommunication capacity (see Figure). The U.S. lost its global leadership in terms of installed bandwidth in 2011, being replaced by China, which hosts more than twice as much national bandwidth potential in 2014 (29% versus 13% of the global total). == Versus the digital divide == The global digital divide is a special case of the digital divide; the focus is set on the fact that "Internet has developed unevenly throughout the world" causing some countries to fall behind in technology, education, labor, democracy, and tourism. The concept of the digital divide was originally popularized regarding the disparity in Internet access between rural and urban areas of the United States of America; the global digital divide mirrors this disparity on an international scale. The global digital divide also contributes to the inequality of access to goods and services available through technology. Computers and the Internet provide users with improved education, which can lead to higher wages; the people living in nations with limited access are therefore disadvantaged. This global divide is often characterized as falling along what is sometimes called the North–South divide of "northern" wealthier nations and "southern" poorer ones. == Obstacles to a solution == Some people argue that necessities need to be considered before achieving digital inclusion, such as an ample food supply and quality health care. Minimizing the global digital divide requires considering and addressing the following types of access: === Physical access === Involves "the distribution of ICT devices per capita…and land lines per thousands". Individuals need to obtain access to computers, landlines, and networks in order to access the Internet. This access barrier is also addressed in Article 21 of the convention on the Rights of Persons with Disabilities by the United Nations. === Financial access === The cost of ICT devices, traffic, applications, technician and educator training, software, maintenance, and infrastructures require ongoing financial means. Financial access and "the levels of household income play a significant role in widening the gap". === Socio-demographic access === Empirical tests have identified that several socio-demographic characteristics foster or limit ICT access and usage. Among different countries, educational levels and income are the most powerful explanatory variables, with age being a third one. While a Global Gender Gap in access and usage of ICT's exist, empirical evidence shows that this is due to unfavorable conditions concerning employment, education and income and not to technophobia or lower ability. In the contexts understudy, women with the prerequisites for access and usage turned out to be more active users of digital tools than men. In the US, for example, the figures for 2018 show 89% of men and 88% of women use the Internet. === Cognitive access === In order to use computer technology, a certain level of information literacy is needed. Further challenges include information overload and the ability to find and use reliable information. === Design access === Computers need to be accessible to individuals with different learning and physical abilities including complying with Section 508 of the Rehabilitation Act as amended by the Workforce Investment Act of 1998 in the United States. === Institutional access === In illustrating institutional access, Wilson states "the numbers of users are greatly affected by whether access is offered only through individual homes or whether it is offered through schools, community centers, religious institutions, cybercafés, or post offices, especially in poor countries where computer access at work or home is highly limited". === Political access === Guillen & Suarez argue that "democratic political regimes enable faster growth of the Internet than authoritarian or totalitarian regimes." The Internet is considered a form of e-democracy, and attempting to control what citizens can or cannot view is in contradiction to this. Recently situations in Iran and China have denied people the ability to access certain websites and disseminate information. Iran has prohibited the use of high-speed Internet in the country and has removed many satellite dishes in order to prevent the influence of Western culture, such as music and television. === Cultural access === Many experts claim that bridging the digital divide is not sufficient and that the images and language needed to be conveyed in a language and images that can be read across different cultural lines. A 2013 study conducted by Pew Research Center noted how participants taking the survey in Spanish were nearly twice as likely not to use the internet. == Examples == In the early 21st century, residents of developed countries enjoy many Internet services which are not yet widely available in developing countries, including: Mobile phones and small electronic communication devices; E-communities and social-networking; Fast broadband Internet connections, enabling advanced Internet applications; Affordable and widespread Internet access, either through personal computers at home or work, through public terminals in public libraries and Internet cafes, and through wireless access points; E-commerce enabled by efficient electronic payment networks like credit cards and reliable shipping services; Virtual globes featuring street maps searchable down to individual street addresses and detailed satellite and aerial photography; Online research systems which enable users to peruse newspaper and magazine articles that may be centuries old, without having to leave home; Electronic readers such as Kindle, Sony Reader, Samsung Papyrus and Iliad by iRex Technologies; Price engines which help consumers find the best possible online prices and similar services which find the best possible prices at local retailers; Electronic services delivery of government services, such as the ability to pay taxes, fees, and fines online. Further civic engagement through e-government and other sources such as finding information about candidates regarding political situations. == Proposed remedies == There are four specific arguments why it is important to "bridge the gap": Economic equality – For example, the telephone is often seen as one of the most important components, because having access to a working telephone can lead to higher safety. If there were to be an emergency, one could easily call for help if one could use a nearby phone. In another example, many work-related tasks are online, and people without access to the Internet may not be able to complete work up to company standards. The Internet is regarded by some as a basic component of civic life that developed countries ought to guarantee for their citizens. Additionally, welfare services, for example, are sometimes offered via the Internet. Social mobility – Computer and Internet use is regarded as being very important to development and success. However, some children are not getting as much technical education as others, because lower socioeconomic areas cannot afford to provide schools with computer facilities. For this reason, some kids are being separated and not receiving the same chance as others to be successful. Democracy – Some people believe that eliminating the digital divide would help countries become healthier democracies. They argue that communities would become much more involved in events such as elections or decision making. Economic growth – It is believed that less-developed nations could gain quick access to economic growth if the information infrastructure were to be developed and well used. By improving the latest technologies, certain countries and industries can gain a competitive advantage. While these four arguments are meant to lead to a solution to the digital divide, there are a couple of other components that need to be considered. The first one is rural living versus s

  Read more →
• 

  Bridgefy

  Bridgefy is a Mexican software company with offices in Mexico and California, the United States, dedicated to developing mesh-networking technology for mobile apps. It was founded circa 2014 by Jorge Rios, Roberto Betancourt and Diego Garcia who conceived the idea while participating in a tech competition called StartupBus. Bridgefy's smartphone ad hoc network technology, apparently using Bluetooth Mesh, is licensed to other apps. The app gained popularity during protests in different countries since it can operate without Internet, using Bluetooth instead. Aware of the security issues of not using cryptography and the criticism surrounding it, Bridgefy announced in late October 2020 that they adopted the Signal protocol, in both their app and SDK, to keep information private, though security researchers have demonstrated that Bridgefy's usage of the Signal Protocol is insecure. == Usage == The app gained popularity as a communication tactic during the 2019–2020 Hong Kong protests and Citizenship Amendment Act protests in India, because it requires people who want to intercept the message to be physically close because of Bluetooth's limited range, and the ability to daisy-chain devices to send messages further than Bluetooth's range. == Security == In August 2020, researchers published a paper describing numerous attacks against the application, which allow de-anonymizing users, building social graphs of users’ interactions (both in real time and after the fact), decrypting and reading direct messages, impersonating users to anyone else on the network, completely shutting down the network, performing active man-in-the-middle attacks to read messages and even modify them. In response to the disclosures, developers acknowledged that "no part of the Bridgefy app is encrypted now" and gave a vague promise to release a new version "encrypted with top security protocols". Later developers said they plan to switch to Signal Protocol, which is widely recognized by cryptographers and used by Signal and WhatsApp. The Signal Protocol was integrated into the Bridgefy app and SDK by late October 2020, with the developers claiming to have included improvements such as the impossibility of a third person impersonating any other user, man-in-the-middle attacks done by modifying stored keys, and historical proximity tracking, among others. However, in 2022, the same security researchers, now including Kenny Paterson, published a paper describing how Bridgefy's usage of the Signal Protocol was incorrect, failing to remedy the previously discovered issues. The researchers performed a demonstration, showing that it was possible for users to intercept messages intended for others without the sender noticing. The researchers disclosed the vulnerabilities to the developers of Bridgefy in August 2021, but, according to the researchers, the developers had yet to resolve the issues as of June 2022. On July 31, 2023, the security firm 7asecurity released a blog post and pentest report of a white box penetration test and overall security review of the Bridgefy app in collaboration with the platform's developers. Their review, which began in November 2022 and concluded in May 2023, identified multiple critical vulnerabilities throughout the application. Many of the issues were fixed, or partially fixed, before the end of the audit, including user impersonation and biometric bypass. Bridgefy also published a blog post on August 8, 2023, announcing the audit results.

  Read more →
• 

  Retrieval-augmented generation

  Retrieval-augmented generation (RAG) is a technique that enables large language models (LLMs) to retrieve and incorporate new information from external data sources. With RAG, LLMs first refer to a specified set of documents, then respond to user queries. These documents supplement information from the LLM's pre-existing training data. This allows LLMs to use domain-specific and/or updated information that is not available in the training data. For example, this enables LLM-based chatbots to access internal company data or generate responses based on authoritative sources. RAG improves LLMs by incorporating information retrieval before generating responses. Unlike LLMs that rely on static training data, RAG pulls relevant text from databases, uploaded documents, or web sources. According to Ars Technica, "RAG is a way of improving LLM performance, in essence by blending the LLM process with a web search or other document look-up process to help LLMs stick to the facts." This method helps reduce AI hallucinations, which have caused chatbots to describe policies that don't exist, or recommend nonexistent legal cases to lawyers that are looking for citations to support their arguments. RAG also reduces the need to retrain LLMs with new data, saving on computational and financial costs. Beyond efficiency gains, RAG also allows LLMs to include sources in their responses, so users can verify the cited sources. This provides greater transparency, as users can cross-check retrieved content to ensure accuracy and relevance. The term retrieval-augmented generation (RAG) was introduced in a 2020 paper that described combining a parametric language model with a non-parametric external memory accessed through retrieval at inference time. == RAG and LLM limitations == LLMs can provide incorrect information. For example, when Google first demonstrated its LLM tool "Google Bard" (later re-branded to Gemini), the LLM provided incorrect information about the James Webb Space Telescope. This error contributed to a $100 billion decline in Google's stock value. RAG is used to prevent these errors, but it does not solve all the problems. For example, LLMs can generate misinformation even when pulling from factually correct sources if they misinterpret the context. MIT Technology Review gives the example of an AI-generated response stating, "The United States has had one Muslim president, Barack Hussein Obama." The model retrieved this from an academic book rhetorically titled Barack Hussein Obama: America's First Muslim President? The LLM did not "know" or "understand" the context of the title, generating a false statement. LLMs with RAG are programmed to prioritize new information. This technique has been called "prompt stuffing." Without prompt stuffing, the LLM's input is generated by a user; with prompt stuffing, additional relevant context is added to this input to guide the model's response. This approach provides the LLM with key information early in the prompt, encouraging it to prioritize the supplied data over pre-existing training knowledge. == Process == Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating an information-retrieval mechanism that allows models to access and utilize additional data beyond their original training set. Ars Technica notes that "when new information becomes available, rather than having to retrain the model, all that's needed is to augment the model's external knowledge base with the updated information" ("augmentation"). IBM states that "in the generative phase, the LLM draws from the augmented prompt and its internal representation of its training data to synthesize" an answer. === RAG key stages === Typically, the data to be referenced is converted into LLM embeddings, numerical representations in the form of a large vector space. RAG can be used on unstructured (usually text), semi-structured, or structured data (for example knowledge graphs). These embeddings are then stored in a vector database to allow for document retrieval. Given a user query, a document retriever is first called to select the most relevant documents that will be used to augment the query. This comparison can be done using a variety of methods, which depend in part on the type of indexing used. The model feeds this relevant retrieved information into the LLM via prompt engineering of the user's original query. Newer implementations (as of 2023) can also incorporate specific augmentation modules with abilities such as expanding queries into multiple domains and using memory and self-improvement to learn from previous retrievals. Finally, the LLM can generate output based on both the query and the retrieved documents. Some models incorporate extra steps to improve output, such as the re-ranking of retrieved information, context selection, and fine-tuning. == Applications == Retrieval-augmented generation is used in applications where generated responses need to be grounded in external or frequently updated information. Commonly cited use cases include search engines, question-answering systems, customer support chatbots, enterprise knowledge assistants, content generation, recommendation systems, retail and e-commerce, and industrial or manufacturing workflows. In healthcare, RAG has been studied as a way to ground large language model outputs in external medical knowledge sources, although reviews have noted continuing challenges around evaluation, ethics, and clinical reliability. == Improvements == Improvements to the basic process above can be applied at different stages in the RAG flow. === Encoder === These methods focus on the encoding of text as either dense or sparse vectors. Sparse vectors, which encode the identity of a word, are typically dictionary-length and contain mostly zeros. Dense vectors, which encode meaning, are more compact and contain fewer zeros. Various enhancements can improve the way similarities are calculated in the vector stores (databases). Performance improves by optimizing how vector similarities are calculated. Dot products enhance similarity scoring, while approximate nearest neighbor (ANN) searches improve retrieval efficiency over K-nearest neighbors (KNN) searches. Accuracy may be improved with Late Interactions, which allow the system to compare words more precisely after retrieval. This helps refine document ranking and improve search relevance. Hybrid vector approaches may be used to combine dense vector representations with sparse one-hot vectors, taking advantage of the computational efficiency of sparse dot products over dense vector operations. Other retrieval techniques focus on improving accuracy by refining how documents are selected. Some retrieval methods combine sparse representations, such as SPLADE, with query expansion strategies to improve search accuracy and recall. === Retriever-centric methods === These methods aim to enhance the quality of document retrieval in vector databases: Pre-training the retriever using the Inverse Cloze Task (ICT), a technique that helps the model learn retrieval patterns by predicting masked text within documents. Supervised retriever optimization aligns retrieval probabilities with the generator model's likelihood distribution. This involves retrieving the top-k vectors for a given prompt, scoring the generated response's perplexity, and minimizing KL divergence between the retriever's selections and the model's likelihoods to refine retrieval. Reranking techniques can refine retriever performance by prioritizing the most relevant retrieved documents during training. === Language model === By redesigning the language model with the retriever in mind, a 25-time smaller network can get comparable perplexity as its much larger counterparts. Because it is trained from scratch, this method (Retro) incurs the high cost of training runs that the original RAG scheme avoided. The hypothesis is that by giving domain knowledge during training, Retro needs less focus on the domain and can devote its smaller weight resources only to language semantics. The redesigned language model is shown here. It has been reported that Retro is not reproducible, so modifications were made to make it so. The more reproducible version is called Retro++ and includes in-context RAG. === Chunking === Chunking involves various strategies for breaking up the data into vectors so the retriever can find details in it. Three types of chunking strategies are: Fixed length with overlap. This is fast and easy. Overlapping consecutive chunks helps to maintain semantic context across chunks. Syntax-based chunks can break the document up into sentences. Libraries such as spaCy or NLTK can also help. File format-based chunking. Certain file types have natural chunks built in, and it's best to respect them. For example, code files are best chunked and vectorized as whole functions or classes. HTML files should leave

  or base64 encoded elements

  Read more →

  

  Asymmetric follow

  An asymmetric follow social network is one which allows many people to follow an individual or account without having to follow them back. It is also known as asynchronous follow or sometimes asymmetric friendship. Asymmetric follow is a common pattern on Twitter, where someone may have thousands of followers, but themselves follow few (or no) accounts. In September 2010 Facebook started experimenting with a similar feature, which Facebook calls "Subscribe To."

  Read more →

  

  Pull technology

  Pull coding or client pull is a style of network communication, where the initial request for data originates from the client, and then is responded to by the server. The reverse is known as push technology, where the server pushes data to clients. Pull requests form the foundation of network computing, where many clients request data from centralized servers. Pull is used extensively on the Internet for HTTP page requests from websites. A push can also be simulated using multiple pulls within a short amount of time. For example, when pulling POP3 email messages from a server, a client can make regular pull requests, every few minutes. To the user, the email then appears to be pushed, as emails appear to arrive close to real-time. A trade-off of this system is that it places a heavier load on both the server and network to function correctly. Many web feeds, such as RSS are technically pulled by the client. With RSS, the user's RSS reader polls the server periodically for new content; the server does not send information to the client unrequested. This continual polling is inefficient and has contributed to the shutdown or reduction of several popular RSS feeds that could not handle the bandwidth. For solving this problem, the WebSub protocol, as another example of a push code, was devised. Podcasting is specifically a pull technology. When a new podcast episode is published to an RSS feed, it sits on the server until it is requested by a feed reader, mobile podcasting app, or directory. Directories such as Apple Podcasts (iTunes), The Blubrry Directory, and many apps' directories request the RSS feed periodically to update the Podcast's listing on those platforms. Subscribers to those RSS feeds via app or reader will get the episodes when they request the RSS feed next time, independent of when the directory listing updates.

  Read more →