Google AI Image Generator — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.
Picture Prowler was an early piece of photo management software developed around and meant to show off Xing Technology's JPEG image decompression library during the early 1990s. Little known today, it featured thumbnail based picture management, printing, etc. The primary developer was Ray Bunnage from compression / decompression libraries developed by Howard Gordon and Chris Eddy.
Read more →
AI content watermarking is the process of embedding imperceptible yet detectable signals into content generated by artificial intelligence systems, such as text, images, audio, or video. The technique allows the content to be traced and identified as machine-generated without compromising its quality for the end user. AI watermarking has emerged as a key approach to address growing concerns about misinformation, deepfakes, copyright infringement, and the traceability of synthetic content in the context of the rapid development of generative artificial intelligence. Unlike traditional visible watermarks used in photography, AI content watermarks are typically invisible to humans and can only be detected and deciphered algorithmically. The concept is distinct from the watermarking of AI models themselves (to prevent model theft) and from the watermarking of training data (to combat unauthorized data use). Modern AI watermarking schemes are typically formalized as a pair of algorithms, an embedding (or generation) algorithm and a detection algorithm, sharing a secret key, whose performance is evaluated along three competing axes: quality (the watermark must not noticeably degrade outputs), detectability (the watermark must be statistically distinguishable from unwatermarked content), and robustness (the watermark must persist under adversarial or incidental modifications). == Background == Digital watermarking has been used for decades to protect physical and digital media, from paper currency to photographs. Classical schemes typically embedded a fixed bit-string into a fixed cover signal, with robustness criteria defined against a small fixed set of distortions such as JPEG compression or additive Gaussian noise. The rapid advancement of generative AI in the early 2020s, however, created a new and qualitatively different demand: rather than protecting a single artifact, watermarks for AI content must be embedded automatically across an open-ended distribution of generated outputs while remaining robust to a much wider class of adversarial transformations, including paraphrasing, image regeneration via diffusion models, and re-recording. Large image generation models such as DALL-E, Stable Diffusion, and Midjourney, along with large language models like ChatGPT, made it possible to produce highly realistic synthetic text, images, audio, and video at scale, raising significant ethical and security concerns. In July 2023, the Biden administration secured voluntary commitments from leading AI companies, including OpenAI, Alphabet, Meta, and Amazon, to develop watermarking and other provenance technologies to help users identify AI-generated content. == Formal definitions and design goals == Most modern AI watermarking schemes can be formalized as a pair of algorithms ( W m , D e t e c t ) {\displaystyle ({\mathsf {Wm}},{\mathsf {Detect}})} parameterized by a secret key k {\displaystyle k} . The embedding algorithm W m {\displaystyle {\mathsf {Wm}}} takes a generative model M {\displaystyle M} (and optionally a prompt) and returns a watermarked output x {\displaystyle x} ; the detection algorithm D e t e c t ( x , k ) {\displaystyle {\mathsf {Detect}}(x,k)} outputs a real-valued score (typically a p-value or log-likelihood ratio) used to decide whether x {\displaystyle x} was produced by the watermarked generator. The literature evaluates such schemes along several largely conflicting criteria: Criteria for evaluation include imperceptibility or quality preservation, measured for text via perplexity and human preference judgments, and for images and audio via metrics such as PSNR, SSIM, LPIPS, or PESQ. Detectability is typically expressed as the true positive rate at a fixed false positive rate (e.g. 1% or 10^-6), or as the number of tokens or pixels needed to reach a given confidence level. Robustness refers to the requirement that the watermark should survive expected modifications like JPEG or MP3 compression, cropping, noise, paraphrasing, or machine translation. Distortion-freeness is a stronger property requiring that the marginal distribution of any single watermarked output be statistically identical to the unwatermarked model's distribution. Schemes due to Aaronson, Christ et al., and Kuditipudi et al. are distortion-free in this sense, while the original Kirchenbauer et al. scheme is not. Forgery resistance or unforgeability means an adversary without the secret key should be unable to produce content that passes detection. == Techniques == AI watermarking techniques vary significantly depending on the type of content being watermarked. At its core, the process involves two main stages: embedding (or encoding) the watermark, and detection. There are two primary methods for embedding: watermarking during content generation, which requires access to the AI model itself but is generally more robust, and post-generation watermarking, which can be applied to content from any source, including closed-source models. Watermarks can be broadly classified as visible, including overt marks such as logos or text overlays, or imperceptible, which are detectable only by algorithms. They can also be classified by durability: robust watermarks are designed to withstand common transformations such as compression, cropping, and re-encoding, while fragile watermarks are easily destroyed by any alteration, making them useful for tamper detection. A further axis distinguishes zero-bit watermarks, which only signal "this content was generated by model M," from multi-bit watermarks, which embed an arbitrary payload (such as a user identifier) that can be recovered at detection time. === Text === Text watermarking is considered one of the most challenging modalities because natural language offers relatively limited redundancy compared to images or audio. Modern approaches for large language models alter the autoregressive sampling process so that some statistical signature is left in the choice of tokens, while leaving the surface form of the text unchanged. The literature distinguishes three main families of generation-time text watermarks. Logit-biasing schemes (e.g. KGW) add a fixed bias δ {\displaystyle \delta } to a pseudorandomly selected subset of vocabulary logits before softmax sampling. Reweighting or sampling-based schemes (e.g. SynthID-Text) compose multiple pseudorandom tournaments over the model's full distribution. Distortion-free schemes based on the Gumbel-max trick or inverse transform sampling (Aaronson 2022; Kuditipudi et al. 2023; Christ et al. 2024) preserve the marginal output distribution of the model. ==== KGW: token-probability shifting ==== The pioneering "green list / red list" scheme of Kirchenbauer et al. (KGW), introduced at ICML 2023, is the foundation for most subsequent text watermarks. At each decoding step t {\displaystyle t} , a pseudorandom function (PRF) keyed by a secret k {\displaystyle k} is applied to a context window of h {\displaystyle h} previous tokens to deterministically partition the vocabulary V {\displaystyle V} of size N {\displaystyle N} into a "green list" G ⊂ V {\displaystyle G\subset V} of size γ N {\displaystyle \gamma N} and its complement, the "red list" R = V ∖ G {\displaystyle R=V\setminus G} , where γ ∈ ( 0 , 1 ) {\displaystyle \gamma \in (0,1)} (typically γ = 1 / 2 {\displaystyle \gamma =1/2} ) is the green fraction. A logits processor then increments every green-list logit by a fixed bias δ > 0 {\displaystyle \delta >0} before softmax: ℓ v ′ = ℓ v + δ ⋅ 1 [ v ∈ G ] {\displaystyle \ell '_{v}=\ell _{v}+\delta \cdot \mathbf {1} [v\in G]} so that, after sampling, green tokens are over-represented but generation is not constrained to green tokens alone; high-entropy positions tolerate the bias gracefully, while low-entropy positions (where one token dominates the logits) override the watermark and preserve correctness on factual content. Detection requires only the secret key and the candidate text, not the language model itself. The detector recomputes the partition g ( ⋅ ) {\displaystyle g(\cdot )} for each token, counts the number of green hits | G | hits {\displaystyle |G|_{\text{hits}}} in a sequence of length T {\displaystyle T} , and computes a one-proportion z-test statistic: z = | G | hits − γ T T γ ( 1 − γ ) {\displaystyle z={\frac {|G|_{\text{hits}}-\gamma T}{\sqrt {T\gamma (1-\gamma )}}}} Under the null hypothesis that the text was written by an unwatermarked source (human or another model), the green-hit count is approximately binomially distributed with mean γ T {\displaystyle \gamma T} ; a large positive z {\displaystyle z} rejects the null hypothesis. The original paper reports that fewer than 25 watermarked tokens are sufficient to detect a watermark with a false positive rate below 10^-5 on the OPT-1.3B model. A follow-up study by the same group documented robustness under temperature sampling, top-p (nucleus) sampling, and human paraphrasing, and proposed sliding-window
Read more →
Spanner is a distributed SQL database management and storage service developed by Google. It provides features such as global transactions, strongly consistent reads, and automatic multi-site replication and failover. Spanner is used in Google F1, the database for its advertising business Google Ads, as well as Gmail and Google Photos. == Features == Spanner stores large amounts of mutable structured data. Spanner allows users to perform arbitrary queries using SQL with relational data while maintaining strong consistency and high availability for that data with synchronous replication. Key features of Spanner: Transactions can be applied across rows, columns, tables, and databases within a Spanner universe. Clients can control the replication and placement of data using automatic multi-site replication and failover. Replication is synchronous and strongly consistent. Reads are strongly consistent and data is versioned to allow for stale reads: clients can read previous versions of data, subject to garbage collection windows. Supports a native SQL interface for reading and writing data. Support for Graph Query Language == History == Spanner was first described in 2012 for internal Google data centers. Spanner's SQL capability was added in 2017 and documented in a SIGMOD 2017 paper. It became available as part of Google Cloud Platform in 2017, under the name "Cloud Spanner". == Architecture == Spanner uses the Paxos algorithm as part of its operation to shard (partition) data across up to hundreds of servers. It makes heavy use of hardware-assisted clock synchronization using GPS clocks and atomic clocks to ensure global consistency. TrueTime is the brand name for Google's distributed cloud infrastructure, which provides Spanner with the ability to generate monotonically increasing timestamps in data centers around the world. Google's F1 SQL database management system (DBMS) is built on top of Spanner, replacing Google's custom MySQL variant.
Read more →
In databases, a synonym is an alias or alternate name for a table, view, sequence, or other schema object. They are used mainly to make it intuitive for users to access database objects owned by other users. They also hide the underlying object's identity and make it harder for a malicious program or user to target the underlying object (security through obscurity). Because a synonym is just an alternate name for an object, it requires no storage other than its definition. When an application uses a synonym, the DBMS forwards the request to the synonym's underlying base object. By coding your programs to use synonyms instead of database object names, you insulate yourself from any changes in the name, ownership, or object locations, at the cost of adding another layer that also needs to be maintained. Users can also have different needs, for example some may wish to use a shorter name to refer to database objects they often query, which can be done with aliases without having to rename the underlying object and alter the code referring to it. Synonyms are very powerful from the point of view of allowing users access to objects that do not lie within their schema. All synonyms have to be created explicitly with the CREATE SYNONYM command and the underlying objects can be located in the same database or in other databases that are connected by database links There are two major uses of synonyms: Object invisibility: Synonyms can be created to keep the original object hidden from the user. Location invisibility: Synonyms can be created as aliases for tables and other objects that are not part of the local database. When a table or a procedure is created, it is created in a particular schema, and other users can access it only by using that schema's name as a prefix to the object's name. The way around for this is for the schema owner creates a synonym with the same name as the table name. == Public synonyms == Public synonyms are owned by special schema in the Oracle Database called PUBLIC. As mentioned earlier, public synonyms can be referenced by all users in the database. Public synonyms are usually created by the application owner for the tables and other objects such as procedures and packages so the users of the application can see the objects The following code shows how to create a public synonym for the employee table: Now any user can see the table by just typing the original table name. If you wish, you could provide a different table name for that table in the CREATE SYNONYM statement. Remember that the DBA must create public synonyms. Just because you can see a table through public (or private) synonym doesn’t mean that you can also perform SELECT, INSERT, UPDATE or DELETE operations on the table. To be able to perform those operations, a user needs specific privileges for the underlying object, either directly or through roles from the application owner. == Private synonyms == A private synonym is a synonym within a database schema that a developer typically uses to mask the true name of a table, view stored procedure, or other database object in an application schema. Private synonyms, unlike public synonyms, can be referenced only by the schema that owns the table or object. You may want to create private synonyms when you want to refer to the same table by different contexts. Private synonym overrides public synonym definitions. You create private synonyms the same way you create public synonyms, but you omit the PUBLIC keyword in the CREATE statement. The following example shows how to create a private synonym called addresses for the locations table. Note that once you create the private synonym, you can refer to the synonym exactly as you would the original table name. == Drop a synonym == Synonyms, both private and public, are dropped in the same manner by using the DROP SYNONYM command, but there is one important difference. If you are dropping a public synonym; you need to add the keyword PUBLIC after the keyword DROP. The ALL_SYNONYMS (or DBA_SYNONYMS) view provides information on all synonyms in your database.
Read more →
Machine translation is a sub-field of computational linguistics that investigates the use of software to translate text or speech from one natural language to another. In the 1950s, machine translation became a reality in research, although references to the subject can be found as early as the 17th century. The Georgetown experiment, which involved successful fully automatic translation of more than sixty Russian sentences into English in 1954, was one of the earliest recorded projects. Researchers of the Georgetown experiment asserted their belief that machine translation would be a solved problem within a few years. In the Soviet Union, similar experiments were performed shortly after. Consequently, the success of the experiment ushered in an era of significant funding for machine translation research in the United States. The achieved progress was much slower than expected; in 1966, the ALPAC report found that ten years of research had not fulfilled the expectations of the Georgetown experiment and resulted in dramatically reduced funding. Interest grew in statistical models for machine translation, which became more common and also less expensive in the 1980s as available computational power increased. Although there exists no autonomous system of "fully automatic high quality translation of unrestricted text," there are many programs now available that are capable of providing useful output within strict constraints. Several of these programs are available online, such as Google Translate and the SYSTRAN system that powers AltaVista's BabelFish (which was replaced by Microsoft Bing translator in May 2012). == The beginning == The origins of machine translation can be traced back to the work of Al-Kindi, a 9th-century Arabic cryptographer who developed techniques for systemic language translation, including cryptanalysis, frequency analysis, and probability and statistics, which are used in modern machine translation. The idea of machine translation later appeared in the 17th century. In 1629, René Descartes proposed a universal language, with equivalent ideas in different tongues sharing one symbol. In the mid-1930s the first patents for "translating machines" were applied for by Georges Artsrouni, for an automatic bilingual dictionary using punched tape. Russian Peter Troyanskii submitted a more detailed proposal that included both the bilingual dictionary and a method for dealing with grammatical roles between languages, based on the grammatical system of Esperanto. This system was separated into three stages: stage one consisted of a native-speaking editor in the source language to organize the words into their logical forms and to exercise the syntactic functions; stage two required the machine to "translate" these forms into the target language; and stage three required a native-speaking editor in the target language to normalize this output. Troyanskii's proposal remained unknown until the late 1950s, by which time computers were well-known and utilized. == The early years == The first set of proposals for computer based machine translation was presented in 1949 by Warren Weaver, a researcher at the Rockefeller Foundation, "Translation memorandum". These proposals were based on information theory, successes in code breaking during the Second World War, and theories about the universal principles underlying natural language. A few years after Weaver submitted his proposals, research began in earnest at many universities in the United States. On 7 January 1954 the Georgetown–IBM experiment was held in New York at the head office of IBM. This was the first public demonstration of a machine translation system. The demonstration was widely reported in the newspapers and garnered public interest. The system itself, however, was no more than a "toy" system. It had only 250 words and translated 49 carefully selected Russian sentences into English – mainly in the field of chemistry. Nevertheless, it encouraged the idea that machine translation was imminent and stimulated the financing of the research, not only in the US but worldwide. Early systems used large bilingual dictionaries and hand-coded rules for fixing the word order in the final output which was eventually considered too restrictive in linguistic developments at the time. For example, generative linguistics and transformational grammar were exploited to improve the quality of translations. During this period operational systems were installed. The United States Air Force used a system produced by IBM and Washington University in St. Louis, while the Atomic Energy Commission and Euratom, in Italy, used a system developed at Georgetown University. While the quality of the output was poor it met many of the customers' needs, particularly in terms of speed. At the end of the 1950s, Yehoshua Bar-Hillel was asked by the US government to look into machine translation, to assess the possibility of fully automatic high-quality translation by machines. Bar-Hillel described the problem of semantic ambiguity or double-meaning, as illustrated in the following sentence: Little John was looking for his toy box. Finally he found it. The box was in the pen. The word pen may have two meanings: the first meaning, something used to write in ink with; the second meaning, a container of some kind. To a human, the meaning is obvious, but Bar-Hillel claimed that without a "universal encyclopedia" a machine would never be able to deal with this problem. At the time, this type of semantic ambiguity could only be solved by writing source texts for machine translation in a controlled language that uses a vocabulary in which each word has exactly one meaning. == The 1960s, the ALPAC report and the seventies == Research in the 1960s in both the Soviet Union and the United States concentrated mainly on the Russian–English language pair. The objects of translation were chiefly scientific and technical documents, such as articles from scientific journals. The rough translations produced were sufficient to get a basic understanding of the articles. If an article discussed a subject deemed to be confidential, it was sent to a human translator for a complete translation; if not, it was discarded. A great blow came to machine-translation research in 1966 with the publication of the ALPAC report. The report was commissioned by the US government and delivered by ALPAC, the Automatic Language Processing Advisory Committee, a group of seven scientists convened by the US government in 1964. The US government was concerned that there was a lack of progress being made despite significant expenditure. The report concluded that machine translation was more expensive, less accurate and slower than human translation, and that despite the expenditures, machine translation was not likely to reach the quality of a human translator in the near future. The report recommended, however, that tools be developed to aid translators – automatic dictionaries, for example – and that some research in computational linguistics should continue to be supported. The publication of the report had a profound impact on research into machine translation in the United States, and to a lesser extent the Soviet Union and United Kingdom. Research, at least in the US, was almost completely abandoned for over a decade. In Canada, France and Germany, however, research continued. In the US the main exceptions were the founders of SYSTRAN (Peter Toma) and Logos (Bernard Scott), who established their companies in 1968 and 1970 respectively and served the US Department of Defense. In 1970, the SYSTRAN system was installed for the United States Air Force, and subsequently by the Commission of the European Communities in 1976. The METEO System, developed at the Université de Montréal, was installed in Canada in 1977 to translate weather forecasts from English to French, and was translating close to 80,000 words per day or 30 million words per year until it was replaced by a competitor's system on 30 September 2001. While research in the 1960s concentrated on limited language pairs and input, demand in the 1970s was for low-cost systems that could translate a range of technical and commercial documents. This demand was spurred by the increase of globalisation and the demand for translation in Canada, Europe, and Japan. == The 1980s and early 1990s == By the 1980s, both the diversity and the number of installed systems for machine translation had increased. A number of systems relying on mainframe technology were in use, such as SYSTRAN, Logos, Ariane-G5, and Metal. As a result of the improved availability of microcomputers, there was a market for lower-end machine translation systems. Many companies took advantage of this in Europe, Japan, and the USA. Systems were also brought onto the market in China, Eastern Europe, Korea, and the Soviet Union. During the 1980s there was a lot of activity in MT in Japan especially. With the fifth-generation co
Read more →
Automotive security refers to the branch of computer security focused on the cyber risks related to the automotive context. The increasingly high number of ECUs in vehicles and, alongside, the implementation of multiple different means of communication from and towards the vehicle in a remote and wireless manner led to the necessity of a branch of cybersecurity dedicated to the threats associated with vehicles. Not to be confused with automotive safety. == Causes == The implementation of multiple ECUs (Electronic Control Units) inside vehicles began in the early '70s thanks to the development of integrated circuits and microprocessors that made it economically feasible to produce the ECUs on a large scale. Since then the number of ECUs has increased to up to 100 per vehicle. These units nowadays control almost everything in the vehicle, from simple tasks such as activating the wipers to more safety-related ones like brake-by-wire or ABS (Anti-lock Braking System). Autonomous driving is also strongly reliant on the implementation of new, complex ECUs such as the ADAS, alongside sensors (lidars and radars) and their control units. Inside the vehicle, the ECUs are connected with each other through cabled or wireless communication networks, such as CAN bus (controller area network), MOST bus (Media Oriented System Transport), FlexRay (Automotive Network Communications Protocol) or RF (radio frequency) as in many implementations of TPMSs (tire-pressure monitoring systems). Many of these ECUs require data received through these networks that arrive from various sensors to operate and use such data to modify the behavior of the vehicle (e.g., the cruise control modifies the vehicle's speed depending on signals arriving from a button usually located on the steering wheel). Since the development of cheap wireless communication technologies such as Bluetooth, LTE, Wi-Fi, RFID and similar, automotive producers and OEMs have designed ECUs that implement such technologies with the goal of improving the experience of the driver and passengers. Safety-related systems such as the OnStar from General Motors, telematic units, communication between smartphones and the vehicle's speakers through Bluetooth, Android Auto and Apple CarPlay. == Threat model == Threat models of the automotive world are based on both real-world and theoretically possible attacks. Most real-world attacks aim at the safety of the people in and around the car, by modifying the cyber-physical capabilities of the vehicle (e.g., steering, braking, accelerating without requiring actions from the driver), while theoretical attacks have been supposed to focus also on privacy-related goals, such as obtaining GPS data on the vehicle, or capturing microphone signals and similar. Regarding the attack surfaces of the vehicle, they are usually divided in long-range, short-range, and local attack surfaces: LTE and DSRC can be considered long-range ones, while Bluetooth and Wi-Fi are usually considered short-range although still wireless. Finally, USB, OBD-II and all the attack surfaces that require physical access to the car are defined as local. An attacker that is able to implement the attack through a long-range surface is considered stronger and more dangerous than the one that requires physical access to the vehicle. In 2015 the possibility of attacks on vehicles already on the market has been proven possible by Miller and Valasek, that managed to disrupt the driving of a Jeep Cherokee while remotely connecting to it through remote wireless communication. === Controller area network attacks === The most common network used in vehicles and the one that is mainly used for safety-related communication is CAN, due to its real-time properties, simplicity, and cheapness. For this reason the majority of real-world attacks have been implemented against ECUs connected through this type of network. The majority of attacks demonstrated either against actual vehicles or in testbeds fall in one or more of the following categories: ==== Sniffing ==== Sniffing in the computer security field generally refers to the possibility of intercepting and logging packets or more generally data from a network. In the case of CAN, since it is a bus network, every node listens to all communication on the network. It is useful for the attacker to read data to learn the behavior of the other nodes of the network before implementing the actual attack. Usually, the final goal of the attacker is not to simply sniff the data on CAN, since the packets passing on this type of network are not usually valuable just to read. ==== Denial of service ==== Denial of service (DoS) in information security is usually described as an attack that has the objective of making a machine or a network unavailable. DoS attacks against ECUs connected to CAN buses can be done both against the network, by abusing the arbitration protocol used by CAN to always win the arbitration, and targeting the single ECU, by abusing the error handling protocol of CAN. In this second case the attacker flags the messages of the victim as faulty to convince the victim of being broken and therefore shut itself off the network. ==== Spoofing ==== Spoofing attacks comprise all cases in which an attacker, by falsifying data, sends messages pretending to be another node of the network. In automotive security usually spoofing attacks are divided into masquerade and replay attacks. Replay attacks are defined as all those where the attacker pretends to be the victim and sends sniffed data that the victim sent in a previous iteration of authentication. Masquerade attacks are, on the contrary, spoofing attacks where the data payload has been created by the attacker. == Real life automotive threat example == Security researchers Charlie Miller and Chris Valasek have successfully demonstrated remote access to a wide variety of vehicle controls using a Jeep Cherokee as the target. They were able to control the radio, environmental controls, windshield wipers, and certain engine and brake functions. The method used to hack the system was implementation of pre-programmed chip into the controller area network (CAN) bus. By inserting this chip into the CAN bus, he was able to send arbitrary message to CAN bus. One other thing that Miller has pointed out is the danger of the CAN bus, as it broadcasts the signal which the message can be caught by the hackers throughout the network. The control of the vehicle was all done remotely, manipulating the system without any physical interaction. Miller states that he could control any of some 1.4 million vehicles in the United States regardless of the location or distance, the only thing needed is for someone to turn on the vehicle to gain access. The work by Miller and Valasek replicated earlier work completed and published by academics in 2010 and 2011 on a different vehicle. The earlier work demonstrated the ability to compromise a vehicle remotely, over multiple wireless channels (including cellular), and the ability to remotely control critical components on the vehicle post-compromise, including the telematics unit and the car's brakes. While the earlier academic work was publicly visible, both in peer-reviewed scholarly publications and in the press, the Miller and Valesek work received even greater public visibility. == Security measures == The increasing complexity of devices and networks in the automotive context requires the application of security measures to limit the capabilities of a potential attacker. Since the early 2000 many different countermeasures have been proposed and, in some cases, applied. Following, a list of the most common security measures: Sub-networks: to limit the attacker capabilities even if he/she manages to access the vehicle from remote through a remotely connected ECU, the networks of the vehicle are divided in multiple sub-networks, and the most critical ECUs are not placed in the same sub-networks of the ECUs that can be accessed from remote. Gateways: the sub-networks are divided by secure gateways or firewalls that block messages from crossing from a sub-network to the other if they were not intended to. Intrusion Detection Systems (IDS): on each critical sub-network, one of the nodes (ECUs) connected to it has the goal of reading all data passing on the sub-network and detect messages that, given some rules, are considered malicious (made by an attacker). The arbitrary messages can be caught by the passenger by using IDS which will notify the owner regarding with unexpected message. Authentication protocols: in order to implement authentication on networks where it is not already implemented (such as CAN), it is possible to design an authentication protocol that works on the higher layers of the ISO OSI model, by using part of the data payload of a message to authenticate the message itself. Hardware Security Modules: since many ECUs are not powerful enough to keep real-time delays whi
Read more →
MyRadar is a free weather forecasting application developed by Andy Green and his Orlando, Florida-based company ACME AtronOmatic (ACME). The app began operations in 2008 and ran on government-provided weather and radar data for its first decade. In 2019, ACME launched personal satellites to improve predictions of ongoing weather. The app received funding to improve its radar and imaging from the Federal Communications Commission (FCC), National Oceanic and Atmospheric Administration (NOAA), and the Office of Naval Research (ONR). ACME created a weather data satellite constellation named "Hyperspectral Orbital Remote Imaging Spectrometer" (HORIS), which utilizes machine learning and artificial intelligence (AI) to create a current weather map. With the introduction of additional features, including the detection of wildfires and illegal fishing, the app has more broadly become an environmental intelligence app since 2022. In 2024, the app partnered with the Total Traffic and Weather Network (TTWN) to provide traffic flow and incident data for users with paying subscriptions via CarPlay and Android Auto. == History == The app's creator, Andy Green, had created internet tech since the 1980s. His first major project was the development of a public access internet service company based in Rhode Island, which he later sold to finance the creation of ACME AtronOmatic ("ACME" for short), based in Orlando, Florida. The first major app created by ACME was called "Flightwise", which provided users with flight tracking information. In summer 2008, Green had the idea to use the animated location tracker already built-in to Flightwise to make a stand-alone weather forecasting app after wondering if a meal he was eating outdoors would get rained out. MyRadar was launched in 2012 out of an office in Orlando. Despite running solely off of free government-provided weather and radar data for the first decade after launch, Green said the app "took off like wildfire" in downloads. In December 2017, the app partnered with "TripIt" to provide users with information about flight delays and gate changes, eliminating the need for a separate app like Flightwise. In 2019, ACME launched their first personal satellite for the app, a small prototype from New Zealand, as part of an effort to provide detailed imagery and improved predictions of ongoing weather unique to the app. More satellites were eventually launched by ACME to create a weather data satellite constellation named "Hyperspectral Orbital Remote Imaging Spectrometer" (HORIS), monitored by ground stations maintained by Kongsberg Satellite Services. HORIS operates MyRadar by taking the environmental data and imagery it collects and pairing it with machine learning and artificial intelligence (AI) to create a real-time weather map. In 2022, HORIS was expanded upon after ACME won approval from the Federal Communications Commission (FCC) to improve their satellite constellation to include 250 satellites or more. The main batch of satellites were PocketQubes, which entered the atmosphere on May 2, 2022, by Rocket Lab Electron launched from New Zealand, with the additional purpose to test and validate the existing satellites in orbit. In October 2022, ACME received a US$150,000 Small Business Innovation Research (SBIR) grant from the National Oceanic and Atmospheric Administration (NOAA) to improve the app's wildfire detection and air quality measurement technology to better detect smoke, aerosols, fire hotspots using satellites and aerial drones. On August 18, 2023, phase two of the NOAA grant was approved, providing an additional US$650,000 to aid in the app's aforementioned goals by launching a pair of CubeSat satellites to provide high-definition infrared imagery. On September 8, 2023, ACME secured another US$1,200,000 in crowd funding to aid accomplishing the goals of the NOAA grant by expanding the app's workforce from 35 to 100 employees by the end of 2024. In January 2024, MyRadar partnered with Total Traffic and Weather Network (TTWN) to provide traffic data overlaid with its pre-existing weather graphics for users in the United States. The partnership allowed for the app to additionally become a tool for navigation. This officially became a feature days later on January 8, 2024, when the app was made compatible with Apple's CarPlay. On February 7, 2024, the Android equivalent Android Auto also gained the ability to display the app on car interfaces. In March 2024, the app launched a "meteorological wedding planning service" in the United States and Canada for prices between US$1,000 and US$5,000, in which users can request a personal meteorologist to provide an in-person meeting about the best dates for a wedding, and on-call local weather updates the day of. Scheduled for February 2025, four more satellites to help with the NOAA-sponsored wildfire detection are to be launched, and the first by ACME to have AI processing in the satellites themself and not computers on the ground, allowing for quicker transfer of information. == Features and general information == The app's primary function is to provide weather forecasting and prediction to users. The app includes toggleable options to track and send alerts to users for rain, wind patterns, earthquakes, tornadoes, tropical cyclones, wildfires, and more. In early 2020, a feature was added to track orbital objects such as the International Space Station. In May 2022, with the imagery improvement of HORIS, the app gained the secondary abilities to better monitor algae blooms, coral reefs, illegal fishing, and wildfires. In January and February 2024, the ability to display traffic flow and incident data in a feature called "RouteCast" was added, and can be displayed in video and 3D options via CarPlay and Android Auto for users with paying subscriptions. The app also provides annual tropical storm and tornado outlooks for their respective seasons, gathered through satellite and aerial drone data, as well as through on the ground storm chasers.
Read more →
In computing, a materialized view is a database object that contains the results of a query. For example, it may be a local copy of data located remotely, or may be a subset of the rows and/or columns of a table or join result, or may be a summary using an aggregate function. The process of setting up a materialized view is sometimes called materialization. This is a form of caching the results of a query, similar to memoization of the value of a function in functional languages, and it is sometimes described as a form of precomputation. As with other forms of precomputation, database users typically use materialized views for performance reasons, i.e. as a form of optimization. Materialized views that store data based on remote tables were also known as snapshots (deprecated Oracle terminology). In any database management system following the relational model, a view is a virtual table representing the result of a database query. Whenever a query or an update addresses an ordinary view's virtual table, the DBMS converts these into queries or updates against the underlying base tables. A materialized view takes a different approach: the query result is cached as a concrete ("materialized") table (rather than a view as such) that may be updated from the original base tables from time to time. This enables much more efficient access, at the cost of extra storage and of some data being potentially out-of-date. Materialized views find use especially in data warehousing scenarios, where frequent queries of the actual base tables can be expensive. In a materialized view, indexes can be built on any column. In contrast, in a normal view, it's typically only possible to exploit indexes on columns that come directly from (or have a mapping to) indexed columns in the base tables; often this functionality is not offered at all. == Implementations == === Oracle === Materialized views were implemented first by the Oracle Database: the Query rewrite feature was added from version 8i. Example syntax to create a materialized view in Oracle: === PostgreSQL === In PostgreSQL, version 9.3 and newer natively support materialized views. In version 9.3, a materialized view is not auto-refreshed, and is populated only at time of creation (unless WITH NO DATA is used). It may be refreshed later manually using REFRESH MATERIALIZED VIEW. In version 9.4, the refresh may be concurrent with selects on the materialized view if CONCURRENTLY is used. Example syntax to create a materialized view in PostgreSQL: === SQL Server === Microsoft SQL Server differs from other RDBMS by the way of implementing materialized view via a concept known as "Indexed Views". The main difference is that such views do not require a refresh because they are in fact always synchronized to the original data of the tables that compound the view. To achieve this, it is necessary that the lines of origin and destination are "deterministic" in their mapping, which limits the types of possible queries to do this. This mechanism has been realised since the 2000 version of SQL Server. Example syntax to create a materialized view in SQL Server: === Stream processing frameworks === Apache Kafka (since v0.10.2), Apache Spark (since v2.0), Apache Flink, Kinetica DB, Materialize, RisingWave, and Epsio all support materialized views on streams of data. === Others === Materialized views are also supported in Sybase SQL Anywhere. In IBM Db2, they are called "materialized query tables". ClickHouse supports materialized views that automatically refresh on merges. MySQL doesn't support materialized views natively, but workarounds can be implemented by using triggers or stored procedures or by using the open-source application Flexviews. Materialized views can be implemented in Amazon DynamoDB using data modification events captured by DynamoDB Streams. Google announced in 8 April 2020 the availability of materialized views for BigQuery as a beta release.
Read more →
Screen space ambient occlusion (SSAO) is a computer graphics technique for efficiently approximating the ambient occlusion effect in real time. It was developed by Vladimir Kajalin while working at Crytek and was used for the first time in 2007 by the video game Crysis, also developed by Crytek. == Implementation == The algorithm is implemented as a pixel shader, analyzing the scene depth buffer which is stored in a texture. For every pixel on the screen, the pixel shader samples the depth values around the current pixel and tries to compute the amount of occlusion from each of the sampled points. In its simplest implementation, the occlusion factor depends only on the depth difference between sampled point and current point. Without additional smart solutions, such a brute force method would require about 200 texture reads per pixel for good visual quality. This is not acceptable for real-time rendering on current graphics hardware. In order to get high quality results with far fewer reads, sampling is performed using a randomly rotated kernel. The kernel orientation is repeated every N screen pixels in order to have only high-frequency noise in the final picture. In the end this high frequency noise is greatly removed by a NxN post-process blurring step taking into account depth discontinuities (using methods such as comparing adjacent normals and depths). Such a solution allows a reduction in the number of depth samples per pixel to about 16 or fewer while maintaining a high quality result, and allows the use of SSAO in soft real-time applications like computer games. Compared to other ambient occlusion solutions, SSAO has the following advantages: Independent from scene complexity. No data pre-processing needed, no loading time and no memory allocations in system memory. Works with dynamic scenes. Works in the same consistent way for every pixel on the screen. No CPU usage – it can be executed completely on the GPU. May be easily integrated into any modern graphics pipeline. SSAO also has the following disadvantages: Rather local and in many cases view-dependent, as it is dependent on adjacent texel depths which may be generated by any geometry whatsoever. Hard to correctly smooth/blur out the noise without interfering with depth discontinuities, such as object edges (the occlusion should not "bleed" onto objects). Because SSAO operates only on the current depth buffer, it can miss occluding geometry that is not rasterized into the z-buffer and may produce undersampling-related artifacts.
Read more →
Cowrie is a medium interaction SSH and Telnet honeypot designed to log brute force attacks and shell interaction performed by an attacker. Cowrie also functions as an SSH and telnet proxy to observe attacker behavior to another system. Cowrie was developed from Kippo. == Reception == Cowrie has been referenced in published papers. The Book "Hands-On Ethical Hacking and Network Defense" includes Cowrie in a list of 5 commercial honeypots. === Prior uses === Discussing a honeypot effort called the Project Heisenberg Cloud by Rapid7, Bob Rudis, the company's chief data scientist, told eWEEK, "There are custom Rapid7-developed low- and medium-interaction honeypots used within the framework, along with open-source ones, such as Cowrie." Doug Rickert has experimented with the open-source Cowrie SSH honeypot and wrote about it on Medium. Putting up a simple honeypot isn't difficult, and there are many open-source products besides Cowrie, including the original Honeyd to MongoDB and NoSQL honeypots, to ones that emulate web servers. Some appear to be SCADA or other more advanced applications. === Best practices === Researchers at the SysAdmin, Audit, Network and Security (SANS) institute urged administrators and security researchers to run the latest version of Cowrie on a honeypot to monitor shifts in the type of passwords being scanned for and pattern of attacks on IoT devices. === Discussion and further resources === Attack Detection and Forensics Using Honeypot in an IoT Environment calls Cowrie a "medium interaction honeypot" and describes results from using it for 40 days to capture "all communicated sessions in log files." The book Advances on Data Science also devotes chapter two to "Cowrie Honeypot Dataset and Logging." ICCWS 2018 13th International Conference on Cyber Warfare and Security describes using Cowrie. On the Move to Meaningful Internet Systems: OTM 2019 Conferences includes details of using Cowrie. Splunk, a security tool that can receive information from honeypots, outlines how to set up a honeypot using the open-source Cowrie package.
Read more →
A user-defined function (UDF) is a function provided by the user of a program or environment, in a context where the usual assumption is that functions are built into the program or environment. UDFs are usually written for the requirement of its creator. == BASIC language == In some old implementations of the BASIC programming language, user-defined functions are defined using the "DEF FN" syntax. More modern dialects of BASIC are influenced by the structured programming paradigm, where most or all of the code is written as user-defined functions or procedures, and the concept becomes practically redundant. == COBOL language == In the COBOL programming language, a user-defined function is an entity that is defined by the user by specifying a FUNCTION-ID paragraph. A user-defined function must return a value by specifying the RETURNING phrase of the procedure division header and they are invoked using the function-identifier syntax. See the ISO/IEC 1989:2014 Programming Language COBOL standard for details. As of May 2022, the IBM Enterprise COBOL for z/OS 6.4 (IBM COBOL) compiler contains support for user-defined functions. == Databases == In relational database management systems, a user-defined function provides a mechanism for extending the functionality of the database server by adding a function, that can be evaluated in standard query language (usually SQL) statements. The SQL standard distinguishes between scalar and table functions. A scalar function returns only a single value (or NULL), whereas a table function returns a (relational) table comprising zero or more rows, each row with one or more columns. User-defined functions in SQL are declared using the CREATE FUNCTION statement. For example, a user-defined function that converts Celsius to Fahrenheit (a temperature scale used in USA) might be declared like this: Once created, a user-defined function may be used in expressions in SQL statements. For example, it can be invoked where most other intrinsic functions are allowed. This also includes SELECT statements, where the function can be used against data stored in tables in the database. Conceptually, the function is evaluated once per row in such usage. For example, assume a table named Elements, with a row for each known chemical element. The table has a column named BoilingPoint for the boiling point of that element, in Celsius. The query would retrieve the name and the boiling point from each row. It invokes the CtoF user-defined function as declared above in order to convert the value in the column to a value in Fahrenheit. Each user-defined function carries certain properties or characteristics. The SQL standard defines the following properties: Language - defines the programming language in which the user-defined function is implemented; examples include SQL, C, C# and Java. Parameter style - defines the conventions that are used to pass the function parameters and results between the implementation of the function and the database system (only applicable if language is not SQL). Specific name - a name for the function that is unique within the database. Note that the function name does not have to be unique, considering overloaded functions. Some SQL implementations require that function names are unique within a database, and overloaded functions are not allowed. Determinism - specifies whether the function is deterministic or not. The determinism characteristic has an influence on the query optimizer when compiling a SQL statement. SQL-data access - tells the database management system whether the function contains no SQL statements (NO SQL), contains SQL statements but does not access any tables or views (CONTAINS SQL), reads data from tables or views (READS SQL DATA), or actually modifies data in the database (MODIFIES SQL DATA). User-defined functions should not be confused with stored procedures. Stored procedures allow the user to group a set of SQL commands. A procedure can accept parameters and execute its SQL statements depending on those parameters. A procedure is not an expression and, thus, cannot be used like user-defined functions. Some database management systems allow the creation of user defined functions in languages other than SQL. Microsoft SQL Server, for example, allows the user to use .NET languages including C# for this purpose. DB2 and Oracle support user-defined functions written in C or Java programming languages. === SQL Server 2000 === There are three types of UDF in Microsoft SQL Server 2000: scalar functions, inline table-valued functions, and multistatement table-valued functions. Scalar functions return a single data value (not a table) with RETURNS clause. Scalar functions can use all scalar data types, with exception of timestamp and user-defined data types. Inline table-valued functions return the result set of a single SELECT statement. Multistatement table-valued functions return a table, which was built with many TRANSACT-SQL statements. User-defined functions can be invoked from a query like built‑in functions such as OBJECT_ID, LEN, DATEDIFF, or can be executed through an EXECUTE statement like stored procedures. Performance Notes: User-defined functions are subroutines made of one or more Transact-SQL statements that can be used to encapsulate code for reuse. It takes zero or more arguments and evaluates a return value. Has both control-flow and DML statements in its body similar to stored procedures. Does not allow changes to any Global Session State, like modifications to database or external resource, such as a file or network. Does not support output parameter. DEFAULT keyword must be specified to pass the default value of parameter. Errors in UDF cause UDF to abort which, in turn, aborts the statement that invoked the UDF. === Apache Hive === Apache Hive defines, in addition to the regular user-defined functions (UDF), also user-defined aggregate functions (UDAF) and table-generating functions (UDTF). Hive enables developers to create their own custom functions with Java. === Apache Doris === Apache Doris, an open-source real-time analytical database, allows external users to contribute their own UDFs written in C++ to it.
Read more →
TiDB (; "Ti" stands for Titanium) is an open-source NewSQL database that supports Hybrid Transactional and Analytical Processing (HTAP) workloads. Designed to be MySQL compatible, it is developed and supported primarily by PingCAP and licensed under Apache 2.0. It is also available as a paid product. TiDB drew its initial design inspiration from Google's Spanner and F1 papers. == Release history == See all TiDB release notes. On December 19, 2024, TiDB 8.5 GA was released. On May 24, 2024, TiDB 8.1 GA was released. On December 1, 2023, TiDB 7.5 GA was released. On May 31, 2023, TiDB 7.1 GA was released. On April 7, 2022, TiDB 6.0 GA was released. On April 7, 2021 TiDB 5.0 GA was released. On May 28, 2020, TiDB 4.0 GA was released. On June 28, 2019, TiDB 3.0 GA was released. On April 27, 2018, TiDB 2.0 GA was released. On October 16, 2017, TiDB 1.0 GA was released. == Main features == === Horizontal scalability === TiDB can expand both SQL processing and storage capacity by adding new nodes. === MySQL compatibility === TiDB acts like it is a MySQL 8.0 server to applications. A user can continue to use all of the existing MySQL client libraries. Because TiDB's SQL processing layer is built from scratch, it is not a MySQL fork. === Distributed transactions with strong consistency === TiDB internally shards a table into small range-based chunks that are referred to as "Regions". Each Region defaults to approximately 100 MB in size, and TiDB uses a two-phase commit internally to ensure that regions are maintained in a transactionally consistent way. === Cloud native === TiDB is designed to work in the cloud. The storage layer of TiDB, called TiKV, became a Cloud Native Computing Foundation (CNCF) member project in August 2018, as a Sandbox level project, and became an incubation-level hosted project in May 2019. TiKV graduated from CNCF in September 2020. === Real-time HTAP === TiDB can support both online transaction processing (OLTP) and online analytical processing (OLAP) workloads. TiDB has two storage engines: TiKV, a rowstore, and TiFlash, a columnstore. === High availability === TiDB uses the Raft consensus algorithm to ensure that data is available and replicated throughout storage in Raft groups. In the event of failure, a Raft group will automatically elect a new leader for the failed member, and self-heal the TiDB cluster. === Vector Search === TiDB has a vector data type and vector indexes. This allows TiDB to be used as Vector database in AI Retrieval-augmented generation applications. == Deployment methods == === Kubernetes with Operator === TiDB can be deployed in a Kubernetes-enabled cloud environment by using TiDB Operator. An Operator is a method of packaging, deploying, and managing a Kubernetes application. It is designed for running stateful workloads and was first introduced by CoreOS in 2016. TiDB Operator was originally developed by PingCAP and open-sourced in August, 2018. TiDB Operator can be used to deploy TiDB on a laptop, Google Cloud Platform’s Google Kubernetes Engine, and Amazon Web Services’ Elastic Container Service for Kubernetes. === TiUP === TiDB 4.0 introduces TiUP, a cluster operation and maintenance tool. It helps users quickly install and configure a TiDB cluster with a few commands. == Tools == TiDB has a series of open-source tools built around it to help with data replication and migration for existing MySQL and MariaDB users. === TiDB Data Migration (DM) === TiDB Data Migration (DM) is suited for replicating data from already sharded MySQL or MariaDB tables to TiDB. A common use case of DM is to connect MySQL or MariaDB tables to TiDB, treating TiDB almost as a slave, then directly run analytical workloads on this TiDB cluster in near real-time. === Backup & Restore === Backup & Restore (BR) is a distributed backup and restore tool for TiDB cluster data. === Dumpling === Dumpling is a data export tool that exports data stored in TiDB or MySQL. It lets users make logical full backups or full dumps from TiDB or MySQL. === TiDB Lightning === TiDB Lightning is a tool that supports high speed full-import of a large MySQL dump into a new TiDB cluster. This tool is used to populate an initially empty TiDB cluster with much data, in order to speed up testing or production migration. The import speed improvement is achieved by parsing SQL statements into key-value pairs, then directly generate Sorted String Table (SST) files to RocksDB. === TiCDC === TiCDC is a change data capture tool which streams data from TiDB to other systems like Apache Kafka.
Read more →
In machine learning, feature hashing, also known as the hashing trick (by analogy to the kernel trick), is a fast and space-efficient way of vectorizing features, i.e. turning arbitrary features into indices in a vector or matrix. It works by applying a hash function to the features and using their hash values as indices directly (after a modulo operation), rather than looking the indices up in an associative array. In addition to its use for encoding non-numeric values, feature hashing can also be used for dimensionality reduction. This trick is often attributed to Weinberger et al. (2009), but there exists a much earlier description of this method published by John Moody in 1989. == Motivation == === Motivating example === In a typical document classification task, the input to the machine learning algorithm (both during learning and classification) is free text. From this, a bag of words (BOW) representation is constructed: the individual tokens are extracted and counted, and each distinct token in the training set defines a feature (independent variable) of each of the documents in both the training and test sets. Machine learning algorithms, however, are typically defined in terms of numerical vectors. Therefore, the bags of words for a set of documents is regarded as a term-document matrix where each row is a single document, and each column is a single feature/word; the entry i, j in such a matrix captures the frequency (or weight) of the j'th term of the vocabulary in document i. (An alternative convention swaps the rows and columns of the matrix, but this difference is immaterial.) Typically, these vectors are extremely sparse—according to Zipf's law. The common approach is to construct, at learning time or prior to that, a dictionary representation of the vocabulary of the training set, and use that to map words to indices. Hash tables and tries are common candidates for dictionary implementation. E.g., the three documents John likes to watch movies. Mary likes movies too. John also likes football. can be converted, using the dictionary to the term-document matrix ( John likes to watch movies Mary too also football 1 1 1 1 1 0 0 0 0 0 1 0 0 1 1 1 0 0 1 1 0 0 0 0 0 1 1 ) {\displaystyle {\begin{pmatrix}{\textrm {John}}&{\textrm {likes}}&{\textrm {to}}&{\textrm {watch}}&{\textrm {movies}}&{\textrm {Mary}}&{\textrm {too}}&{\textrm {also}}&{\textrm {football}}\\1&1&1&1&1&0&0&0&0\\0&1&0&0&1&1&1&0&0\\1&1&0&0&0&0&0&1&1\end{pmatrix}}} (Punctuation was removed, as is usual in document classification and clustering.) The problem with this process is that such dictionaries take up a large amount of storage space and grow in size as the training set grows. On the contrary, if the vocabulary is kept fixed and not increased with a growing training set, an adversary may try to invent new words or misspellings that are not in the stored vocabulary so as to circumvent a machine learned filter. To address this challenge, Yahoo! Research attempted to use feature hashing for their spam filters. Note that the hashing trick isn't limited to text classification and similar tasks at the document level, but can be applied to any problem that involves large (perhaps unbounded) numbers of features. === Mathematical motivation === Mathematically, a token is an element t {\displaystyle t} in a finite (or countably infinite) set T {\displaystyle T} . Suppose we only need to process a finite corpus, then we can put all tokens appearing in the corpus into T {\displaystyle T} , meaning that T {\displaystyle T} is finite. However, suppose we want to process all possible words made of the English letters, then T {\displaystyle T} is countably infinite. Most neural networks can only operate on real vector inputs, so we must construct a "dictionary" function ϕ : T → R n {\displaystyle \phi :T\to \mathbb {R} ^{n}} . When T {\displaystyle T} is finite, of size | T | = m ≤ n {\displaystyle |T|=m\leq n} , then we can use one-hot encoding to map it into R n {\displaystyle \mathbb {R} ^{n}} . First, arbitrarily enumerate T = { t 1 , t 2 , . . , t m } {\displaystyle T=\{t_{1},t_{2},..,t_{m}\}} , then define ϕ ( t i ) = e i {\displaystyle \phi (t_{i})=e_{i}} . In other words, we assign a unique index i {\displaystyle i} to each token, then map the token with index i {\displaystyle i} to the unit basis vector e i {\displaystyle e_{i}} . One-hot encoding is easy to interpret, but it requires one to maintain the arbitrary enumeration of T {\displaystyle T} . Given a token t ∈ T {\displaystyle t\in T} , to compute ϕ ( t ) {\displaystyle \phi (t)} , we must find out the index i {\displaystyle i} of the token t {\displaystyle t} . Thus, to implement ϕ {\displaystyle \phi } efficiently, we need a fast-to-compute bijection h : T → { 1 , . . . , m } {\displaystyle h:T\to \{1,...,m\}} , then we have ϕ ( t ) = e h ( t ) {\displaystyle \phi (t)=e_{h(t)}} . In fact, we can relax the requirement slightly: It suffices to have a fast-to-compute injection h : T → { 1 , . . . , n } {\displaystyle h:T\to \{1,...,n\}} , then use ϕ ( t ) = e h ( t ) {\displaystyle \phi (t)=e_{h(t)}} . In practice, there is no simple way to construct an efficient injection h : T → { 1 , . . . , n } {\displaystyle h:T\to \{1,...,n\}} . However, we do not need a strict injection, but only an approximate injection. That is, when t ≠ t ′ {\displaystyle t\neq t'} , we should probably have h ( t ) ≠ h ( t ′ ) {\displaystyle h(t)\neq h(t')} , so that probably ϕ ( t ) ≠ ϕ ( t ′ ) {\displaystyle \phi (t)\neq \phi (t')} . At this point, we have just specified that h {\displaystyle h} should be a hashing function. Thus we reach the idea of feature hashing. == Algorithms == === Feature hashing (Weinberger et al. 2009) === The basic feature hashing algorithm presented in (Weinberger et al. 2009) is defined as follows. First, one specifies two hash functions: the kernel hash h : T → { 1 , 2 , . . . , n } {\displaystyle h:T\to \{1,2,...,n\}} , and the sign hash ζ : T → { − 1 , + 1 } {\displaystyle \zeta :T\to \{-1,+1\}} . Next, one defines the feature hashing function: ϕ : T → R n , ϕ ( t ) = ζ ( t ) e h ( t ) {\displaystyle \phi :T\to \mathbb {R} ^{n},\quad \phi (t)=\zeta (t)e_{h(t)}} Finally, extend this feature hashing function to strings of tokens by ϕ : T ∗ → R n , ϕ ( t 1 , . . . , t k ) = ∑ j = 1 k ϕ ( t j ) {\displaystyle \phi :T^{}\to \mathbb {R} ^{n},\quad \phi (t_{1},...,t_{k})=\sum _{j=1}^{k}\phi (t_{j})} where T ∗ {\displaystyle T^{}} is the set of all finite strings consisting of tokens in T {\displaystyle T} . Equivalently, ϕ ( t 1 , . . . , t k ) = ∑ j = 1 k ζ ( t j ) e h ( t j ) = ∑ i = 1 n ( ∑ j : h ( t j ) = i ζ ( t j ) ) e i {\displaystyle \phi (t_{1},...,t_{k})=\sum _{j=1}^{k}\zeta (t_{j})e_{h(t_{j})}=\sum _{i=1}^{n}\left(\sum _{j:h(t_{j})=i}\zeta (t_{j})\right)e_{i}} ==== Geometric properties ==== We want to say something about the geometric property of ϕ {\displaystyle \phi } , but T {\displaystyle T} , by itself, is just a set of tokens, we cannot impose a geometric structure on it except the discrete topology, which is generated by the discrete metric. To make it nicer, we lift it to T → R T {\displaystyle T\to \mathbb {R} ^{T}} , and lift ϕ {\displaystyle \phi } from ϕ : T → R n {\displaystyle \phi :T\to \mathbb {R} ^{n}} to ϕ : R T → R n {\displaystyle \phi :\mathbb {R} ^{T}\to \mathbb {R} ^{n}} by linear extension: ϕ ( ( x t ) t ∈ T ) = ∑ t ∈ T x t ζ ( t ) e h ( t ) = ∑ i = 1 n ( ∑ t : h ( t ) = i x t ζ ( t ) ) e i {\displaystyle \phi ((x_{t})_{t\in T})=\sum _{t\in T}x_{t}\zeta (t)e_{h(t)}=\sum _{i=1}^{n}\left(\sum _{t:h(t)=i}x_{t}\zeta (t)\right)e_{i}} There is an infinite sum there, which must be handled at once. There are essentially only two ways to handle infinities. One may impose a metric, then take its completion, to allow well-behaved infinite sums, or one may demand that nothing is actually infinite, only potentially so. Here, we go for the potential-infinity way, by restricting R T {\displaystyle \mathbb {R} ^{T}} to contain only vectors with finite support: ∀ ( x t ) t ∈ T ∈ R T {\displaystyle \forall (x_{t})_{t\in T}\in \mathbb {R} ^{T}} , only finitely many entries of ( x t ) t ∈ T {\displaystyle (x_{t})_{t\in T}} are nonzero. Define an inner product on R T {\displaystyle \mathbb {R} ^{T}} in the obvious way: ⟨ e t , e t ′ ⟩ = { 1 , if t = t ′ , 0 , else. ⟨ x , x ′ ⟩ = ∑ t , t ′ ∈ T x t x t ′ ⟨ e t , e t ′ ⟩ {\displaystyle \langle e_{t},e_{t'}\rangle ={\begin{cases}1,{\text{ if }}t=t',\\0,{\text{ else.}}\end{cases}}\quad \langle x,x'\rangle =\sum _{t,t'\in T}x_{t}x_{t'}\langle e_{t},e_{t'}\rangle } As a side note, if T {\displaystyle T} is infinite, then the inner product space R T {\displaystyle \mathbb {R} ^{T}} is not complete. Taking its completion would get us to a Hilbert space, which allows well-behaved infinite sums. Now we have an inner product space, with enough structure to describe the geometry of the feature hashing function ϕ : R T → R n {\displaystyle \phi :\ma
Read more →
Collision detection is the computational problem of detecting an intersection of two or more objects in virtual space. More precisely, it deals with the questions of if, when, and where two or more objects intersect. Collision detection is a classic problem of computational geometry with applications in computer graphics, physical simulation, video games, robotics (including autonomous driving), and computational physics. Collision detection algorithms can be divided into operating on 2D or 3D spatial objects. == Overview == Collision detection is closely linked to calculating the distance between objects, as objects collide when the distance between them is less than or equal to zero. Negative distances indicate that one object has penetrated another. Performing collision detection requires more context than just the distance between the objects. Accurately identifying the points of contact on both objects' surfaces is also essential for computing a physically accurate collision response. The complexity of this task increases with the level of detail in the objects' representations: the more intricate the model, the greater the computational cost. Collision detection frequently involves dynamic objects, adding a temporal dimension to distance calculations. Instead of simply measuring distance between static objects, collision detection algorithms often aim to determine whether the objects' motion will bring them to a point in time when their distance is zero—an operation that adds significant computational overhead. In collision detection involving multiple objects, a naive approach would require detecting collisions for all pairwise combinations of objects. As the number of objects increases, the number of required comparisons grows rapidly: for n {\displaystyle n} objects, n ( n − 1 ) / 2 {n(n-1)}/{2} intersection tests are needed with a naive approach. This quadratic growth makes such an approach computationally expensive as n {\displaystyle n} increases. Due to the complexity mentioned above, collision detection is a computationally intensive process. Nevertheless, it is essential for interactive applications like video games, robotics, and real-time physics engines. To manage these computational demands, extensive efforts have gone into optimizing collision detection algorithms. A commonly used approach towards accelerating the required computations is to divide the process into two phases: the broad phase and the narrow phase. The broad phase aims to answer the question of whether objects might collide, using a conservative but efficient approach to rule out pairs that clearly do not intersect, thus avoiding unnecessary calculations. Objects that cannot be definitively separated in the broad phase are passed to the narrow phase. Here, more precise algorithms determine whether these objects actually intersect. If they do, the narrow phase often calculates the exact time and location of the intersection. == Broad phase == This phase aims at quickly finding objects or parts of objects for which it can be quickly determined that no further collision test is needed. A useful property of such approach is that it is output sensitive. In the context of collision detection this means that the time complexity of the collision detection is proportional to the number of objects that are close to each other. An early example of that is the I-COLLIDE where the number of required narrow phase collision tests was O ( n + m ) {\displaystyle O(n+m)} where n {\displaystyle n} is the number of objects and m {\displaystyle m} is the number of objects at close proximity. This is a significant improvement over the quadratic complexity of the naive approach. === Spatial partitioning === Several approaches can be grouped under the spatial partitioning umbrella, which includes octrees (for 3D), quadtrees (for 2D), binary space partitioning (or BSP trees) and other, similar approaches. If one splits space into a number of simple cells, and if two objects can be shown not to be in the same cell, then they need not be checked for intersection. Dynamic scenes and deformable objects require updating the partitioning which can add overhead. === Bounding volume hierarchy === Bounding Volume Hierarchy (BVH) is a tree structure over a set of bounding volumes. Collision is determined by doing a tree traversal starting from the root. If the bounding volume of the root doesn't intersect with the object of interest, the traversal can be stopped. If, however there is an intersection, the traversal proceeds and checks the branches for each there is an intersection. Branches for which there is no intersection with the bounding volume can be culled from further intersection test. Therefore, multiple objects can be determined to not intersect at once. BVH can be used with deformable objects such as cloth or soft-bodies but the volume hierarchy has to be adjusted as the shape deforms. For deformable objects we need to be concerned about self-collisions or self intersections. BVH can be used for that end as well. Collision between two objects is computed by computing intersection between the bounding volumes of the root of the tree as there are collision we dive into the sub-trees that intersect. Exact collisions between the actual objects, or its parts (often triangles of a triangle mesh) need to be computed only between intersecting leaves. The same approach works for pair wise collision and self-collisions. === Exploiting temporal coherence === During the broad-phase, when the objects in the world move or deform, the data-structures used to cull collisions have to be updated. In cases where the changes between two frames or time-steps are small and the objects can be approximated well with axis-aligned bounding boxes, the sweep and prune algorithm can be a suitable approach. Several key observation make the implementation efficient: Two bounding-boxes intersect if, and only if, there is overlap along all three axes; overlap can be determined, for each axis separately, by sorting the intervals for all the boxes; and lastly, between two frames updates are typically small (making sorting algorithms optimized for almost-sorted lists suitable for this application). The algorithm keeps track of currently intersecting boxes, and as objects move, re-sorting the intervals helps keep track of the status. === Pairwise pruning === Once a pair of physical bodies has been selected for further investigation, collisions need to be checked more carefully. However, in many applications, individual objects (if they are not too deformable) are described by a set of smaller primitives, mainly triangles. So there are two sets of triangles, S = S 1 , S 2 , … , S n {\displaystyle S={S_{1},S_{2},\dots ,S_{n}}} and T = T 1 , T 2 , … , T n {\displaystyle T={T_{1},T_{2},\dots ,T_{n}}} (for simplicity, each set has the same number of triangles.) The obvious thing to do is to check all triangles S j {\displaystyle S_{j}} against all triangles T k {\displaystyle T_{k}} for collisions, but this involves n 2 {\displaystyle n^{2}} comparisons, which is highly inefficient. If possible, it is desirable to use a pruning algorithm to reduce the number of pairs of triangles that need to be checked. The most widely used family of algorithms is known as the hierarchical bounding volumes method. As a preprocessing step, for each object (e.g., S {\displaystyle S} and T {\displaystyle T} ) calculates a hierarchy of bounding volumes. Then, at each time step, when collisions need to be checked between S {\displaystyle S} and T {\displaystyle T} , the hierarchical bounding volumes are used to reduce the number of pairs of triangles under consideration. For simplicity, provide an example using bounding spheres, although it has been noted that spheres are undesirable in many cases. If E {\displaystyle E} is a set of triangles, a bounding sphere is pre-calculated. B ( E ) {\displaystyle B(E)} . There are many ways of choosing B ( E ) {\displaystyle B(E)} , B ( E ) {\displaystyle B(E)} is a sphere that completely contains E {\displaystyle E} and is as small as possible. Ahead of time, B ( S ) {\displaystyle B(S)} and B ( T ) {\displaystyle B(T)} can be computed. Clearly, if these two spheres do not intersect (and that is very easy to test), then neither do S {\displaystyle S} and T {\displaystyle T} . This is not much better than an n-body pruning algorithm, however. If E = E 1 , E 2 , … , E m {\displaystyle E={E_{1},E_{2},\dots ,E_{m}}} is a set of triangles, then split it into two halves L ( E ) := E 1 , E 2 , … , E m / 2 {\displaystyle L(E):={E_{1},E_{2},\dots ,E_{m/2}}} and R ( E ) := E m / 2 + 1 , … , E m − 1 , E m {\displaystyle R(E):={E_{m/2+1},\dots ,E_{m-1},E_{m}}} . Apply this to S {\displaystyle S} and T {\displaystyle T} , and calculate (ahead of time) the bounding spheres B ( L ( S ) ) , B ( R ( S ) ) {\displaystyle B(L(S)),B(R(S))} and B ( L ( T ) ) , B ( R ( T ) ) {\displaystyle B(L(T)),B(R(T))} . T
Read more →
Taimi ( TAY-mee) is a dating app that caters to the LGBTQI+ community. The network matches its registered users based on their selected preferences and location. Originally an online dating service for gay men, by 2022 Taimi had become an app for all members of the LGBTQI+ community. It operates in more than 138 countries, including the US, UK, the Netherlands, Spain, Central and South America, Ukraine, and other European and Asian countries. Taimi runs on iOS and Android. The mobile app has a free and subscription-based premium version and offers a number of services for communication, including live streaming, chatting, and video calling. There is also an active blog that regularly posts articles and news about events of interest to the LGBTQ+ community. The application does not provide for non-Google e-mail log option, either phone number or Facebook account, during the registration process. The data controller for the non EU/UK users is based in a company, called Social Impact Inc., with its registered address at 1180 North Town Center Drive Suite 100, Las Vegas, Nevada, 89144, United States of America. == History == Taimi was launched in 2017 by Social Impact, Inc. in Las Vegas. Its founder, Alex Pasykov, originally called the app "Tame Me," a name that gradually morphed into Taimi. Over time, Taimi expanded into other countries, and expanding its reach to the LGBTQ+ community, so that, by 2022, it was fully inclusive of the entire queer community. In November 2020 the app was redesigned, with a new interface, branding, and logo. As of 2024, there are over 25 million registered users of Taimi worldwide. Pasykov states that he is an ally of the LGBTQ+ community and that he is focused on, among other things, partnering with NGOs to fight Homophobia and "regressive policies and laws" that negatively impact the community. == Features == Users register on the app and complete a profile, including personal information and preferences for compatibility, dating style, and relationship goals. An algorithm then finds and presents recommendations that a user accepts or rejects. Users are then free to chat via text or video with people they have connected with. Safety and security features include a two-step authentication process and an automated account verification along with a clear reporting system when breaches or policy violations occur. User responses to new features and policies drive changes and modifications that are made to all aspects of the site. == Partnerships and Collaborations == Taimi has a long history of collaborations and partnerships in Pride events, both in the US and abroad, including fund-raising efforts. Taimi has partnered with Rakuten Viber to create a bot focused on educating its members on key LGBTQ+ topics and to allow queer Viber users to connect. In 2023, Taimi collaborated with the Known Agency in an "America the Beautiful" campaign to shine a spotlight on current anti-LGBTQ+ policies and laws in a number of US states, and to counter these by highlighting the values and freedoms upon which America was founded. The campaign was nominated for The Drum Awards in the category "OOH For Good" and honored with the ANA Multicultural Excellence Award. Taimi also partnered with Goodparts, a queer-owned and operated retailer, in a "Body Beautiful" campaign focused on love and acceptance of all body types. In this campaign, well-known LGBTQ+ artists are providing artwork for Goodpart's product packaging. From October 31 to December 13, 2023, Taimi showed the "Taimi Moments" video, created in collaboration with Raygun Agency, on large screens between performances of LGBTQ+ artists Doja Cat, Ice Spice, and Doechii on their Scarlet Tour. In spring 2024, Taimi launched Queer Paradise, a series of live events in Southern California to celebrate diversity, sexual exploration, and dating fluidity. Each event in the series was curated to give the full spectrum of groups within the LGBTQ+ community a space to express their authentic selves. Taimi's partners for Queer Paradise include Hawtmess Productions, Eden Entertainment Group, Hump Events, Girls Gays & Theys, Damn Good Dyke Nights, and Gaybors Agency. In summer 2024, with support from GLAAD, Taimi has updated features and self-expression tools to better serve the LGBTQ+ people seeking connection in the app. Taimi allowed members to select multiple sexualities, unified the list of sexualities across all genders, added more pronoun options, and created a more inclusive and improved list of subcategories for non-binary users. Also, in summer 2024, Taimi has partnered with gender-affirming underwear brand Urbody to release a capsule collection. Focused on gender inclusivity and sexual fluidity, the capsule collection includes a range of underwear and compression tops intended to promote "joy, self-love and empowerment."
Read more →