AI Art Pragmata

AI Art Pragmata — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  LIVAC Synchronous Corpus

  LIVAC is an uncommon language corpus dynamically maintained since 1995. Different from other existing corpora, LIVAC has adopted a rigorous and regular "Windows" approach in processing and filtering massive media texts from representative Chinese speech communities such as Beijing, Hong Kong, Macau, Taipei, Singapore, Shanghai, as well as Guangzhou, and Shenzhen. The contents are thus deliberately repetitive in most cases, represented by textual samples drawn from editorials, local and international news, cross-Taiwan Strait news, as well as news on finance, sports and entertainment. By 2023, more than 3 billion characters of news media texts have been filtered, of which 700 million characters have been processed and analyzed and have yielded an expanding Pan-Chinese dictionary of 2.5 million words from the Pan-Chinese printed media. Through rigorous analysis based on computational linguistic methodology, LIVAC has at the same time accumulated a large amount of accurate and meaningful statistical data on the Chinese language and on their diverse speech communities in the Pan-Chinese context, and the results show considerable and important long standing as well as evolving variations. The "Windows" approach is the most innovative feature of LIVAC and has enabled Pan-Chinese media texts to be quantitatively analyzed according to various attributes such as locations, time and subject domains. Thus, various types of comparative studies and applications in information technology as well as development of often related innovative applications have been possible. Moreover, LIVAC has allowed longitudinal developments to be taken into account, facilitating Key Word in Context (KWIC) search and comprehensive study of target words and their underlying concepts as well as linguistic structures over the past 25 years, based on the above mentioned variables of location, time and subject. Results from the extensive and accumulative data analysis contained in LIVAC have enabled the cultivation of textual databases of proper names, place names, organization names, new words, and bi-weekly and annual rosters of media figures. Related applications have included the establishment of verb and adjective databases, the formulation of sentiment indices, and related opinion mining, to measure and compare the popularity of global media figures in the Chinese media (LIVAC Annual Pan-Chinese Celebrity Rosters, later renamed as the Pan-Chinese Newsmaker Rosters). Notable among these are the decades long periodic reviews of the 25 years of annual pan-Chinese rosters since 2000 and compilation of new word databases (LIVAC Annual Pan-Chinese New Word Rosters). On this basis, the analysis of the emergence, diffusion and transformation of new words, and the publication of dictionaries of neologisms have been made possible. A recent focus is on the relative balance between disyllabic words and growing trisyllabic words in the Chinese language, and the comparative study of light verbs in three Chinese speech communities. as well as the link between the language use and use of language as a reflection of epochal change in China. A new LIVAC version 3.1 was launched in February 2024. == Corpus data processing == Accessing media texts, manual input, etc. Text unification including conversion from simplified to traditional Chinese characters, stored as Big5 and Unicode versions Automatic word segmentation Automatic alignment of parallel texts Manual verification, part-of-speech tagging Extraction of words and addition to regional sub-corpora Combination of regional sub-corpora to update the LIVAC corpus, and master lexical database == Labeling for data curation == Categories used include general terms and proper names, such as: general names, surnames, semi titles; geographical, organizations and commercial entities, etc.; time, prepositions, locations, etc.; stack-words; loanwords; case-word; numerals, etc. Construction of databases of proper names, place names, and specific terms, etc. Generate rosters: "new word rosters", "celebrity or media personality rosters", "place name rosters", compound words and matched words Other parts of speech tagging for sub-database, such as common nouns, numerals, numeral classifiers, different types of verbs, and of adjectives, pronouns, adverbs, prepositions, conjunctions, particles marking mood, onomatopoeia, interjection, etc. == Applications == Compilation of Pan-Chinese dictionaries or local dictionaries Information technology research, such as predictive Chinese text input for mobile phones, automatic speech to text conversion, opinion mining Comparative studies on linguistic and cultural developments in the Pan-Chinese regions, especially in a critical period of history in modern China. Language teaching and learning research, and speech-to-text conversion Customized service on linguistic research and lexical search for international corporations and government agencies The above applications are provided by the following functions: Word Segmentation Search Phrase Search Example Sentence Selection Multi-word Comparison Word Cloud

  Read more →
• 

  Social media measurement

  Social media measurement, also called social media controlling, is the management practice of evaluating successful social media communications of brands, companies, or other organizations. Key performance indicators may be measured by extracting information from social media channels, such as blogs, wikis, micro-blogs such as Twitter, social networking sites, or video/photo sharing websites, forums from time to time. It is also used by companies to gauge current trends in the industry. The process first gathers data from different websites and then performs analysis based on different metrics like time spent on the page, click through rate, content share, comments, text analytics to identify positive or negative emotions about the brand. Some other social media metrics include share of voice, owned mentions, and earned mentions. The social media measurement process starts with defining a goal that needs to be achieved and defining the expected outcome of the process. The expected outcome varies per the goal and is usually measured by a variety of metrics. This is followed by defining possible social strategies to be used to achieve the goal. Then the next step is designing strategies to be used and setting up configuration tools that ease the process of collecting the data. In the next step, strategies and tools are deployed in real-time. This step involves conducting Quality Assurance tests of the methods deployed to collect the data. And in the final step, data collected from the system is analyzed and if the need arises, it is refined on the run time to enhance the methodologies used. The last step ensures that the result obtained is more aligned with the goal defined in the first step. == Data Acquisition == Acquiring data from social media is in demand of an exploring the user participation and population with the purpose of retrieving and collecting so many kinds of data(ex: comments, downloads etc.). There are several prevalent techniques to acquire data such as Network traffic analysis, Ad-hoc application and Crawling Network Traffic Analysis - Network traffic analysis is the process of capturing network traffic and observing it closely to determine what is happening in the network. It is primarily done to improve the performance, security and other general management of the network. However concerned about the potential tort of privacy on the Internet, network traffic analysis is always restricted by the government. Furthermore, high-speed links are not adaptable to traffic analysis because of the possible overload problem according to the packet sniffing mechanism Ad-hoc Application - Ad-hoc application is a kind of application that provides services and games to social network users by developing the APIs offered by social network companies (Facebook Developer Platform). The infrastructure of Ad-hoc application allows the user to interact with the interface layer instead of the application servers. The API provides a path for application to access information after the user login. Moreover, the size of the data set collected vary with the popularity of the social media platform i.e. social media platforms having high number of users will have more data than platforms having less user base. Scraping is a process in which the APIs collect online data from social media. The data collected from Scraping is in raw format. However, having access to these types of data is a bit difficult because of its commercial value. Crawling - Crawling is a process in which a web crawler creates indexes of all the words in a web-page, stores them, then follows all the hyperlinks and indexes on that page and again stores them. It is the most popular technique for data acquisition and is also well known for its easy operation based on prevalent Object-Orientated Programming Language (Java or Python etc.). And most important, social network companies (YouTube, Flicker, Facebook, Instagram, etc.) are friendly to crawling techniques by providing public APIs == Applications == === For branding === Monitoring social media allows researchers to find insights into a brand's overall visibility on social media, to measure the impact of campaigns, to identify opportunities for engagement, to assess competitor activity and share of voice, and to detect impending crises. It can also provide valuable information about emerging trends and what consumers and clients think about specific topics, brands or products. This is the work of a cross-section of groups that include market researchers, PR staff, marketing teams, social-engagement, and community staff, agencies and sales teams. Several different providers have developed tools to facilitate the monitoring of a variety of social media channels - from blogging to internet video to internet forums. This allows companies to track what consumers say about their brands and actions. Companies can then react to these conversations and interact with consumers through social media platforms. === In government === Apart from commercial applications, social media monitoring has become a pervasive technique applied by public organizations and governments. Monitoring is a tradition within the public sector, and social-media monitoring provides a real-time approach to detecting and responding to social developments. Governments have come to realize the need for strategies to cope with surprises from the rapid expansion of public issues. Sobkowicz introduced a framework with three blocks of social-media opinion tracking, simulating and forecasting. It includes: real-time detection of emotions, topics and opinions information-flow modelling and agent-based simulation modeling of opinion networks Bekkers introduced the application of social media monitoring in the Netherlands. Public organizations in the Netherlands (such as the Tax Agency and the Education Ministry) have started to use social media monitoring to obtain better insights into the sentiments of target groups. On the one hand, the public sector will be enabled to provide timely and efficient answers to the public by using social media monitoring techniques, but on the other hand, they also have to deal with concerns about ethical issues such as transparency and privacy. == Quantifying social media == Social media management software (SMMS) is an application program or software that facilitates an organization's ability to successfully engage in social media across different communication channels. SMMS is used to monitor inbound and outbound conversations, support customer interaction, audit or document social marketing initiatives and evaluate the usefulness of a social media presence. It can be difficult to measure all social media conversations. Due to privacy settings and other issues, not all social media conversations can be found and reported by monitoring tools. However, whilst social media monitoring cannot give absolute figures, it can be extremely useful for identifying trends and for benchmarking, in addition to the uses mentioned above. These findings can, in turn, influence and shape future business decisions. In order to access social media data (posts, Tweets, and meta-data) and to analyze and monitor social media, many companies use software technologies built for business. These range from in-platform analytics dashboards to dedicated third-party platforms, which offer more advanced capabilities including cross-platform audience intelligence, sentiment analysis, and trend detection at scale. == Location-based == Most social media networks allow users to add a location to their posts (reference all of our feeds). The location can be classified as either 'at-the-location' or 'about-the-location'. "'At-the-location' services can be defined as services where location-based content is created at the geographic location. 'About-the-location' services can be defined as services which are referring to a particular location but the content is not necessarily created in this particular physical place." The added information available from geotagged (link to Geotagging article) posts means that they can be displayed on a map. This means that a location can be used as the start of a social media search rather than a keyword or hashtag. This has major implications for disaster relief, event monitoring, safety and security professionals since a large portion of their job is related to tracking and monitoring specific locations. == Technologies used == Various monitoring platforms use different technologies for social media monitoring and measurement. These technology providers may connect to the API provided by social platforms that are created for 3rd party developers to develop their own applications and services that access data. Facebook's Graph API is one such API that social media monitoring solution products would connect to pull data from. Some social media monitoring and analytics companies use calls to data providers each time an end-user d

  Read more →
• 

  Air Force Network

  Air Force Network (AFNet) is an Indian Air Force (IAF) owned, operated and managed digital information grid. The AFNet replaces the Indian Air Force's (IAF) old communication network set-up using the tropo-scatter technology of the 1950s making it a true net-centric combat force. The IAF project is part of the overall mission to network all three services; The Indian Army, The Indian Navy and The Indian Air Force. The former Defence Minister AK Antony inaugurated the IAF's the AFNET on 14 September 2010 dedicating it to the people of India, for their direct or indirect participation in the communication revolution. == Background == Armed Forces in India has been using troposcatters as primary means of military communications since the 1950s, thereby occupying huge and expensive 2G and 3G spectrums which otherwise could have been used for expanding and de-clogging the civilian wireless communication network. The rapid expansion of civilian mobile telephony leading to need for larger bandwidth for wireless communication and commercial need to operate the 3G network necessitated the Government of India to have the Indian Armed Forces vacate the spectrum occupied by them. Thus the government of India through Department of Telecommunication (DoT) started a project called "Network for Spectrum" to set up a fiber optics network for the exclusive use of Indian Armed Forces in exchange for spectrum being released by the Defence Forces. The aim of 'Network for Spectrum' being twofold - to facilitate the growth of national tele-density on the one hand, and ensuring modernization of defence communications with the state-of-the-art communication infrastructure, and to support net-centric military operations. The Department of Telecom and the Ministry of Defence signed the memorandum of understanding for vacating the spectrum and setting up dedicated network for the use of defence forces. In this MoU, DoT agreed to laying of 40,000 route kilometres of optical fibre cable connecting 219 Army stations, 33 Navy stations and 162 points for the Air Force. It further agreed to setting up an exclusive defence band and Defence Interest Zone along 100 km of the international border, where spectrum will be reserved only for use by the Armed Forces. The total cost of implementing "Network for Spectrum" project is estimated to be ₹ 10,000 crores. AFNet is Indian Air Force component of Digital Information Grid under "Network for Spectrum" project and the AFNet has been extended and connected to the Digital Information Grid Project under implementation for the Indian Navy and the Indian Army on 2015. == Project Origin == The Air Force Network (AFNet) had been developed by the Indian Air Force at a cost of ₹1,077 crore (US$235.53 million) in collaboration with HCL Technologies and Bharat Sanchar Nigam Limited. It will replace the Air Force's more than half-a-century-old telecom network. This project is part of the defence ministry's initiative to digitize the communication systems of the three armed forces under "Network for Spectrum" initiative to improve coordination among themselves and other Military and Strategic Institution. IAF was the first to complete this gigabyte digital information grid implemented under the AFNet project. AFNet will be connected and extended to a Unified Digital Grid encompassing all the legs of Indian Armed Forces. The then defence minister, A. K. Antony, inaugurated the AFNet, IAF's gigabyte digital information grid. The grid is aimed at improving the network-centric warfare capability of the Air Force. The event also saw the presence of other personalities including the then Minister of Communication & IT, A. Raja; the Marshal of the Air Force, Arjan Singh; the Chief of the Air Staff, the Chief of the Army Staff and other officials from the three services and members of the Industry. The event also featured a practice interception of a simulated aerial target by a MiG-29 which took off from an airbase in the Punjab sector using the AFNet capabilities. Further capabilities in line with network centric warfare were also demonstrated. This included sharing information, videos and pictures by operational assets and platforms like UAVs and AWACS to decision-makers who are several hundred kilometres apart. == Technology, Design & Structure == AFNet incorporates the latest traffic transportation technology in form of Internet Protocol (IP) packets over the network using Multiprotocol Label Switching (MPLS). A large Voice over Internet Protocol (VoIP) layer with stringent quality of service enforcement will facilitate robust, high quality voice, video and conferencing solutions. AFNet will prove to be an effective force multiplier for intelligence analysis, mission planning and control, post-mission feedback and related activities like maintenance, logistics and administration. A comprehensive design with multi-layer security precautions for “Defence in Depth” have been planned by incorporating encryption technologies, Intrusion Prevention Systems to ensure the resistance of the IT system against information manipulation and eavesdropping. The network is secured with a host of advanced state-of-the-art encryption technologies. It is designed for high reliability with redundancy built into the network design itself. The AFNet is also capable of transmitting video from unmanned surveillance aircraft (UAV), pictures from airborne warning and control systems (AWACS) to decision makers on the ground and providing intelligence inputs from remote areas. The AFNet is also expected to facilitate accelerated economic growth by providing radio frequency spectrum for telecommunication purposes. AFNET will be the largest Multi-protocol Label Switching (MPLS) network in the defence segment. == Demonstration == At the AFNet launch, the IAF showcased a practice interception of simulated enemy targets by a pair of Mig-29 fighter aircraft airborne from an advanced airbase in the Punjab sector using the gigabyte digital information grid. During the AFNet-assisted operations, the Indian fighter jets neutralised intruding targets in the western sector, which was played out live on the giant screens at the Air Force auditorium offering a glimpse of the harnessed potential of the system. The final orders for engaging the enemy targets were issued live by Antony, whose queries about how the operation went were responded to by the pilot as "excellent". Various other functionalities contributing towards Network Centric Warfare were also showcased. These consisted of facilitating video from Unmanned Aerial Vehicle (UAV), pictures from an AWACS aircraft to the decision-makers on ground sitting hundreds of kilometres away, providing intelligence inputs from far-flung areas at central locations seamlessly. This was possible mainly because of the robust networking platform provided by AFNet. == Integrated Air Command and Control System == Integrated Air Command and Control System (IACCS) is an automated command and control system for air defence operated by the Indian Air Force. IACCS operations rides the AFNET backbone integrating all ground-based and airborne sensors, air defense weapon systems and command and control (C2) nodes. Subsequent integration with other services networks and civil radars will provide an integrated Air Situation Picture to operators to carry out AD role. The project was envisaged in 1995 following the Purulia arms drop case and was a part of IAF’s first Air Power Doctrinal manual issued in the 2000s, later revised in 2022. The first node in the western sectors had been operationalised by September 2010. The first five nodes located in the western and south western sectors were commissioned in 2011. The Air Force was preparing to seek clearance for five further nodes which would cover the rest of the nation including the island territories. Through the IACCS, IAF will connect all of its space, air and ground assets quickly, for total awareness of a region. This will offer connectivity for all the ground platforms and airborne platforms (including AEW&C), as a part of the network centricity of IAF. The IACCS also facilitates real-time transport of images, data and voice, amongst satellites, aircraft and ground stations. By 2018, five IACCS nodes had been established including Barnala (Punjab), Wadsar (Gujarat), Aya Nagar (Delhi), Jodhpur (Rajasthan) and Ambala (Haryana). Following this, under Phase-II, 4 additional nodes and 10 sub-nodes are to be set up. The major nodes will be established in the Eastern, Central, Southern and Andaman and Nicobar sectors. The second phase will cost ₹8,000 crore (equivalent to ₹110 billion or US$1.1 billion in 2023). IACCS successfully integrated all operating radars, including its own, the Army's, and civilian ones, in 2023. This enabled the autonomous firing response capability to take down incoming missiles, aircraft, and UAVs. The Akashteer system of the Indian Army is being integrated with the IACCS

  Read more →
• 

  Hike Messenger

  Hike Messenger, aka Hike Sticker Chat, is a multifunctional Indian social media and social networking service offering instant messaging (IM) and Voice over IP (VoIP) services that was launched on December 11, 2012, by Kavin Bharti Mittal. Hike functioned through SMS. The app registration used a s‍tandard, one-time password (OTP) based authentication process. It was estimated to be worth $1.4 billion and had more than 100 million registered users. It went defunct on January 6, 2021, as they were unable to compete with global messaging platforms. The app re-appeared on google play store and apple app store on 19 September 2025. == History == Hike Messenger was launched on December 12, 2012, by its founder, Kavin Bharti Mittal. The majority of users were from India, with 80% under the age of 25. The company purchased startups like TinyMogul and Hoppr in 2015. After buying US-based free voice calling company Zip Phones, Hike provided VoIP calling services. On March 5, 2015, Hike launched the 'Great Indian Sticker Challenge' to create more stickers. In February 2017, Hike acquired the social networking app Pulse. From version 5.0, it became the first social messaging app to start a mobile payment service in India. The timeline feature came back after multiple user requests and the introduction of a personalized digital envelope called Blue Packets for sending monetary gifts through a built-in wallet. In 2017, the acquisition of Bengaluru-based startup Creo was announced to enable third-party developers to build services on top of the Hike platform. In 2018, Hike provided 1 billion users with internet access by targeting smaller cities. In January 2019, the company discarded the previous super-app approach, and began launching specialized apps for specific use-cases. In May 2019, Hike announced a collaboration with Indraprastha Institute of Information Technology, Delhi (IIIT-D) to develop a variety of machine learning models. In April 2019, the company launched its first standalone app, Hike Sticker Chat. A separate content app Hike News & Content was also launched. In 2021, Hike shut down its messaging service and shifted focus to gaming and community platforms. It launched Rush, a real-money gaming app featuring casual titles like ludo and carrom, which scaled to over 10 million users and generated more than US$500 million in gross revenue over four years. The company also introduced Vibe, an approval-only community app, as part of its pivot away from the super-app and messaging model. In September 2025, following the passage of the Promotion and Regulation of Online Gaming Act, which banned real-money gaming in India, Hike announced its complete closure. Founder Kavin Bharti Mittal stated that while the company had begun international expansion, scaling globally under the new regulatory regime would require a full reset that was not a viable use of capital or resources. On 19 September 2025, hike was relaunched on play store and app store by the name hike messenger. == Application == === Timeline of Features === On 15 April 2014, Hike introduced unlimited free SMS via a service called Hike Offline, through credits earned by users from regular chatting, as connectivity is still a major issue in many parts of India. In an attempt to appeal to its younger users, Hike introduced features that find resonance with the local market, such as Last Seen Privacy and localized sticker packs. It also introduced a two-way chat theme, allowing users to change the chat background for themselves and for their friends simultaneously. The app also started showing live Cricket scores in collaboration with Cricbuzz, as well as news, casual games, and social media feeds. Hike also added a file transfer service, allowing files less than 100MB of all formats, with a view on further increasing the size limit to 1 GB. With the launch of version 2.9.2.0 in January 2015, Hike implemented support for sending uncompressed images and a "quick upload" feature optimized for 2G speed. Later that month, Hike introduced a voice calling feature for its users. In September 2015, Hike launched free group call support with up to 100 people in a simultaneous conference call environment. In November 2016, Hike announced the launch of a feature called Stories that allows people to share real-life moments using fun live filters which automatically get deleted after 48 hours, and a new camera design with localized filters. Hike 4.0 launched on 26 August 2015 with the tagline 'Got a Gang? Get on Hike'. Hike 4.0 was an optimization-focused update, increasing the performance of the app on poor networks. It supported photo filters, doodles, and bite-sized news updates in under 100 characters. Hike launched News Feed with Hindi language support on 29 September 2015 to cater for the needs of the non-English population. Hike launched version 3.5 as the biggest update for Windows Phone 8.1 during December 2015 which changed the user interface for more simpler navigation, supported sending unlimited non-media files and documents of any format and better group admin settings. It also included ten brand new chat themes. Hike launched a microapp feature which was live for two days on 8 May 2016, as a Mother's Day special in which users could add images, quotes or messages as a token of love with customized e-cards and stickers on their timeline not only on Hike, but also on other platforms. On 26 October 2016, Hike Messenger rolled out the beta version of a video calling feature ahead of WhatsApp starting with the Android users which also lets recipients preview a video call before deciding to take it and is optimized to even work under 2G conditions. On 24 December 2016, Hike rolled out a short 20-second Video Stories feature that can be directly shared with friends or posted on a public timeline with different filters in collaboration with content creators with the same 48-hour time limit before being automatically deleted. The Stories feature continues to receive constant future updates to include and enable content, public story option, private user messaging and geo-tagging. In September 2017, Hike launched personalized sticker packs with 20,000+ graphical stickers for over 500 colleges that covered around 1,000 colleges by December 2018 across India which can be used across different geographies, and are highly customized for users with availability in 40+ local languages that support automatic sticker suggestions where the application suggests the best reply for any sticker message and also allows users to "nudge", a feature used to ping the receiver. Hike started supporting user comments on friend's posts, added a specific message reply function, a redesigned camera interface to support front flash and user mentions with the help of the @ symbol. In December, 2017, Hike launched group voting, bill splitting, checklists and event reminders for group chat that supports up to 1,000 users both on iOS and Android platform. Hike launched another feature called Hike Land, which is a virtual world with beta trial to start from March 2020, that will use Hike Moji where online users with their digital avatar can hang out with other users and will be built inside the Hike Sticker Chat application. It is mainly targeted but not restricted towards 16 to 21 years age group of people. Without unveiling much about Hike Land, a separate website has been created with option to reserve spots by giving details like name, gender and phone number that will link the user profile from the Hike Sticker Chat account though it is not a necessity. ==== Hike Direct ==== The Hike Direct feature is based on the technology known as WiFi Direct, which initially was also called WiFi P2P and got introduced to users by October 2015, which enables sharing of files such as music, apps, videos without a live internet connection within a 100-meter radius by creating a wireless network between two or more devices with a transfer speed of 100MB per minute. For privacy and security reasons, Hike didn't show the recipient's location or proximity and works only when two users are connected in the same room by adding one another into the contact list. ==== Hike Wallet ==== In June 2017, Hike announced the launch of version 5.0 with multiple new features like User Chat Themes, Night Mode and Magic Selfie. along with a built-in Wallet partnered with Yes Bank. This feature was first rolled out to Android users followed by iOS users at a later stage. Hike collaborated with Airtel Payment Bank to power its digital payment wallet by November 2017 where Hike users have access to Airtel Payments Bank's merchant & utility payment services and know your customer (KYC) infrastructure with 5 million transactions happening from services like recharge and P2P. Hike formed a partnership with Ola Cabs to bring a taxi and auto-rickshaw booking facility from 14 February 2018. With Hike Wallet facility users could now book bus tickets with 3

  Read more →
• 

  Separable filter

  A separable filter in image processing can be written as product of two more simple filters. Typically a 2-dimensional convolution operation is separated into two 1-dimensional filters. This reduces the computational costs on an N × M {\displaystyle N\times M} image with a m × n {\displaystyle m\times n} filter from O ( M ⋅ N ⋅ m ⋅ n ) {\displaystyle {\mathcal {O}}(M\cdot N\cdot m\cdot n)} down to O ( M ⋅ N ⋅ ( m + n ) ) {\displaystyle {\mathcal {O}}(M\cdot N\cdot (m+n))} . == Examples == 1. A two-dimensional smoothing filter: 1 3 [ 1 1 1 ] ∗ 1 3 [ 1 1 1 ] = 1 9 [ 1 1 1 1 1 1 1 1 1 ] {\displaystyle {\frac {1}{3}}{\begin{bmatrix}1\\1\\1\end{bmatrix}}{\frac {1}{3}}{\begin{bmatrix}1&1&1\end{bmatrix}}={\frac {1}{9}}{\begin{bmatrix}1&1&1\\1&1&1\\1&1&1\end{bmatrix}}} 2. Another two-dimensional smoothing filter with stronger weight in the middle: 1 4 [ 1 2 1 ] ∗ 1 4 [ 1 2 1 ] = 1 16 [ 1 2 1 2 4 2 1 2 1 ] {\displaystyle {\frac {1}{4}}{\begin{bmatrix}1\\2\\1\end{bmatrix}}{\frac {1}{4}}{\begin{bmatrix}1&2&1\end{bmatrix}}={\frac {1}{16}}{\begin{bmatrix}1&2&1\\2&4&2\\1&2&1\end{bmatrix}}} 3. The Sobel operator, used commonly for edge detection: [ 1 2 1 ] ∗ [ 1 0 − 1 ] = [ 1 0 − 1 2 0 − 2 1 0 − 1 ] {\displaystyle {\begin{bmatrix}1\\2\\1\end{bmatrix}}{\begin{bmatrix}1&0&-1\end{bmatrix}}={\begin{bmatrix}1&0&-1\\2&0&-2\\1&0&-1\end{bmatrix}}} This works also for the Prewitt operator. In the examples, there is a cost of 3 multiply–accumulate operations for each vector which gives six total (horizontal and vertical). This is compared to the nine operations for the full 3x3 matrix. Another notable example of a separable filter is the Gaussian blur whose performance can be greatly improved the bigger the convolution window becomes.

  Read more →
• 

  Pamphlet war

  A pamphlet war is a protracted argument or discussion through printed media, especially between the time the printing press became common, and when state intervention like copyright laws made such public discourse more difficult. The purpose was to defend or attack a certain perspective or idea. Pamphlet wars have occurred multiple times throughout history, as both social and political platforms. Pamphlet wars became viable platforms for this protracted discussion with the advent and spread of the printing press. Cheap printing presses, and increased literacy made the late 17th century a key stepping stone for the development of pamphlet wars, a period of prolific use of this type of debate. Over 2200 pamphlets were published between 1600–1715 alone. Pamphlet wars are generally credited for powering many key social changes of the era, including the Reformation and the Revolution Controversy, the English philosophical debate set off by the French Revolution. == History of the pamphlet in England == Throughout Europe in the 16th century, printed tracts were used to argue religious doctrine and foment support for religious causes. In England, Henry VIII used print literature to justify his break from the Catholic Church. During the subsequent reigns of Edward and Mary, print polemics escalated into propaganda warfare, as print media gained enormous potential to sway common opinion. By the 1560s, print was widely used to convey news. In 1562, the first pamphlets appeared, which discussed the English forces sent to aid the Protestant French Huguenots. In 1569, pamphlets reported the revolt of the Northern Earls and the subsequent Rebellion of the same year. In the 1580s, pamphlets began to replace broadsheet ballads as the means to convey information to the general public. Over the next century, the pamphlet became the principal means of garnering support for a cause or an idea, and was particularly influential during the English Civil Wars (1642-1651) and the Glorious Revolution of 1688. Through the ensuing decades, the pamphlet lost some popularity due to the emergence of newspapers and journals, but continued to be an important medium of public debate, as illustrated by the Revolution Controversy a full century later in the 1790s. == Pamphlet printing == Coming from a Latin word, "pamphlet" literally means "small book." In the early days of printing, the format of the book or pamphlet depended on the size of the paper used and the number of times it was folded. If a page was only folded once, it was called a folio. If it was folded twice, it was known as a quarto. An octave was a paper folded three times. A pamphlet was usually 1-12 sheets of paper folded in quarto, or 8-96 pages. It was sold for one or two pennies apiece. The printing of a pamphlet involved many people: the author, the printer, suppliers, print-makers, compositor, correctors, pressmen, binders, and distributors. Once the pamphleteer had written the pamphlet, it was sent to the printing house to be corrected, set into type, and printed. The papers were then given to the printer's warehouse-keeper, who bundled the copies and sent them to the bookseller, who was probably the one financing the printing. He was responsible to bind the pamphlets, usually by sewing them, and then sold them wholesale to individual bookselling vendors. The booksellers then sold them from a stall in the marketplace. == Pamphlet subjects == Pamphlets began as the means of conveyance for religious debates, and therefore religious topics were one of the main subjects they dealt with. The definition of a pamphlet came to mean a short work dealing with social, political, or religious issues. Typical topics included the Civil war, Church of England doctrines, Acts of Parliament, the Popish Plot (see below), the Stuart Era, and Cromwell propaganda. In addition, pamphlets were also used for romantic fiction, autobiography, scurrilous personal abuse, and social criticism. They contained much of the propaganda of the 17th century in the midst of the religious and political turmoil. They were also used for debates between the Puritans and the Anglican. During the Glorious Revolution, pamphlets were political weapons. == Authors == There were many authors of pamphlets. However some of the more popular authors include Daniel Defoe, Thomas Hobbes, Jonathan Swift, John Milton, and Samuel Pepys. Also included in the midst are Thomas Nashe, Joseph Addison, Richard Steele, and Matthew Prior. In 1591–1592, Robert Greene released a series of pamphlets which later inspired many other authors including Thomas Middleton and Thomas Dekker. == Critics == Pamphlets, along with their vast popularity, received criticism. There were many in the time period who believed that pamphlets were full of foolishness. They thought the pamphlets were not good enough literature and that they would turn people from "good" writing. They believed that pamphlets would be the end of the great volumes of literature and that great writing would be forgotten. == News reporting == Pamphlets made a great difference in the way news was reported to the general public. With the publication of pamphlets, it was no longer difficult for people to hear of events taking place far away. The closer the occurrence was to London, the easier and faster people heard of it. For example, the Battle of Edgehill took place on 23 October 1642. The first pamphlet reporting the incident was printed on 25 October 24 hours after some of the orders reported had been given. While not entirely accurate, and hurriedly made, the pamphlet nonetheless was able to tell the general public what had happened in the battle. A more accurate, specific, and readable account was available in a pamphlet printed on 26 October, and the "authorized" version was available only five days after the battle took place. == Marprelate pamphlets == In 1588, a series of pamphlets marked a turning point for the Puritans, dividing them from other Protestants in the country. The authors wrote under the pseudonym of Martin Marprelate and his two sons of the same name. The true identities of the authors were never discovered. The pamphlets aimed to provoke authorities to take action against censorship. The series was among the first to ask questions directly of its readers. == Early pamphlet wars == === Elizabethan pamphlet wars === As a means of forming or swaying public opinion, pamphlets like these had a part in influencing society, even as the content was itself influenced by society. During the 16th century and continuing for a short while in the early 17th century in England there was rise in the use of pamphlet wars to discuss a myriad of issues spanning from the civil war, to religious freedoms and the roles of women in society. The Queen herself participated in these discussions, making sure that she was widely read and understood by her people in order to gain favour and establish herself as the monarch despite being a woman. Examples of her use of this medium appear in To the Troops at Tilbury written in 1588, On Mary's Execution written in 1586, and many more. Another famous writer of this period to take advantage of the pamphlet was Emilia Lanier, famous for her arguments about the role of women. A common idea promoted by many literary works and the general attitude towards women, Lanier's work "Eve's Apology in Defence of Women" refuted the belief that Eve is responsible for the fall of man. A very uncommon and unpopular stance to take, Lanier accomplishes her defence through structuring it as an apology, one of the earliest subversive feminist texts. Similarly, Francis Bacon wrote his Essays to promote his idea of morality and other complicated social issues. For example, his work, "Of Love" examines the various understandings of the concept of love, particularly as it was perceived during the Elizabethan era. === Eikon Series === From 1649 until 1651, some five pamphlets were published in a debate about the execution of King Charles I of England (1600-1649). Prior to his execution, King Charles wrote the first pamphlet in the discussion, Eikon Basilike’’ (from the Greek “eikon” for image and “basileus” for king). The subtitle of this work - Portraiture of His Sacred Majesty in His Solitudes and Sufferings - indicates that Charles sought to portray himself as a martyr to the cause of regal prerogative. In the following months, several response pamphlets were published (collectively known as the "Eikon" series), including: Eikon Alethine, Eikon e Pistes, Eikonoklastes, and Eikon Aklastos,” alternately attacking or defending the king, his regicide, and his self-portrait in “Eikon Basilike.” == Popish Plot and Elizabeth Cellier == In the 1680s, after being acquitted of the "Meal-Tub Plot" for which she was accused, Elizabeth Cellier wrote Malice Defeated, which, along with The Matchless Picaro, sparked a pamphlet war surrounding debate of the ascension of a Catholic king to the thro

  Read more →
• 

  Clustered file system

  A clustered file system (CFS) is a file system which is shared by being simultaneously mounted on multiple servers. There are several approaches to clustering, most of which do not employ a clustered file system (only direct attached storage for each node). Clustered file systems can provide features like location-independent addressing and redundancy which improve reliability or reduce the complexity of the other parts of the cluster. Parallel file systems are a type of clustered file system that spread data across multiple storage nodes, usually for redundancy or performance. == Shared-disk file system == A shared-disk file system uses a storage area network (SAN) to allow multiple computers to gain direct disk access at the block level. Access control and translation from file-level operations that applications use to block-level operations used by the SAN must take place on the client node. The most common type of clustered file system, the shared-disk file system – by adding mechanisms for concurrency control – provides a consistent and serializable view of the file system, avoiding corruption and unintended data loss even when multiple clients try to access the same files at the same time. Shared-disk file-systems commonly employ some sort of fencing mechanism to prevent data corruption in case of node failures, because an unfenced device can cause data corruption if it loses communication with its sister nodes and tries to access the same information other nodes are accessing. The underlying storage area network may use any of a number of block-level protocols, including SCSI, iSCSI, HyperSCSI, ATA over Ethernet (AoE), Fibre Channel, network block device, and InfiniBand. There are different architectural approaches to a shared-disk filesystem. Some distribute file information across all the servers in a cluster (fully distributed). === Examples === == Distributed file systems == Distributed file systems do not share block level access to the same storage but use a network protocol. These are commonly known as network file systems, even though they are not the only file systems that use the network to send data. Distributed file systems can restrict access to the file system depending on access lists or capabilities on both the servers and the clients, depending on how the protocol is designed. The difference between a distributed file system and a distributed data store is that a distributed file system allows files to be accessed using the same interfaces and semantics as local files – for example, mounting/unmounting, listing directories, read/write at byte boundaries, system's native permission model. Distributed data stores, by contrast, require using a different API or library and have different semantics (most often those of a database). === Design goals === Distributed file systems may aim for "transparency" in a number of aspects. That is, they aim to be "invisible" to client programs, which "see" a system which is similar to a local file system. Behind the scenes, the distributed file system handles locating files, transporting data, and potentially providing other features listed below. Access transparency: clients are unaware that files are distributed and can access them in the same way as local files are accessed. Location transparency: a consistent namespace exists encompassing local as well as remote files. The name of a file does not give its location. Concurrency transparency: all clients have the same view of the state of the file system. This means that if one process is modifying a file, any other processes on the same system or remote systems that are accessing the files will see the modifications in a coherent manner. Failure transparency: the client and client programs should operate correctly after a server failure. Heterogeneity: file service should be provided across different hardware and operating system platforms. Scalability: the file system should work well in small environments (1 machine, a dozen machines) and also scale gracefully to bigger ones (hundreds through tens of thousands of systems). Replication transparency: Clients should not have to be aware of the file replication performed across multiple servers to support scalability. Migration transparency: files should be able to move between different servers without the client's knowledge. === History === The Incompatible Timesharing System used virtual devices for transparent inter-machine file system access in the 1960s. More file servers were developed in the 1970s. In 1976, Digital Equipment Corporation created the File Access Listener (FAL), an implementation of the Data Access Protocol as part of DECnet Phase II which became the first widely used network file system. In 1984, Sun Microsystems created the file system called "Network File System" (NFS) which became the first widely used Internet Protocol based network file system. Other notable network file systems are Andrew File System (AFS), Apple Filing Protocol (AFP), NetWare Core Protocol (NCP), and Server Message Block (SMB) which is also known as Common Internet File System (CIFS). In 1986, IBM announced client and server support for Distributed Data Management Architecture (DDM) for the System/36, System/38, and IBM mainframe computers running CICS. This was followed by the support for IBM Personal Computer, AS/400, IBM mainframe computers under the MVS and VSE operating systems, and FlexOS. DDM also became the foundation for Distributed Relational Database Architecture, also known as DRDA. There are many peer-to-peer network protocols for open-source distributed file systems for cloud or closed-source clustered file systems, e. g.: 9P, AFS, Coda, CIFS/SMB, DCE/DFS, WekaFS, Lustre, PanFS, Google File System, Mnet, Chord Project. === Examples === == Network-attached storage == Network-attached storage (NAS) provides both storage and a file system, like a shared disk file system on top of a storage area network (SAN). NAS typically uses file-based protocols (as opposed to block-based protocols a SAN would use) such as NFS (popular on UNIX systems), SMB/CIFS (Server Message Block/Common Internet File System) (used with MS Windows systems), AFP (used with Apple Macintosh computers), or NCP (used with OES and Novell NetWare). == Design considerations == === Avoiding single point of failure === The failure of disk hardware or a given storage node in a cluster can create a single point of failure that can result in data loss or unavailability. Fault tolerance and high availability can be provided through data replication of one sort or another, so that data remains intact and available despite the failure of any single piece of equipment. For examples, see the lists of distributed fault-tolerant file systems and distributed parallel fault-tolerant file systems. === Performance === A common performance measurement of a clustered file system is the amount of time needed to satisfy service requests. In conventional systems, this time consists of a disk-access time and a small amount of CPU-processing time. But in a clustered file system, a remote access has additional overhead due to the distributed structure. This includes the time to deliver the request to a server, the time to deliver the response to the client, and for each direction, a CPU overhead of running the communication protocol software. === Concurrency === Concurrency control becomes an issue when more than one person or client is accessing the same file or block and want to update it. Hence updates to the file from one client should not interfere with access and updates from other clients. This problem is more complex with file systems due to concurrent overlapping writes, where different writers write to overlapping regions of the file concurrently. This problem is usually handled by concurrency control or locking which may either be built into the file system or provided by an add-on protocol. == History == IBM mainframes in the 1970s could share physical disks and file systems if each machine had its own channel connection to the drives' control units. In the 1980s, Digital Equipment Corporation's TOPS-20 and OpenVMS clusters (VAX/ALPHA/IA64) included shared disk file systems.

  Read more →
• 

  Social media newsroom

  A social media newsroom is a company resource, set up to increase the functionality and usability of the traditional online newsroom. Social media newsrooms (SMNs) are intended to encourage dialogue and information sharing. Unlike online newsrooms, content is accessible to more than just journalists, but to all those with whom the company engages such as bloggers, their prospects, customers, business partners and investors. It gives these stakeholders access to news, public relations announcements, images, audio, video and other multimedia files. In addition to posting press releases and corporate news, companies can integrate other social content from sites such as YouTube, Flickr and Slideshow as well as streams from corporate Twitter accounts. Traditional tools for journalists such as corporate fast facts, leadership information, a multimedia library, financial information, awards and other recent media coverage are also included in an SMN. Examples of companies effectively using social media newsrooms include Opel Group, Pressat, First Direct, MyNewsdesk, Scania and Newport Beach.

  Read more →
• 

  XLNet

  The XLNet was an autoregressive Transformer designed as an improvement over BERT, with 340M parameters and trained on 33 billion words. It was released on 19 June 2019, under the Apache 2.0 license. It achieved state-of-the-art results on a variety of natural language processing tasks, including language modeling, question answering, and natural language inference. == Architecture == The main idea of XLNet is to model language autoregressively like the GPT models, but allow for all possible permutations of a sentence. Concretely, consider the following sentence:My dog is cute.In standard autoregressive language modeling, the model would be tasked with predicting the probability of each word, conditioned on the previous words as its context: We factorize the joint probability of a sequence of words x 1 , … , x T {\displaystyle x_{1},\ldots ,x_{T}} using the chain rule: Pr ( x 1 , … , x T ) = Pr ( x 1 ) Pr ( x 2 | x 1 ) Pr ( x 3 | x 1 , x 2 ) … Pr ( x T | x 1 , … , x T − 1 ) . {\displaystyle \Pr(x_{1},\ldots ,x_{T})=\Pr(x_{1})\Pr(x_{2}|x_{1})\Pr(x_{3}|x_{1},x_{2})\ldots \Pr(x_{T}|x_{1},\ldots ,x_{T-1}).} For example, the sentence "My dog is cute" is factorized as: Pr ( My , dog , is , cute ) = Pr ( My ) Pr ( dog | My ) Pr ( is | My , dog ) Pr ( cute | My , dog , is ) . {\displaystyle \Pr({\text{My}},{\text{dog}},{\text{is}},{\text{cute}})=\Pr({\text{My}})\Pr({\text{dog}}|{\text{My}})\Pr({\text{is}}|{\text{My}},{\text{dog}})\Pr({\text{cute}}|{\text{My}},{\text{dog}},{\text{is}}).} Schematically, we can write it as → My → My dog → My dog is → My dog is cute . {\displaystyle {\texttt {}}{\texttt {}}{\texttt {}}{\texttt {}}\to {\text{My }}{\texttt {}}{\texttt {}}{\texttt {}}\to {\text{My dog }}{\texttt {}}{\texttt {}}\to {\text{My dog is }}{\texttt {}}\to {\text{My dog is cute}}.} However, for XLNet, the model is required to predict the words in a randomly generated order. Suppose we have sampled a randomly generated order 3241, then schematically, the model is required to perform the following prediction task: → is → dog is → dog is cute → My dog is cute {\displaystyle {\texttt {}}{\texttt {}}{\texttt {}}{\texttt {}}\to {\texttt {}}{\texttt {}}{\text{is }}{\texttt {}}\to {\texttt {}}{\text{dog is }}{\texttt {}}\to {\texttt {}}{\text{dog is cute}}\to {\text{My dog is cute}}} By considering all permutations, XLNet is able to capture longer-range dependencies and better model the bidirectional context of words. === Two-Stream Self-Attention === To implement permutation language modeling, XLNet uses a two-stream self-attention mechanism. The two streams are: Content stream: This stream encodes the content of each word, as in standard causally masked self-attention. Query stream: This stream encodes the content of each word in the context of what has gone before. In more detail, it is a masked cross-attention mechanism, where the queries are from the query stream, and the key-value pairs are from the content stream. The content stream uses the causal mask M causal = [ 0 − ∞ − ∞ … − ∞ 0 0 − ∞ … − ∞ 0 0 0 … − ∞ ⋮ ⋮ ⋮ ⋱ ⋮ 0 0 0 … 0 ] {\displaystyle M_{\text{causal}}={\begin{bmatrix}0&-\infty &-\infty &\dots &-\infty \\0&0&-\infty &\dots &-\infty \\0&0&0&\dots &-\infty \\\vdots &\vdots &\vdots &\ddots &\vdots \\0&0&0&\dots &0\end{bmatrix}}} permuted by a random permutation matrix to P M causal P − 1 {\displaystyle PM_{\text{causal}}P^{-1}} . The query stream uses the cross-attention mask P ( M causal − ∞ I ) P − 1 {\displaystyle P(M_{\text{causal}}-\infty I)P^{-1}} , where the diagonal is subtracted away specifically to avoid the model "cheating" by looking at the content stream for what the current masked token is. Like the causal masking for GPT models, this two-stream masked architecture allows the model to train on all tokens in one forward pass. == Training == Two models were released: XLNet-Large, cased: 110M parameters, 24-layer, 1024-hidden, 16-heads XLNet-Base, cased: 340M parameters, 12-layer, 768-hidden, 12-heads. It was trained on a dataset that amounted to 32.89 billion tokens after tokenization with SentencePiece. The dataset was composed of BooksCorpus, and English Wikipedia, Giga5, ClueWeb 2012-B, and Common Crawl. It was trained on 512 TPU v3 chips, for 5.5 days. At the end of training, it still under-fitted the data, meaning it could have achieved lower loss with more training. It took 0.5 million steps with an Adam optimizer, linear learning rate decay, and a batch size of 8192.

  Read more →
• 

  Ultra (cryptography)

  Ultra was the designation adopted by British military intelligence in June 1941 for wartime signals intelligence obtained by breaking high-level encrypted enemy radio and teleprinter communications at the Government Code and Cypher School (GC&CS) at Bletchley Park. Ultra eventually became the standard designation among the western Allies for all such intelligence. The name arose because the intelligence obtained was considered more important than that designated by the highest British security classification then used (Most Secret) and so was regarded as being Ultra Secret. Several other cryptonyms had been used for such intelligence. The code name "Boniface" was used as a cover name for Ultra. In order to ensure that the successful code-breaking did not become apparent to the Germans, British intelligence created a fictional MI6 master spy, Boniface, who controlled a fictional series of agents throughout Germany. Information obtained through code-breaking was often attributed to the human intelligence from the Boniface network. The U.S. used the codename Magic for its decrypts from Japanese sources, including the "Purple" cipher. Much of the German cipher traffic was encrypted on the Enigma machine. Used properly, the German military Enigma would have been virtually unbreakable; in practice, shortcomings in operation allowed it to be broken. The term "Ultra" has often been used almost synonymously with "Enigma decrypts". However, Ultra also encompassed decrypts of the German Lorenz SZ 40/42 machines that were used by the German High Command, and the Hagelin machine. Many observers, at the time and later, regarded Ultra as immensely valuable to the Allies. Winston Churchill was reported to have told King George VI, when presenting to him Stewart Menzies (head of the Secret Intelligence Service and the person who controlled distribution of Ultra decrypts to the government): "It is thanks to the secret weapon of General Menzies, put into use on all the fronts, that we won the war!" F. W. Winterbotham quoted the western Supreme Allied Commander, Dwight D. Eisenhower, at war's end describing Ultra as having been "decisive" to Allied victory. Sir Harry Hinsley, Bletchley Park veteran and official historian of British Intelligence in World War II, made a similar assessment of Ultra, saying that while the Allies would have won the war without it, "the war would have been something like two years longer, perhaps three years longer, possibly four years longer than it was." However, Hinsley and others have emphasized the difficulties of counterfactual history in attempting such conclusions, and some historians, such as John Keegan, have said the shortening might have been as little as the three months it took the United States to deploy the atomic bomb. == Sources of intelligence == Most Ultra intelligence was derived from reading radio messages that had been encrypted with cipher machines, complemented by material from radio communications using traffic analysis and direction finding. In the early phases of the war, particularly during the eight-month Phoney War, the Germans could transmit most of their messages using land lines and so had no need to use radio. This meant that those at Bletchley Park had some time to build up experience of collecting and starting to decrypt messages on the various radio networks. German Enigma messages were the main source, with those of the German air force (the Luftwaffe) predominating, as they used radio more and their operators were particularly ill-disciplined. === German === ==== Enigma ==== "Enigma" refers to a family of electro-mechanical rotor cipher machines. These produced a polyalphabetic substitution cipher and were widely thought to be unbreakable in the 1920s, when a variant of the commercial Model D was first used by the Reichswehr. The German Army (Heer), Navy, Air Force, Nazi party, Gestapo and German diplomats used Enigma machines in several variants. Abwehr (German military intelligence) used a four-rotor machine without a plugboard and Naval Enigma used different key management from that of the army or air force, making its traffic far more difficult to cryptanalyse; each variant required different cryptanalytic treatment. The commercial versions were not as secure and Dilly Knox of GC&CS is said to have broken one before the war. German military Enigma was first broken in December 1932 by Marian Rejewski and the Polish Cipher Bureau, using a combination of brilliant mathematics, the services of a spy in the German office responsible for administering encrypted communications, and good luck. The Poles read Enigma to the outbreak of World War II and beyond, in France. At the turn of 1939, the Germans made the systems ten times more complex, which required a tenfold increase in Polish decryption equipment, which they could not meet. On 25 July 1939, the Polish Cipher Bureau handed reconstructed Enigma machines and their techniques for decrypting ciphers to the French and British. Gordon Welchman wrote, Ultra would never have got off the ground if we had not learned from the Poles, in the nick of time, the details both of the German military Enigma machine, and of the operating procedures that were in use. At Bletchley Park, some of the key people responsible for success against Enigma included mathematicians Alan Turing and Hugh Alexander and, at the British Tabulating Machine Company, chief engineer Harold Keen. After the war, interrogation of German cryptographic personnel led to the conclusion that German cryptanalysts understood that cryptanalytic attacks against Enigma were possible but were thought to require impracticable amounts of effort and investment. The Poles' early start at breaking Enigma and the continuity of their success gave the Allies an advantage when World War II began. ==== Lorenz cipher ==== In June 1941, the Germans started to introduce on-line stream cipher teleprinter systems for strategic point-to-point radio links, to which the British gave the code-name Fish. Several systems were used, principally the Lorenz SZ 40/42 (codenamed "Tunny" by the British) and Geheimfernschreiber ("Sturgeon"). These cipher systems were cryptanalysed, particularly Tunny, which the British thoroughly penetrated. It was eventually attacked using Colossus machines, which were the first digital programme-controlled electronic computers. In many respects the Tunny work was more difficult than for the Enigma, since the British codebreakers had no knowledge of the machine producing it and no head-start such as that the Poles had given them against Enigma. Although the volume of intelligence derived from this system was much smaller than that from Enigma, its importance was often far higher because it produced primarily high-level, strategic intelligence that was sent between Wehrmacht high command (Oberkommando der Wehrmacht, OKW). The eventual bulk decryption of Lorenz-enciphered messages contributed significantly, and perhaps decisively, to the defeat of Nazi Germany. Nevertheless, the Tunny story has become much less well known among the public than the Enigma one. At Bletchley Park, some of the key people responsible for success in the Tunny effort included mathematicians W. T. "Bill" Tutte and Max Newman and electrical engineer Tommy Flowers. === Italian === In June 1940, the Italians were using book codes for most of their military messages, except for the Italian Navy, which in early 1941 had started using a version of the Hagelin rotor-based cipher machine C-38. This was broken from June 1941 onwards by the Italian subsection of GC&CS at Bletchley Park. === Japanese === In the Pacific theatre, a Japanese cipher machine, called "Purple" by the Americans, was used for highest-level Japanese diplomatic traffic. It produced a polyalphabetic substitution cipher, but unlike Enigma, was not a rotor machine, being built around electrical stepping switches. It was broken by the US Army Signal Intelligence Service and disseminated as Magic. Detailed reports by the Japanese ambassador to Germany were encrypted on the Purple machine. His reports included reviews of German assessments of the military situation, reviews of strategy and intentions, reports on direct inspections by the ambassador (in one case, of Normandy beach defences), and reports of long interviews with Hitler. The Japanese are said to have obtained an Enigma machine in 1937, although it is debated whether they were given it by the Germans or bought a commercial version, which, apart from the plugboard and internal wiring, was the German Heer/Luftwaffe machine. Having developed a similar machine, the Japanese did not use the Enigma machine for their most secret communications. The chief fleet communications code system used by the Imperial Japanese Navy was called JN-25 by the Americans, and by early 1942 the US Navy had made considerable progress in decrypting Japanese naval messages. The US Army also made progress on the

  Read more →
• 

  BitFunnel

  BitFunnel is the search engine indexing algorithm and a set of components used in the Bing search engine, which were made open source in 2016. BitFunnel uses bit-sliced signatures instead of an inverted index in an attempt to reduce operations cost. == History == Progress on the implementation of BitFunnel was made public in early 2016, with the expectation that there would be a usable implementation later that year. In September 2016, the source code was made available via GitHub. A paper discussing the BitFunnel algorithm and implementation was released as through the Special Interest Group on Information Retrieval of the Association for Computing Machinery in 2017 and won the Best Paper Award. == Components == BitFunnel consists of three major components: BitFunnel – the text search/retrieval system itself WorkBench – a tool for preparing text for use in BitFunnel NativeJIT – a software component that takes expressions that use C data structures and transforms them into highly optimized assembly code == Algorithm == === Initial problem and solution overview === The BitFunnel paper describes the "matching problem", which occurs when an algorithm must identify documents through the usage of keywords. The goal of the problem is to identify a set of matches given a corpus to search and a query of keyword terms to match against. This problem is commonly solved through inverted indexes, where each searchable item is maintained with a map of keywords. In contrast, BitFunnel represents each searchable item through a signature. A signature is a sequence of bits which describe a Bloom filter of the searchable terms in a given searchable item. The bloom filter is constructed through hashing through several bit positions. === Theoretical implementation of bit-string signatures === The signature of a document (D) can be described as the logical-or of its term signatures: S D → = ⋃ t ∈ D S t → {\displaystyle {\overrightarrow {S_{D}}}=\bigcup _{t\in D}{\overrightarrow {S_{t}}}} Similarly, a query for a document (Q) can be defined as a union: S Q → = ⋃ t ∈ Q S t → {\displaystyle {\overrightarrow {S_{Q}}}=\bigcup _{t\in Q}{\overrightarrow {S_{t}}}} Additionally, a document D is a member of the set M' when the following condition is satisfied: S Q → ∩ S D → = S Q → {\displaystyle {\overrightarrow {S_{Q}}}\cap {\overrightarrow {S_{D}}}={\overrightarrow {S_{Q}}}} This knowledge is then combined to produce a formula where M' is identified by documents which match the query signature: M ′ = { D ∈ C ∣ S Q → ∩ S D → = S Q → } {\displaystyle M'=\left\{D\in C\mid {\overrightarrow {S_{Q}}}\cap {\overrightarrow {S_{D}}}={\overrightarrow {S_{Q}}}\right\}} These steps and their proofs are discussed in the 2017 paper. === Pseudocode for bit-string signatures === This algorithm is described in the 2017 paper. M ′ = ∅ foreach D ∈ C do if S D → ∩ S Q → = S Q → then M ′ = M ′ ∪ { D } endif endfor {\displaystyle {\begin{array}{l}M'=\emptyset \\{\texttt {foreach}}\ D\in C\ {\texttt {do}}\\\qquad {\texttt {if}}\ {\overrightarrow {S_{D}}}\cap {\overrightarrow {S_{Q}}}={\overrightarrow {S_{Q}}}\ {\texttt {then}}\\\qquad \qquad M'=M'\cup \{D\}\\\qquad {\texttt {endif}}\\{\texttt {endfor}}\end{array}}}

  Read more →
• 

  Data room

  Data rooms are secure spaces used for housing data, usually of a privileged or confidential nature. They can be physical data rooms, virtual data rooms (VDRs), or data centers. They are primarily used for a variety of corporate purposes, including data storage, document exchange, file sharing, financial transactions, and legal proceedings. Today, data rooms are central to workflows in mergers and acquisitions, venture capital, and corporate restructuring, increasingly utilizing artificial intelligence to securely manage and review large datasets. Historically, data rooms were strictly physical locations heavily guarded and monitored. Today, the vast majority of corporate data rooms are hosted virtually on secure cloud platforms, though physical rooms are still occasionally used for highly sensitive government or proprietary intelligence. == Physical Data Rooms == In mergers and acquisitions (M&A), the traditional data room genuinely consists of a physically secured and continually monitored room, normally in the vendor's offices or those of their legal counsel. Bidders and their advisers visit this room in order to inspect and report on various documents, legal contracts, and financial statements made available during the due diligence process. Historically, physical data rooms presented significant logistical challenges. Often, only one bidder at a time was allowed to enter to maintain document integrity and confidentiality. If new documents or new versions of documents were required, they had to be brought in by courier as hardcopies. Teams involved in large due diligence processes typically had to be flown in from many regions or countries and remain available throughout the process. Because these teams comprised a number of experts in different fields—such as legal counsel, forensic accountants, and industry specialists—the overall cost of keeping such groups on call near the physical data room was often extremely high. == Virtual Data Rooms (VDRs) == To address the costs and logistical bottlenecks of physical data rooms, virtual data rooms (VDRs) were developed to provide secure, online dissemination of confidential information. A VDR is essentially a secure cloud repository with strictly controlled access. Access is managed through secure log-ons supplied by the vendor or authority, which can be disabled at any time if a bidder withdraws from a transaction. Because much of the information released during corporate transactions is highly confidential, VDRs utilize digital rights management (DRM) to control information. Restrictions are applied to the viewers' ability to release data to third parties by disabling forwarding, copying, or printing capabilities. Modern VDRs also employ dynamic watermarking and detailed auditing capabilities. Detailed auditing is required for legal reasons so that a precise digital footprint is kept of who has viewed which version of each document, and for how long. Furthermore, modern VDR platforms are typically built to comply with stringent information security standards such as ISO 27001 and SOC 2. Transitioning from sequential physical data rooms to parallel virtual data rooms has been shown to significantly reduce the duration of M&A transactions while allowing sellers to field multiple bidders simultaneously. == Key Applications == Data rooms are commonly used by legal, accounting, investment banking, and private equity firms. Primary applications include: Mergers and Acquisitions (M&A): VDRs are central to the sell-side M&A process. After potential buyers sign a Non-Disclosure Agreement (NDA) and review a Confidential Information Memorandum (CIM), they are granted data room access to perform deep financial due diligence, such as Quality of Earnings (QoE) analysis and legal liability assessments. Venture Capital and Startups: Startups use data rooms as a centralized location for key operational data, capitalization tables, and financial projections to streamline due diligence for angel investors and venture capital firms during fundraising rounds. Initial Public Offerings (IPOs): Taking a company public requires intense regulatory scrutiny. Data rooms are used to securely share company histories and financial audits with investment bankers, legal teams, and regulatory bodies. Corporate Restructuring and Insolvency: During bankruptcies or corporate carve-outs, data rooms are used to organize outstanding debt profiles, creditor agreements, and operational liabilities. == Emerging Technologies == In recent years, the management of virtual data rooms has increasingly incorporated Artificial Intelligence (AI) and Machine Learning (ML). Generative AI and Natural Language Processing (NLP) tools are now integrated into VDRs to automatically index thousands of documents, perform auto-redaction of personally identifiable information (PII), and assist buy-side analysts in identifying hidden liabilities within unstructured text data during the due diligence phase. Modern AI algorithms can extract line items from financial statements to instantly populate structured databases.

  Read more →
• 

  Coalition for App Fairness

  The Coalition for App Fairness (CAF) is a coalition comprised by companies, who aim to reach a fairer deal for the inclusion of their apps into the Apple App Store or the Google Play Store. The organization's executive director is Meghan DiMuzio and its headquarters are located in Washington, D.C. == Background == In July 2015, Spotify launched an email campaign to urge its App Store subscribers to cancel their subscriptions and start new ones through its website, bypassing the 30% transaction fee for in-app purchases required for iOS applications by technology company Apple Inc. A later update to the Spotify app on iOS was rejected by Apple, prompting Spotify's general counsel Horacio Gutierrez to write a letter to Apple's then-general counsel Bruce Sewell, stating: "This latest episode raises serious concerns under both U.S. and EU competition law. It continues a troubling pattern of behavior by Apple to exclude and diminish the competitiveness of Spotify on iOS and as a rival to Apple Music, particularly when seen against the backdrop of Apple's previous anticompetitive conduct aimed at Spotify … we cannot stand by as Apple uses the App Store approval process as a weapon to harm competitors." In August 2020, Epic Games updated their Fortnite Battle Royale game app on both Apple's App Store and Google's Google Play to include its own storefront that offered a 20% discount on V-Bucks, the in-game currency, if players bought through there rather than through the app stores' storefront, both which take a 30% revenue cut of the sale. Both Apple and Google removed the Fortnite app within hours, as this alternate storefront violated their terms of use that required all in-app purchases to be made through their storefronts. Epic immediately filed lawsuits against both companies challenging their storefront policies on antitrust principles, arguing that their non-negotiable 30% revenue cut is too high and the restrictions against alternate storefronts anticompetitive. Apple countersued Epic over its behavior, leading to a highly publicized 2021 bench trial. Ultimately, Epic largely lost its lawsuit against Apple, though the court did order Apple to allow developers to point users to alternative payment methods. Conversely, Epic won its antitrust lawsuit against Google in late 2023. == Foundation == On 24 September 2020, Epic Games joined forces with thirteen other prominent companies—including the music streaming platform Spotify, Tinder owner Match Group, the encrypted mail service Proton Mail, and the crypto currency website Blockchain.com—to establish the Coalition for App Fairness. It also includes Basecamp. The coalition criticizes the fact that for now the app stores of both Apple and Google charge their clients a 30% fee on any purchases made over their stores. Apple and Google defended themselves by arguing that the 30% transaction fee is a standard in the industry while the Coalition for App Fairness states that there is no other transaction fee which is even close to the 30%. In October 2020, it was reported that the coalition grew from 13 to 40 members since its foundation and received more than 400 applications for membership. In October 2025, X (formerly Twitter) joined CAF. This was seen as a larger pushback in the industry against Apple and Google, and a step towards hopefully passing the Bipartisan Open App Markets Act. == Aims == The group has broadened their demands for the app stores and now also aim for a better treatment for the apps available in the App Store. They claim that Apple favors its own services before other services available on the market and unjustifiably excludes other apps from their App Store. The group has also been viewing other transaction fees like the 5% fee which is charged by credit card companies, and states that Apple charges up to 600% more and would like the 30% fee, which was only included in 2011 by Apple, adapted to a comparable percentage that charge other providers of payment solutions. Its demands are mainly directed at Apple's strict control over its App Store, but to a lesser extent are also directed towards Google. Google allows apps to be downloaded over an independent web link or also another App Store, such as the Epic Game App Store. The organization emphasizes that no app developer should come into the position in which they are discriminated and are not granted the same rights as to the developers of the owner of the app store. == Reactions == In October 2020, Microsoft presented a new framework concerning the access to its Windows 10 operating system by app stores other than the one offered by Microsoft. The new framework is based on the demands of the Coalition for App Fairness. Microsoft emphasized though, that these principles would not apply to the Xbox. In December 2020, Apple announced that they would be lowering the revenue cut Apple takes for app developers making $1M or less from 30% to 15% if app developers fill out an application for the lowered revenue cut. In March 2021, Google followed suit by also lowering the revenue cut from the Play Store from 30% to 15% for the first million in revenue earned by a developer each year. == Notable members == Members listed are notable companies listed as members the groups website: Blockchain.com Deezer Epic Games European Digital SME Alliance Fanfix Life360 Masimo Nium Proton Mail Spotify TapTap Threema Vipps

  Read more →
• 

  Bitmap index

  A bitmap index is a special kind of database index that uses bitmaps. Bitmap indexes have traditionally been considered to work well for low-cardinality columns, which have a modest number of distinct values, either absolutely, or relative to the number of records that contain the data. The extreme case of low cardinality is Boolean data (e.g., does a resident in a city have internet access?), which has two values, True and False. Bitmap indexes use bit arrays (commonly called bitmaps) and answer queries by performing bitwise logical operations on these bitmaps. Bitmap indexes have a significant space and performance advantage over other structures for query of such data. Their drawback is they are less efficient than the traditional B-tree indexes for columns whose data is frequently updated: consequently, they are more often employed in read-only systems that are specialized for fast query - e.g., data warehouses, and generally unsuitable for online transaction processing applications. Some researchers argue that bitmap indexes are also useful for moderate or even high-cardinality data (e.g., unique-valued data) which is accessed in a read-only manner, and queries access multiple bitmap-indexed columns using the AND, OR or XOR operators extensively. Bitmap indexes are also useful in data warehousing applications for joining a large fact table to smaller dimension tables such as those arranged in a star schema. == Example == Continuing the internet access example, a bitmap index may be logically viewed as follows: On the left, Identifier refers to the unique number assigned to each resident, HasInternet is the data to be indexed, the content of the bitmap index is shown as two columns under the heading bitmaps. Each column in the left illustration under the Bitmaps header is a bitmap in the bitmap index. In this case, there are two such bitmaps, one for "has internet" Yes and one for "has internet" No. It is easy to see that each bit in bitmap Y shows whether a particular row refers to a person who has internet access. This is the simplest form of bitmap index. Most columns will have more distinct values. For example, the sales amount is likely to have a much larger number of distinct values. Variations on the bitmap index can effectively index this data as well. We briefly review three such variations. Note: Many of the references cited here are reviewed at (John Wu (2007)). For those who might be interested in experimenting with some of the ideas mentioned here, many of them are implemented in open source software such as FastBit, the Lemur Bitmap Index C++ Library, the Roaring Bitmap Java library and the Apache Hive Data Warehouse system. == Compression == For historical reasons, bitmap compression and inverted list compression were developed as separate lines of research, and only later were recognized as solving essentially the same problem. Software can compress each bitmap in a bitmap index to save space. There has been considerable amount of work on this subject. Though there are exceptions such as Roaring bitmaps, Bitmap compression algorithms typically employ run-length encoding, such as the Byte-aligned Bitmap Code, the Word-Aligned Hybrid code, the Partitioned Word-Aligned Hybrid (PWAH) compression, the Position List Word Aligned Hybrid, the Compressed Adaptive Index (COMPAX), Enhanced Word-Aligned Hybrid (EWAH) and the COmpressed 'N' Composable Integer SEt (CONCISE). These compression methods require very little effort to compress and decompress. More importantly, bitmaps compressed with BBC, WAH, COMPAX, PLWAH, EWAH and CONCISE can directly participate in bitwise operations without decompression. This gives them considerable advantages over generic compression techniques such as LZ77. BBC compression and its derivatives are used in a commercial database management system. BBC is effective in both reducing index sizes and maintaining query performance. BBC encodes the bitmaps in bytes, while WAH encodes in words, better matching current CPUs. "On both synthetic data and real application data, the new word aligned schemes use only 50% more space, but perform logical operations on compressed data 12 times faster than BBC." PLWAH bitmaps were reported to take 50% of the storage space consumed by WAH bitmaps and offer up to 20% faster performance on logical operations. Similar considerations can be done for CONCISE and Enhanced Word-Aligned Hybrid. The performance of schemes such as BBC, WAH, PLWAH, EWAH, COMPAX and CONCISE is dependent on the order of the rows. A simple lexicographical sort can divide the index size by 9 and make indexes several times faster. The larger the table, the more important it is to sort the rows. Reshuffling techniques have also been proposed to achieve the same results of sorting when indexing streaming data. == Encoding == Basic bitmap indexes use one bitmap for each distinct value. It is possible to reduce the number of bitmaps used by using a different encoding method. For example, it is possible to encode C distinct values using log(C) bitmaps with binary encoding. This reduces the number of bitmaps, further saving space, but to answer any query, most of the bitmaps have to be accessed. This makes it potentially not as effective as scanning a vertical projection of the base data, also known as a materialized view or projection index. Finding the optimal encoding method that balances (arbitrary) query performance, index size and index maintenance remains a challenge. Without considering compression, Chan and Ioannidis analyzed a class of multi-component encoding methods and came to the conclusion that two-component encoding sits at the kink of the performance vs. index size curve and therefore represents the best trade-off between index size and query performance. == Binning == For high-cardinality columns, it is useful to bin the values, where each bin covers multiple values and build the bitmaps to represent the values in each bin. This approach reduces the number of bitmaps used regardless of encoding method. However, binned indexes can only answer some queries without examining the base data. For example, if a bin covers the range from 0.1 to 0.2, then when the user asks for all values less than 0.15, all rows that fall in the bin are possible hits and have to be checked to verify whether they are actually less than 0.15. The process of checking the base data is known as the candidate check. In most cases, the time used by the candidate check is significantly longer than the time needed to work with the bitmap index. Therefore, binned indexes exhibit irregular performance. They can be very fast for some queries, but much slower if the query does not exactly match a bin. == History == The concept of bitmap index was first introduced by Professor Israel Spiegler and Rafi Maayan in their research "Storage and Retrieval Considerations of Binary Data Bases", published in 1985. The first commercial database product to implement a bitmap index was Computer Corporation of America's Model 204. Patrick O'Neil published a paper about this implementation in 1987. This implementation is a hybrid between the basic bitmap index (without compression) and the list of Row Identifiers (RID-list). Overall, the index is organized as a B+tree. When the column cardinality is low, each leaf node of the B-tree would contain long list of RIDs. In this case, it requires less space to represent the RID-lists as bitmaps. Since each bitmap represents one distinct value, this is the basic bitmap index. As the column cardinality increases, each bitmap becomes sparse and it may take more disk space to store the bitmaps than to store the same content as RID-lists. In this case, it switches to use the RID-lists, which makes it a B+tree index. == In-memory bitmaps == One of the strongest reasons for using bitmap indexes is that the intermediate results produced from them are also bitmaps and can be efficiently reused in further operations to answer more complex queries. Many programming languages support this as a bit array data structure. For example, Java has the BitSet class and .NET have the BitArray class. Some database systems that do not offer persistent bitmap indexes use bitmaps internally to speed up query processing. For example, PostgreSQL versions 8.1 and later implement a "bitmap index scan" optimization to speed up arbitrarily complex logical operations between available indexes on a single table. For tables with many columns, the total number of distinct indexes to satisfy all possible queries (with equality filtering conditions on either of the fields) grows very fast, being defined by this formula: C n [ n 2 ] ≡ n ! ( n − [ n 2 ] ) ! [ n 2 ] ! {\displaystyle \mathbf {C} _{n}^{\left[{\frac {n}{2}}\right]}\equiv {\frac {n!}{\left(n-\left[{\frac {n}{2}}\right]\right)!\left[{\frac {n}{2}}\right]!}}} . A bitmap index scan combines expressions on different indexes, thus requiring only one index per column t

  Read more →
• 

  Service Assurance Agent

  IP SLA (Internet Protocol Service Level Agreement) is an active computer network measurement technology that was initially developed by Cisco Systems. IP SLA was previously known as Service Assurance Agent (SAA) or Response Time Reporter (RTR). IP SLA is used to track network performance like latency, ping response, and jitter, it also helps to provide service quality. == Functions == Routers and switches enabled with IP SLA perform periodic network tests or measurements such as Hypertext Transfer Protocol (HTTP) GET File Transfer Protocol (FTP) downloads Domain Name System (DNS) lookups User Datagram Protocol (UDP) echo, for VoIP jitter and mean opinion score (MOS) Data-Link Switching (DLSw) (Systems Network Architecture (SNA) tunneling protocol) Dynamic Host Configuration Protocol (DHCP) lease requests Transmission Control Protocol (TCP) connect Internet Control Message Protocol (ICMP) echo (remote ping) The exact number and types of available measurements depends on the IOS version. IP SLA is very widely used in service provider networks to generate time-based performance data. It is also used together with Simple Network Management Protocol (SNMP) and NetFlow, which generate volume-based data. == Usage considerations == For IP SLA tests, devices with IP SLA support are required. IP SLA is supported on Cisco routers and switches since IOS version 12.1. Other vendors like Juniper Networks or Enterasys Networks support IP SLA on some of their devices. IP SLA tests and data collection can be configured either via a console (command-line interface) or via SNMP. When using SNMP, both read and write community strings are needed. The IP SLA voice quality feature was added starting with IOS version 12.3(4)T. All versions after this, including 12.4 mainline, contain the MOS and ICPIF voice quality calculation for the UDP jitter measurement.

  Read more →