AI Face Live

AI Face Live — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  Infogram

  Infogram is a web-based data visualization and infographics platform, created in Riga, Latvia. It allows people to make and share digital charts, infographics and maps. Infogram offers an intuitive WYSIWYG editor that converts users’ data into infographics that can be published, embedded or shared. Users do not need coding skills to use this tool; users include newsrooms, marketing teams, governments, educators and students. The company that created Infogram, also called Infogram, was founded in 2012 in Riga, Latvia and has another office in San Francisco. As of October 2017, Infogram says it has 3 million users who have created charts and infographics that have been viewed more than 1.5 billion times. Infogram was bought by Prezi, a web-based presentation software company, in May 2017. == History == Infogram was founded in February 2012 in Riga, Latvia by Uldis Leiterts, Raimonds Kaže and Alise Dīrika. In January 2013, Infogram won the international Hy Berlin pitch contest. During his pitch, Infogram CEO Uldis Leiterts announced that the company had created more templates and was working with Microsoft to integrate its platform with the contemporaneous version of Microsoft Office. The company also won the 2013 Kantar Information Is Beautiful Award, which “celebrates excellence and beauty in data visualizations, infographics, interactives & information art.” In December 2014, Infogram acquired the Brazil-based data visualization blog, Visualoop. In an effort to expand sales and marketing in the U.S., Infogram secured $1.8 million in funding in February 2014. The announcement was made at TechChill, a startup conference for the Baltics in Riga, Latvia. At the time, the funding was believed to be the largest to date for the company. Infogram won the 2017 National Design Award of Latvia. == Acquisition by Prezi == Prezi, a web-based presentation software company, acquired Infogram in May 2017. Infogram is now a wholly owned subsidiary of Prezi. Infogram was rated #1 on Forbes’ list of “The Best Infographic Tools for 2017,” which was published in September 2017. In October 2017, Infogram announced a new version of its data visualization platform, including a drag-and-drop editor, over 40 new designer templates and social media support.

  Read more →
• 

  Data grid

  A data grid is an architecture or set of services that allows users to access, modify and transfer extremely large amounts of geographically distributed data for research purposes. Data grids make this possible through a host of middleware applications and services that pull together data and resources from multiple administrative domains and then present it to users upon request. The data in a data grid can be located at a single site or multiple sites where each site can be its own administrative domain governed by a set of security restrictions as to who may access the data. Likewise, multiple replicas of the data may be distributed throughout the grid outside their original administrative domain and the security restrictions placed on the original data for who may access it must be equally applied to the replicas. Specifically developed data grid middleware is what handles the integration between users and the data they request by controlling access while making it available as efficiently as possible. == Middleware == Middleware provides all the services and applications necessary for efficient management of datasets and files within the data grid while providing users quick access to the datasets and files. There is a number of concepts and tools that must be available to make a data grid operationally viable. However, at the same time not all data grids require the same capabilities and services because of differences in access requirements, security and location of resources in comparison to users. In any case, most data grids will have similar middleware services that provide for a universal name space, data transport service, data access service, data replication and resource management service. When taken together, they are key to the data grids functional capabilities. === Universal namespace === Since sources of data within the data grid will consist of data from multiple separate systems and networks using different file naming conventions, it would be difficult for a user to locate data within the data grid and know they retrieved what they needed based solely on existing physical file names (PFNs). A universal or unified name space makes it possible to create logical file names (LFNs) that can be referenced within the data grid that map to PFNs. When an LFN is requested or queried, all matching PFNs are returned to include possible replicas of the requested data. The end user can then choose from the returned results the most appropriate replica to use. This service is usually provided as part of a management system known as a Storage Resource Broker (SRB). Information about the locations of files and mappings between the LFNs and PFNs may be stored in a metadata or replica catalogue. The replica catalogue would contain information about LFNs that map to multiple replica PFNs. === Data transport service === Another middleware service is that of providing for data transport or data transfer. Data transport will encompass multiple functions that are not just limited to the transfer of bits, to include such items as fault tolerance and data access. Fault tolerance can be achieved in a data grid by providing mechanisms that ensures data transfer will resume after each interruption until all requested data is received. There are multiple possible methods that might be used to include starting the entire transmission over from the beginning of the data to resuming from where the transfer was interrupted. As an example, GridFTP provides for fault tolerance by sending data from the last acknowledged byte without starting the entire transfer from the beginning. The data transport service also provides for the low-level access and connections between hosts for file transfer. The data transport service may use any number of modes to implement the transfer to include parallel data transfer where two or more data streams are used over the same channel or striped data transfer where two or more steams access different blocks of the file for simultaneous transfer to also using the underlying built-in capabilities of the network hardware or specifically developed protocols to support faster transfer speeds. The data transport service might optionally include a network overlay function to facilitate the routing and transfer of data as well as file I/O functions that allow users to see remote files as if they were local to their system. The data transport service hides the complexity of access and transfer between the different systems to the user so it appears as one unified data source. === Data access service === Data access services work hand in hand with the data transfer service to provide security, access controls and management of any data transfers within the data grid. Security services provide mechanisms for authentication of users to ensure they are properly identified. Common forms of security for authentication can include the use of passwords or Kerberos (protocol). Authorization services are the mechanisms that control what the user is able to access after being identified through authentication. Common forms of authorization mechanisms can be as simple as file permissions. However, need for more stringent controlled access to data is done using Access Control Lists (ACLs), Role-Based Access Control (RBAC) and Tasked-Based Authorization Controls (TBAC). These types of controls can be used to provide granular access to files to include limits on access times, duration of access to granular controls that determine which files can be read or written to. The final data access service that might be present to protect the confidentiality of the data transport is encryption. The most common form of encryption for this task has been the use of SSL while in transport. While all of these access services operate within the data grid, access services within the various administrative domains that host the datasets will still stay in place to enforce access rules. The data grid access services must be in step with the administrative domains access services for this to work. === Data replication service === To meet the needs for scalability, fast access and user collaboration, most data grids support replication of datasets to points within the distributed storage architecture. The use of replicas allows multiple users faster access to datasets and the preservation of bandwidth since replicas can often be placed strategically close to or within sites where users need them. However, replication of datasets and creation of replicas is bound by the availability of storage within sites and bandwidth between sites. The replication and creation of replica datasets is controlled by a replica management system. The replica management system determines user needs for replicas based on input requests and creates them based on availability of storage and bandwidth. All replicas are then cataloged or added to a directory based on the data grid as to their location for query by users. In order to perform the tasks undertaken by the replica management system, it needs to be able to manage the underlying storage infrastructure. The data management system will also ensure the timely updates of changes to replicas are propagated to all nodes. ==== Replication update strategy ==== There are a number of ways the replication management system can handle the updates of replicas. The updates may be designed around a centralized model where a single master replica updates all others, or a decentralized model, where all peers update each other. The topology of node placement may also influence the updates of replicas. If a hierarchy topology is used then updates would flow in a tree like structure through specific paths. In a flat topology it is entirely a matter of the peer relationships between nodes as to how updates take place. In a hybrid topology consisting of both flat and hierarchy topologies updates may take place through specific paths and between peers. ==== Replication placement strategy ==== There are a number of ways the replication management system can handle the creation and placement of replicas to best serve the user community. If the storage architecture supports replica placement with sufficient site storage, then it becomes a matter of the needs of the users who access the datasets and a strategy for placement of replicas. There have been numerous strategies proposed and tested on how to best manage replica placement of datasets within the data grid to meet user requirements. There is not one universal strategy that fits every requirement the best. It is a matter of the type of data grid and user community requirements for access that will determine the best strategy to use. Replicas can even be created where the files are encrypted for confidentiality that would be useful in a research project dealing with medical files. The following section contains several strategies for replica placement. ===== Dynamic replication ===== Dynam

  Read more →
• 

  Change data capture

  In databases, change data capture (CDC) is a set of software design patterns used to determine and track the data that has changed (the "deltas") so that action can be taken using the changed data. The result is a delta-driven dataset. CDC is an approach to data integration that is based on the identification, capture and delivery of the changes made to enterprise data sources. For instance it can be used for incremental update of data loading. CDC occurs often in data warehouse environments since capturing and preserving the state of data across time is one of the core functions of a data warehouse, but CDC can be utilized in any database or data repository system. == Methodology == System developers can set up CDC mechanisms in a number of ways and in any one or a combination of system layers from application logic down to physical storage. In a simplified CDC context, one computer system has data believed to have changed from a previous point in time, and a second computer system needs to take action based on that changed data. The former is the source, the latter is the target. It is possible that the source and target are the same system physically, but that would not change the design pattern logically. Multiple CDC solutions can exist in a single system. === Timestamps on rows === Tables whose changes must be captured may have a column that represents the time of last change. Names such as LAST_UPDATE, LAST_MODIFIED, etc. are common. Any row in any table that has a timestamp in that column that is more recent than the last time data was captured is considered to have changed. Timestamps on rows are also frequently used for optimistic locking so this column is often available. === Version numbers on rows === Database designers give tables whose changes must be captured a column that contains a version number. Names such as VERSION_NUMBER, etc. are common. One technique is to mark each changed row with a version number. A current version is maintained for the table, or possibly a group of tables. This is stored in a supporting construct such as a reference table. When a change capture occurs, all data with the latest version number is considered to have changed. Once the change capture is complete, the reference table is updated with a new version number. (Do not confuse this technique with row-level versioning used for optimistic locking. For optimistic locking each row has an independent version number, typically a sequential counter. This allows a process to atomically update a row and increment its counter only if another process has not incremented the counter. But CDC cannot use row-level versions to find all changes unless it knows the original "starting" version of every row. This is impractical to maintain.) === Status indicators on rows === This technique can either supplement or complement timestamps and versioning. It can configure an alternative if, for example, a status column is set up on a table row indicating that the row has changed (e.g., a boolean column that, when set to true, indicates that the row has changed). Otherwise, it can act as a complement to the previous methods, indicating that a row, despite having a new version number or a later date, still shouldn't be updated on the target (for example, the data may require human validation). === Time/version/status on rows === This approach combines the three previously discussed methods. As noted, it is not uncommon to see multiple CDC solutions at work in a single system, however, the combination of time, version, and status provides a particularly powerful mechanism and programmers should utilize them as a trio where possible. The three elements are not redundant or superfluous. Using them together allows for such logic as, "Capture all data for version 2.1 that changed between 2005-06-01 00:00 and 2005-07-01 00:00 where the status code indicates it is ready for production." === Triggers on tables === May include a publish/subscribe pattern to communicate the changed data to multiple targets. In this approach, triggers log events that happen to the transactional table into another queue table that can later be "played back". For example, imagine an Accounts table, when transactions are taken against this table, triggers would fire that would then store a history of the event or even the deltas into a separate queue table. The queue table might have schema with the following fields: Id, TableName, RowId, Timestamp, Operation. The data inserted for our Account sample might be: 1, Accounts, 76, 2008-11-02 00:15, Update. More complicated designs might log the actual data that changed. This queue table could then be "played back" to replicate the data from the source system to a target. Data capture offers a challenge in that the structure, contents and use of a transaction log is specific to a database management system. Unlike data access, no standard exists for transaction logs. Most database management systems do not document the internal format of their transaction logs, although some provide programmatic interfaces to their transaction logs (for example: Oracle, DB2, SQL/MP, SQL/MX and SQL Server 2008). Other challenges in using transaction logs for change data capture include: Coordinating the reading of the transaction logs and the archiving of log files (database management software typically archives log files off-line on a regular basis). Translation between physical storage formats that are recorded in the transaction logs and the logical formats typically expected by database users (e.g., some transaction logs save only minimal buffer differences that are not directly useful for change consumers). Dealing with changes to the format of the transaction logs between versions of the database management system. Eliminating uncommitted changes that the database wrote to the transaction log and later rolled back. Dealing with changes to the metadata of tables in the database. CDC solutions based on transaction log files have distinct advantages that include: minimal impact on the database (even more so if one uses log shipping to process the logs on a dedicated host). no need for programmatic changes to the applications that use the database. low latency in acquiring changes. transactional integrity: log scanning can produce a change stream that replays the original transactions in the order they were committed. Such a change stream include changes made to all tables participating in the captured transaction. no need to change the database schema == Confounding factors == As often occurs in complex domains, the final solution to a CDC problem may have to balance many competing concerns. === Unsuitable source systems === Change data capture both increases in complexity and reduces in value if the source system saves metadata changes when the data itself is not modified. For example, some Data models track the user who last looked at but did not change the data in the same structure as the data. This results in noise in the Change Data Capture. === Tracking the capture === Actually tracking the changes depends on the data source. If the data is being persisted in a modern database then Change Data Capture is a simple matter of permissions. Two techniques are in common use: Tracking changes using database triggers Reading the transaction log as, or shortly after, it is written. If the data is not in a modern database, CDC becomes a programming challenge. === Push versus pull === Push: the source process creates a snapshot of changes within its own process and delivers rows downstream. The downstream process uses the snapshot, creates its own subset and delivers them to the next process. Pull: the target that is immediately downstream from the source, prepares a request for data from the source. The downstream target delivers the snapshot to the next target, as in the push model. === Alternatives === Sometimes the slowly changing dimension is used as an alternative method. CDC and SCD are similar in that both methods can detect changes in a data set. The most common forms of SCD are type 1 (overwrite), type 2 (maintain history) or 3 (only previous and current value). SCD 2 can be useful if history is needed in the target system. CDC overwrites in the target system (akin to SCD1), and is ideal when only the changed data needs to arrive at the target, i.e. a delta-driven dataset.

  Read more →
• 

  Blocknots

  Blocknots were random sequences of numbers contained in a book and organized by numbered rows and columns and were used as additives in the reciphering of Soviet Union codes, during World War II. The Blocknot consisted of a booklet of fifty sheets of 5-figure random additive, 100 additive groups to a sheet. No sheet was used more than once, thus the blocknots were in effect a form of one-time pad. The Soviet Unions highest grade ciphers that were used in the East, were the 5-figure codebook enciphered with the Blocknot book, and were generally considered unbreakable. == Technical Description == Blocknots were distributed centrally from an office in Moscow. Every Blocknot contained 5-figure groups in a number of sheets, for the enciphering of 5-figure messages. The encipherment was effected by applying additives taken from the pad, of which 50-100 5-figure groups appeared. Each pad had a 5-figure number and each sheet had a 2-figure number running consecutively. There were 5 different types of Blocknots, in two different categories The Individual in which each table of random numbers was used only once. The General in which each page of the Blocknot was valid for one day. The security of the additive sequence rested on the choice of different starting points for each message. In 5-figure messages, the blocknot was one of the first 10 Groups in the message. Its position changed at long intervals, but was always easy to re-identify. The Russians differentiated between three types of blocks: The 3-block, DRIERBLOCK. I-block for Individual Block: 50 pages, additive read off in one direction only. The messages could be used and read only between 2 wireless telegraphy stations on one net. The 6-block, SECHSERBLOCK. Z-block for Circular Block: 30 pages, additive read off in either direction. The messages could be used and read, between all W/T stations in a net. The 2-block, ZWEIERBLOCK. OS-block. Used only in traffic from lower to higher formations. Two other types were used, in lower echelons. Notblock: Used in an emergency. Blocknot used for passing on traffic. The distribution of Blocknots was carried out centrally from Moscow to Army Groups then to Armies. The Army was responsible for their distribution throughout the lower levels of the army down to company level. Independent units took their cipher material with them. Occasionally the same blocknot was distributed to two units on different parts of the front, which enabled Depth to be established. Records of all Blocknots used were kept in Berlin and when a repeat was noticed a BLOCKNOT ANGEBOT message was sent out to all German Signals units, to indicate that it may have been possible to break the code using it. There was no certainty in this. A cryptanalyst with the General der Nachrichtenaufklärung stated while being interrogated by TICOM: It seems that depths of up to 8 were established at the beginning of the Russian Campaign but that no 5-figure code was broken after May 1943 German cryptanalysts who were prisoners of war stated under interrogation, that each of the figures 0 to 9 were placed en clair usually within the first ten groups of the text or sometimes at the end. One indicator was the Blocknot number and the consisted of two random figures, the figure representing the type, and the remaining two, the page of the Blocknot being used. In long messages, 000000 was placed in the message when the end of a page had been reached. == Chi number == The Chi-number was the serial numbering of all 5-figure messages passing through the hands of the Cipher Officer, starting on the first of January and ending on thirty-first December of the current year. It always appeared as the last group in an intercepted message, e.g. 00001 on the 1st January, or when the unit was newly set up. The progression of Chi-numbers was carefully observed and recorded in the form of a graph. A Russian corps had about 10 5-figure messages per day, and Army about 20-30 and a Front about 60–100. After only a relatively short time, the individual curves separated sharply and the type of formation could be recognized by the height of the Chi-number alone. == Monitoring == Blocknots were tracked in a card index, that was maintained by the Signal Intelligence Evaluation Centre (NAAS). The NAAS functionality included evaluation and traffic analysis, cryptanalysis, collation and dissemination of intelligence. The card index, which was one amongst several Card Indexes. A careful recording and study of blocks provided the positive clues in the identification and tracking of formations using 5-figure ciphers. The index was subdivided into two files: Search card index, contained all blocknots and chi-numbers whether or not they were known. Unit card index, contained only known Block and Chi-numbers. Inspector Berger, who was the chief cryptanalyst of NAAS 1 stated that the two files formed: The most important and surest instruments for identifying Russian radio nets, known to him. The Blocknots were also used in the Stationary Intercept Company (Feste), the military unit that were designed to work at a lower level to the NAAS, at the Army level and were semi-motorized, and closer to the front. The Feste used the Blocknot value along with several other parameters to build a network diagram. The network diagram was studied extensively, as part of a 6-stage process, that involved several departments within the Feste. The outcome was a metric which determined the most interesting circuit for traffic monitoring, and least interesting, where monitoring of traffic should cease. == Analysis == Johannes Marquart was a mathematician and cryptanalyst who initially worked for Inspectorate 7/VI and later led Referat Ia of Group IV of the General der Nachrichtenaufklärung. Marquart was assigned the study of the Soviet Union Blocknot traffic. Marquart and his unit conducted extensive research in an attempt to discover the method by which they were produced. All the counts which they made, however, failed to reveal any non-random characteristics in the design of the tables, and while they thought the Blocknots must have been generated by machine, they were never able to draw any concrete deductions as a result of their research. == Example == The Soviet 3rd Guard Tank Army transmits a 5-figure message with the Blocknot of 37581 (one of the first 10 groups in the message). On the same day the Block 37582 was used by the same formation. The next day 37583 appeared. Thereafter, for a period, the Army was not heard by German Wireless telegraphy intercept operators, as it was maintaining wireless silence. After a few days, an unidentified net with the Blocknot 37588 is picked up. This message net is claimed, because of the proximity of the blocks (88/83) to be the 3rd Guard Tank Army. The missing Blocknots 84-87 were presumably used in telegraphic, telephonic or courier communications. The Chi number provides confirmation of the first assumption, based on proximity of blocknots in most cases.

  Read more →
• 

  Manifold hypothesis

  The manifold hypothesis posits that many high-dimensional data sets that occur in the real world actually lie along low-dimensional latent manifolds inside that high-dimensional space. As a consequence of the manifold hypothesis, many data sets that appear to initially require many variables to describe, can actually be described by a comparatively small number of variables, linked to the local coordinate system of the underlying manifold. It is suggested that this principle underpins the effectiveness of machine learning algorithms in describing high-dimensional data sets by considering a few common features. The manifold hypothesis is related to the effectiveness of nonlinear dimensionality reduction techniques in machine learning. Many techniques of dimensional reduction make the assumption that data lies along a low-dimensional submanifold, such as manifold sculpting, manifold alignment, and manifold regularization. The major implications of this hypothesis is that Machine learning models only have to fit relatively simple, low-dimensional, highly structured subspaces within their potential input space (latent manifolds). Within one of these manifolds, it's always possible to interpolate between two inputs, that is to say, morph one into another via a continuous path along which all points fall on the manifold. The ability to interpolate between samples is the key to generalization in deep learning. == The information geometry of statistical manifolds == An empirically-motivated approach to the manifold hypothesis focuses on its correspondence with an effective theory for manifold learning under the assumption that robust machine learning requires encoding the dataset of interest using methods for data compression. This perspective gradually emerged using the tools of information geometry thanks to the coordinated effort of scientists working on the efficient coding hypothesis, predictive coding and variational Bayesian methods. The argument for reasoning about the information geometry on the latent space of distributions rests upon the existence and uniqueness of the Fisher information metric. In this general setting, we are trying to find a stochastic embedding of a statistical manifold. From the perspective of dynamical systems, in the big data regime this manifold generally exhibits certain properties such as homeostasis: We can sample large amounts of data from the underlying generative process. Machine Learning experiments are reproducible, so the statistics of the generating process exhibit stationarity. In a sense made precise by theoretical neuroscientists working on the free energy principle, the statistical manifold in question possesses a Markov blanket.

  Read more →
• 

  Social film

  A social film is a type of interactive film that is presented through the lens of social media. A social film is distributed digitally and integrates with a social networking service, such as Facebook or YouTube. It combines features of web video, social network games and social media. == Key elements == Social films are a more recent phenomenon, and, in turn, there are few precedents for their format. Although there are not many examples of this genre of film, the medium has certain identifiable elements: Casual entertainment Social media User-generated content Game mechanics Using just one of these factors or a combination of them, a social film engages viewers to interact directly with the work. This can be done through usual social media functionality like comments and ranking or adding directly to the narrative itself. Just as with memes, social film distribution relies on the viral spread enabled by social media. This is based on the viral expansion loops model, in which a viewer benefits from sharing the application with friends, exponentially creating new viewers compelled to share the application. == History == One of the first social films to be created was from the YouTube channel lonelygirl15. This social film started in 2006 and was created by Miles Beckett , Mesh Flinders, and Greg Goodfried. They used YouTube posts to create an interactive video series about a fictional character who showcased her life in a vlog format. As the videos went on, more bizarre things would keep happening to the main character, Bree, before she just stopped uploading. This channel was not only the first viral social film, but went on to be one of the first viral YouTube channels to be created. It did take a few years to see any more films in this genre, but 2011 saw many people start to try their hand at making these films. The first social film in this year was a film called Him, Her and Them which was produced and released by Murmur in April 2011. It was distributed exclusively through Facebook and promoted as the first “Facebook film.” The film is composed of short video clips and interactive slideshows, integrating Facebook's Social Graph API. Users participate via text-based additions to the story, which are viewable only by friends within their social network. In May 2011, Canon and Ron Howard teamed up to create Project Imagin8ion, which was a photo contest where photographers submitted photos and the top 8 photos would be the inspiration for a short film. This short film was called "When You Find Me" and could be found exclusively on YouTube. In July 2011, Intel and Toshiba partnered together to create Hollywood's first Social Film experience, a thriller called Inside, directed by D.J. Caruso and starring Emmy Rossum. The project is broken up into several segments across multiple social media platforms including Facebook, YouTube, and Twitter. In this instance, the audience is challenged to help Emmy Rossum's character, Christina, safely make it out of the room she's been trapped in. This particular form of social film is a major undertaking in that it combines social media activity with A-list acting talent to create a user experience that all happens in real time. Although not quite the same idea, Hollywood also started experimenting with the idea of interactive and crowd-sourced films. One of the first examples of this was a short film called "Life In A Day" directed by Kevin Macdonald and produced by Ridley Scott. Kevin asked people from all over the world to submit videos onto YouTube of what they were doing on July 24th, 2010. They combined all of the best videos that were submitted together to create one film of people doing different things all around the world, no matter how boring or simple those things seemed. They took this short to film festivals before releasing it to the public on YouTube in 2011. In August 2012, Intel and Toshiba partnered again to create The Beauty Inside, directed by Drake Doremus, starring Mary Elizabeth Winstead and Topher Grace. It's Hollywood's first social film that gives everyone in the audience a chance to play Alex, the lead role. The experience will be broken up into six filmed episodes interspersed with real-time interactive storytelling that all takes place on Alex's Facebook timeline. In August 2013, Intel and Toshiba released their third entry into the category, The Power Inside, directed by Will Speck and Josh Gordon and starring Harvey Keitel, Analeigh Tipton, and Craig Roberts. It's Hollywood's first social film that asks the audience to audition to help save or destroy the world. The experience is broken up into six filmed episodes interspersed with user-generated content and interactive storytelling on the main character's Facebook timeline. In 2015, Intel partnered with Dell for their fourth entry, What Lives Inside directed by Robert Stromberg and starring Colin Hanks, Catherine O'Hara, and J. K. Simmons. The first of four episodes was released on Hulu on March 25, 2015.

  Read more →
• 

  Short Weather Cipher

  The Short Weather Cipher (German: Wetterkurzschlüssel, abbreviated WKS), also known as the weather short signal book, was a cipher, presented as a codebook, that was used by the radio telegraphists aboard U-boats of the German Navy (Kriegsmarine) during World War II. It was used to condense weather reports into a short 7-letter message, which was enciphered by using the naval Enigma and transmitted by radiomen to intercept stations on shore, where it was deciphered by Enigma and the 7-letter weather report was reconstructed. == History == During World War II, during various times, different versions of the cipher were in operation. The first issue carried the codename Weimar. It was replaced by the edition Eisenach on 20 January 1942. On 10 March 1943, the third edition of the weather key, bearing the codename Naumburg, entered into force. On May 9, 1941, during Operation Primrose, the operation to occupy Åndalsnes and create a diversion south of Trondheim in Norway as part of the Norwegian Campaign, an intact Naval Enigma (M3) cipher machine, a copy of the "Weimar" version of the short weather cipher and a copy of the short signal book (German: Kurzsignalbuch or Kurzsignale for short) was recovered from the submarine U-110, that was captured in the North Atlantic east of Cape Farewell, Greenland. This enabled the cryptanalysts in Bletchley Park to break the encryption of the M3 and to decipher the German submarine radio messages. The Short Weather Cipher was critical in the cryptanalysis of the Naval Enigma M4 and yielded excellent cribs. On 30 October 1942, a copy of the Wetterkurzschlüssel, the short weather cipher, and of the short signal book, the Kurzsignale, were recovered as part of a daring raid on the U-boat U-559, when three Royal Navy sailors, Lieutenant Anthony Fasson, Able Seaman Colin Grazier and NAAFI canteen assistant Tommy Brown, then boarded the abandoned submarine, and recovered the documents after a 90-minute search. They reached the Government Code and Cypher at Bletchley Park after a three-week delay, on 24 November 1942. The documents which cost the lives of Fasson and Grazier proved to be particularly important in breaking the Naval Enigma M4. The version of the short weather cipher recovered was the Eisenach version. Unlike the first version Weimar, the Eisenach did not list the 26 rotor positions that were indicated by a letter, to be used in enciphering weather reports. Thus, Hut 8 cryptanalysts thought that all four rotors were used to encipher weather reports. Testing on the Bombes began to surface weather kisses (identical messages in two cryptosystems). On 13 December 1942, a crib obtained using the Short Weather Cipher gave a key with the Naval Enigma M4 rotatable Umkehrwalze (reversing roller or reflector) in the neutral position, making it equivalent to a standard Enigma and thus making B-Dienst messages potentially breakable on existing bombes. Hut 8 learned that the 4-letter indicators for regular U-boat messages were the same as 3-letter indicators for weather messages the same day, except for one extra letter. This meant that once the key was found for a weather message on any day, the fourth rotor had to be only tested in 26 positions to find the full 4-letter key. By the end of the day on Sunday 13 December, Rodger Winn of the Submarine Tracking Room at Bletchley Park knew that Shark Enigma Cipher was broken. When the third edition of the short signal book was introduced on 10 March 1943, Hut 8 was immediately deprived of cribs. However, by the 19 March, cribs were again being used by Hut 8 personnel, using the method of employing short signal sighting reports. These were reports made by U-boats when contact was made with Kurzsignalheft code book. Hut 8 managed to solve Shark for 90 out of 112 days before the end of June. Kurzsignalheft short sighting reports also used M4 in M3 mode. By the end of June, four-rotor bombes had entered service at Bletchley Park, and by August had been introduced by the US Navy. From September onwards, Shark was generally solved within 24 hours. == Operation == The U-boat encoded weather reports using the Short Weather Cipher, before being enciphered on the Naval Enigma. The shore patrol of the Kriegsmarine, deciphered the message and decoded it, then forwarding it to a central meteorological station, which rebroadcast the data as ship synoptics, after enciphering it with additive tables using a cipher, which was called Germet 3 by Hut 8 personnel. The short weather cipher coded weather reports using a polyphonic single-letter code with X missing. A = +28° ◦ B = +27° ◦ C = +26° ◦ D = +25° ◦ . . . ◦ W = +6° ◦ Y= +5° ◦ Z = +4° ◦ A = +3° ◦ B = +2° ◦ C = +1° ◦ D = 0° ◦ E =−1° ◦ F =−2° ◦ . . . ◦ Z = −21° ◦ In a similar way, water temperature, atmospheric pressure, humidity, wind direction, wind velocity, visibility, degree of cloudiness, geographic latitude, and geographic longitude had to be coded in a prescribed order with the weather report consisted of a single short word. Based on the approximate knowledge of the position of the submarine, the Kriegsmarine telegraphist who received the message could translate the letter "S", according to the above table, which could mean 10 °C or −15 °C, back to the correct temperature. Similarly, the direction and the type of swell was also coded with only a single letter: ----------------------------------------------------- Direction from which | Type of swell the swell comes | low | middle high | high | ----------------------------------------------------- N | a | i | q | NE | b | j | r | E | c | k | s | SE | d | l | t | S | e | m | u | SW | f | n | v | W | g | o | w | NW | h | p | x | No swelling | | | | y Intermittent | | | | z As an example of the cipher, a weather report for 68° North latitude, 20° West longitude (north of Iceland) with atmospheric pressure 972 millibars, temperature minus 5 °C, wind northwest Force 6 (on the Beaufort scale), 3/10 cirrus cloud cover, visibility 5 nautical miles, would be coded as MZNFPED. == Publications == Bauer, Arthur O. (1997), Funkpeilung als alliierte Waffe gegen deutsche U-Boote 1939–1945 [Direction finding as Allied weapon against German submarines from 1939 to 1945] (in German), Diemen, NL: Selbstverlag, ISBN 978-3-00-002142-8 Bauer, Friedrich L. (2007), Decrypted Secrets. Methods and Maxims of Cryptology (4., rev. and extended ed.), Berlin Heidelberg New York: Springer, ISBN 978-3-540-24502-5 Pfeiffer, Paul N. (October 1998), "Breaking the German Weather Ciphers in the Mediterranean Detachment, 849th Signal Intelligence Service", Cryptologia, 22 (4): 354–369, doi:10.1080/0161-119891886975, ISSN 0161-1194 Ulbricht, Heinz (2005), Die Chiffriermaschine Enigma – Trügerische Sicherheit. Ein Beitrag zur Geschichte der Nachrichtendienste [The Enigma cipher machine – Deceptive security. A contribution to the history of the intelligence services], Dissertation, Fachbereich Mathematik und Informatik, Technische Universität Braunschweig (in German)

  Read more →
• 

  Cambridge Analytica

  Cambridge Analytica Ltd. (CA), previously known as SCL USA, was a British political consulting firm that came to prominence through the Facebook–Cambridge Analytica data scandal. It was founded in 2013, as a subsidiary of the private intelligence company and self-described "global election management agency" SCL Group by long-time SCL executives Nigel Oakes, Alexander Nix and Alexander Oakes, with Nix as CEO. Cambridge Analytica was hired by a variety of political actors, including the Trinidadian government in 2010 and the 2016 presidential campaigns of Ted Cruz and Donald Trump. The firm maintained offices in London, New York City, and Washington, D.C. The company closed operations in 2018 due to backlash from the scandal, although firms related to both Cambridge Analytica and its parent firm SCL still exist. == History == Cambridge Analytica was founded in 2013 as a subsidiary of the private intelligence company SCL Group, which describes itself as providing "data, analytics and strategy to governments and military organisations worldwide". The company was part of "an international web of companies" headed by the London-based SCL Group. Cambridge Analytica (SCL USA) was incorporated in January 2013 with its registered office being in Westferry Circus, London and consisting of just one staff member, director and CEO Alexander Nix (also appointed in January 2015). Nix was also the director of nine similar companies sharing the same registered offices in London, including Firecrest technologies, Emerdata and six SCL Group companies including "SCL elections limited". Nigel Oakes, known as the former boyfriend of Lady Helen Windsor, had founded the predecessor SCL Group in the 1990s, and in 2005 Oakes established SCL Group together with his brother Alexander Oakes and Alexander Nix; SCL Group was the parent company of Cambridge Analytica. Former Conservative minister and MP Sir Geoffrey Pattie was the founding chairman of SCL; Lord Ivar Mountbatten also joined Oakes as a director of the company. As a result of the Facebook–Cambridge Analytica data scandal, Nix was removed as CEO and replaced by Julian Wheatland before the company closed. Several of the company's executives were Old Etonians. The company's owners included several of the Conservative Party's largest donors such as billionaire Vincent Tchenguiz, former British Conservative minister Jonathan Marland, Baron Marland and the family of American hedge fund manager Robert Mercer. The company combined misappropriation of digital assets, data mining, data brokerage, and data analysis with strategic communication during electoral processes. While its parent SCL had focused on influencing elections in developing countries since the 1990s, Cambridge Analytica focused more on the western world, including the United Kingdom and the United States; CEO Alexander Nix has said CA was involved in 44 U.S. political races in 2014. In 2015, CA performed data analysis services for Ted Cruz's presidential campaign. In 2016, CA worked for Donald Trump's presidential campaign as well as for Leave.EU (one of the organisations campaigning in the United Kingdom's referendum on European Union membership). CA's role in those campaigns has been controversial and is the subject of ongoing inquiries in both countries. Political scientists question CA's claims about the effectiveness of its methods of targeting voters. == Data scandal == In March 2018, media outlets broke news of Cambridge Analytica's business practices. The New York Times and The Observer reported that the company had acquired and used personal data about Facebook users from an external researcher who had told Facebook he was collecting it for academic purposes. Shortly afterwards, Channel 4 News aired undercover investigative videos showing Nix boasting about using prostitutes, bribery sting operations, and honey traps to discredit politicians on whom it had conducted opposition research, and saying that the company "ran all of (Donald Trump's) digital campaign". In response to the media reports, the Information Commissioner's Office (ICO) of the UK pursued a warrant to search the company's servers. Facebook banned Cambridge Analytica from advertising on its platform, saying that it had been deceived. On 23 March 2018, the British High Court granted the ICO a warrant to search Cambridge Analytica's London offices. As a result, Nix was suspended as CEO, and replaced by Julian Wheatland. The personal data of up to 87 million Facebook users were acquired via the 270,000 Facebook users who used a Facebook app created by Aleksandr Kogan called "This Is Your Digital Life". This was a personality profiling app and asked simple personality questions similar to other Facebook quizzes. Kogan was a scientist and psychologist, also being an employed lecturer for the University of Cambridge from 2012 to 2018. Alexander Nix claimed they had close to five thousand data points on each person who participated. They also gathered information through other data brokers ending with them acquiring millions of data points from American citizens. Kogan's app exploited a feature of Facebook's Graph API (version 1.0), which permitted any third-party app to access not only the app user's data, but also the full profile data of all of that user's Facebook friends, without those friends' knowledge or consent. This platform-wide design was available to all developers and was used by tens of thousands of apps; Facebook CEO Mark Zuckerberg later told the House Energy and Commerce Committee that the company was auditing "tens of thousands" of apps that had had access to large amounts of user data. Because the average Facebook user at the time had approximately 300 friends, the 270,000 users who installed Kogan's app yielded data on up to 87 million people. Facebook deprecated the friends-data API in April 2014 and shut it down entirely in April 2015, but data already collected by apps remained in developers' possession. Kogan passed this data to Cambridge Analytica, breaching Facebook's terms of service. On 1 May 2018, Cambridge Analytica and its parent company SCL filed for insolvency proceedings and closed operations. Alexander Tayler, a former director for Cambridge Analytica, was appointed director of Emerdata on 28 March 2018. Rebekah Mercer, Jennifer Mercer, Alexander Nix and Johnson Chun Shun Ko, who has links to American businessman Erik Prince, are in leadership positions at Emerdata. The Russo brothers are producing an upcoming film on Cambridge Analytica. In 2019 the Federal Trade Commission filed an administrative complaint against Cambridge Analytica for misuse of data. In 2020, the British Information Commissioner's Office closed a three-year inquiry into the company, concluded that Cambridge Analytica was "not involved" in the 2016 Brexit referendum and found no additional evidence for Russia's alleged interference during the campaign. US sensitive polling and election data, however, were passed to Russian Intelligence via a Cambridge Analytica contractor Sam Patten, Trump campaign manager Paul Manafort, and Russian agent Konstantin Kilimnik, who was indicted during the affair. Publicly, parent company SCL Group called itself a "global election management agency", Politico reported it was known for involvement "in military disinformation campaigns to social media branding and voter targeting". SCL gained work on a large number of campaigns for the US and UK governments' war on terror advancing their model of behavioral conflict during the 2000s. SCL's involvement in the political world has been primarily in the developing world where it has been used by the military and politicians to study and manipulate public opinion and political will. Slate writer Sharon Weinberger compared one of SCL's hypothetical test scenarios to fomenting a coup. Among the investors in Cambridge Analytica were some of the Conservative Party's largest donors such as billionaire Vincent Tchenguiz, former Conservative minister Jonathan Marland, Baron Marland, Roger Gabb, the family of American hedge fund manager Robert Mercer, and Steve Bannon. A minimum of 15 million dollars has been invested into the company by Mercer, according to The New York Times. Bannon's stake in the company was estimated at 1 to 5 million dollars, but he divested his holdings in April 2017 as required by his role as White House Chief Strategist. In March 2018, Jennifer Mercer and Rebekah Mercer became directors of Emerdata limited. In March 2018 it became public by Christopher Wylie, that Cambridge Analytica's first activities were founded on a data set, which its parent company SCL bought 2014 from a company named Global Science Research founded by Aleksandr Kogan and his team present across the world who worked as a psychologist at Cambridge. During Boris Johnson's tenure as foreign secretary, the Foreign Office sought advice from Cambridge Analytica and Boris Johnson had a meeting with Alexander N

  Read more →
• 

  Apertus (LLM)

  Apertus is a public large language model, developed by the Swiss AI Initiative (a collaboration between EPFL, ETH Zurich, and the Swiss National Supercomputing Centre). It was released on September 2, 2025, under the free and open-source Apache 2.0 license. Designed initially for business and research use cases around the world, Apertus was trained on over 1800 languages, and comes in 8 billion or 70 billion parameter versions and is available on Hugging Face for download. The model was developed aiming to adhere to European copyright law, and is one of the first examples of AI as a public good in the vein of AI Sovereignty. It is also the first large model to comply with the European Union's Artificial Intelligence Act. At its launch, the model creators emphasized multilinguality, transparency, and auditability as priorities in contrast to commercial frontier model. While international reception was largely positive, the first iteration was significantly behind the capabilities of frontier models and needs adaptation for many use cases with chatbots being a secondary but not a primary use case. As of late 2025, it was considered the largest and most capable fully open model. The capability of future models will depend in part on how much more funding can be secured.

  Read more →
• 

  Cryptosystem

  In cryptography, a cryptosystem is a suite of cryptographic algorithms needed to implement a particular security service, such as confidentiality (encryption). Typically, a cryptosystem consists of three algorithms: one for key generation, one for encryption, and one for decryption. The term cipher (sometimes cypher) is often used to refer to a pair of algorithms, one for encryption and one for decryption. Therefore, the term cryptosystem is most often used when the key generation algorithm is important. For this reason, the term cryptosystem is commonly used to refer to public key techniques; however both "cipher" and "cryptosystem" are used for symmetric key techniques. == Formal definition == Mathematically, a cryptosystem or encryption scheme can be defined as a tuple ( P , C , K , E , D ) {\displaystyle ({\mathcal {P}},{\mathcal {C}},{\mathcal {K}},{\mathcal {E}},{\mathcal {D}})} with the following properties. P {\displaystyle {\mathcal {P}}} is a set called the "plaintext space". Its elements are called plaintexts. C {\displaystyle {\mathcal {C}}} is a set called the "ciphertext space". Its elements are called ciphertexts. K {\displaystyle {\mathcal {K}}} is a set called the "key space". Its elements are called keys. E = { E k : k ∈ K } {\displaystyle {\mathcal {E}}=\{E_{k}:k\in {\mathcal {K}}\}} is a set of functions E k : P → C {\displaystyle E_{k}:{\mathcal {P}}\rightarrow {\mathcal {C}}} . Its elements are called "encryption functions". D = { D k : k ∈ K } {\displaystyle {\mathcal {D}}=\{D_{k}:k\in {\mathcal {K}}\}} is a set of functions D k : C → P {\displaystyle D_{k}:{\mathcal {C}}\rightarrow {\mathcal {P}}} . Its elements are called "decryption functions". For each e ∈ K {\displaystyle e\in {\mathcal {K}}} , there is d ∈ K {\displaystyle d\in {\mathcal {K}}} such that D d ( E e ( p ) ) = p {\displaystyle D_{d}(E_{e}(p))=p} for all p ∈ P {\displaystyle p\in {\mathcal {P}}} . Note; typically this definition is modified in order to distinguish an encryption scheme as being either a symmetric-key or public-key type of cryptosystem. == Examples == A classical example of a cryptosystem is the Caesar cipher. A more contemporary example is the RSA cryptosystem. Another example of a cryptosystem is the Advanced Encryption Standard (AES). AES is a widely used symmetric encryption algorithm that has become the standard for securing data in various applications. Paillier cryptosystem is another example used to preserve and maintain privacy and sensitive information. It is featured in electronic voting, electronic lotteries and electronic auctions.

  Read more →
• 

  Completeness (cryptography)

  In cryptography, a boolean function is said to be complete if the value of each output bit depends on all input bits. This is a desirable property to have in an encryption cipher, so that if one bit of the input (plaintext) is changed, every bit of the output (ciphertext) has an average of 50% probability of changing. The easiest way to show why this is good is the following: consider that if we changed our 8-byte plaintext's last byte, it would only have any effect on the 8th byte of the ciphertext. This would mean that if the attacker guessed 256 different plaintext-ciphertext pairs, he would always know the last byte of every 8byte sequence we send (effectively 12.5% of all our data). Finding out 256 plaintext-ciphertext pairs is not hard at all in the internet world, given that standard protocols are used, and standard protocols have standard headers and commands (e.g. "get", "put", "mail from:", etc.) which the attacker can safely guess. On the other hand, if our cipher has this property (and is generally secure in other ways, too), the attacker would need to collect 264 (~1020) plaintext-ciphertext pairs to crack the cipher in this way.

  Read more →
• 

  Clustered file system

  A clustered file system (CFS) is a file system which is shared by being simultaneously mounted on multiple servers. There are several approaches to clustering, most of which do not employ a clustered file system (only direct attached storage for each node). Clustered file systems can provide features like location-independent addressing and redundancy which improve reliability or reduce the complexity of the other parts of the cluster. Parallel file systems are a type of clustered file system that spread data across multiple storage nodes, usually for redundancy or performance. == Shared-disk file system == A shared-disk file system uses a storage area network (SAN) to allow multiple computers to gain direct disk access at the block level. Access control and translation from file-level operations that applications use to block-level operations used by the SAN must take place on the client node. The most common type of clustered file system, the shared-disk file system – by adding mechanisms for concurrency control – provides a consistent and serializable view of the file system, avoiding corruption and unintended data loss even when multiple clients try to access the same files at the same time. Shared-disk file-systems commonly employ some sort of fencing mechanism to prevent data corruption in case of node failures, because an unfenced device can cause data corruption if it loses communication with its sister nodes and tries to access the same information other nodes are accessing. The underlying storage area network may use any of a number of block-level protocols, including SCSI, iSCSI, HyperSCSI, ATA over Ethernet (AoE), Fibre Channel, network block device, and InfiniBand. There are different architectural approaches to a shared-disk filesystem. Some distribute file information across all the servers in a cluster (fully distributed). === Examples === == Distributed file systems == Distributed file systems do not share block level access to the same storage but use a network protocol. These are commonly known as network file systems, even though they are not the only file systems that use the network to send data. Distributed file systems can restrict access to the file system depending on access lists or capabilities on both the servers and the clients, depending on how the protocol is designed. The difference between a distributed file system and a distributed data store is that a distributed file system allows files to be accessed using the same interfaces and semantics as local files – for example, mounting/unmounting, listing directories, read/write at byte boundaries, system's native permission model. Distributed data stores, by contrast, require using a different API or library and have different semantics (most often those of a database). === Design goals === Distributed file systems may aim for "transparency" in a number of aspects. That is, they aim to be "invisible" to client programs, which "see" a system which is similar to a local file system. Behind the scenes, the distributed file system handles locating files, transporting data, and potentially providing other features listed below. Access transparency: clients are unaware that files are distributed and can access them in the same way as local files are accessed. Location transparency: a consistent namespace exists encompassing local as well as remote files. The name of a file does not give its location. Concurrency transparency: all clients have the same view of the state of the file system. This means that if one process is modifying a file, any other processes on the same system or remote systems that are accessing the files will see the modifications in a coherent manner. Failure transparency: the client and client programs should operate correctly after a server failure. Heterogeneity: file service should be provided across different hardware and operating system platforms. Scalability: the file system should work well in small environments (1 machine, a dozen machines) and also scale gracefully to bigger ones (hundreds through tens of thousands of systems). Replication transparency: Clients should not have to be aware of the file replication performed across multiple servers to support scalability. Migration transparency: files should be able to move between different servers without the client's knowledge. === History === The Incompatible Timesharing System used virtual devices for transparent inter-machine file system access in the 1960s. More file servers were developed in the 1970s. In 1976, Digital Equipment Corporation created the File Access Listener (FAL), an implementation of the Data Access Protocol as part of DECnet Phase II which became the first widely used network file system. In 1984, Sun Microsystems created the file system called "Network File System" (NFS) which became the first widely used Internet Protocol based network file system. Other notable network file systems are Andrew File System (AFS), Apple Filing Protocol (AFP), NetWare Core Protocol (NCP), and Server Message Block (SMB) which is also known as Common Internet File System (CIFS). In 1986, IBM announced client and server support for Distributed Data Management Architecture (DDM) for the System/36, System/38, and IBM mainframe computers running CICS. This was followed by the support for IBM Personal Computer, AS/400, IBM mainframe computers under the MVS and VSE operating systems, and FlexOS. DDM also became the foundation for Distributed Relational Database Architecture, also known as DRDA. There are many peer-to-peer network protocols for open-source distributed file systems for cloud or closed-source clustered file systems, e. g.: 9P, AFS, Coda, CIFS/SMB, DCE/DFS, WekaFS, Lustre, PanFS, Google File System, Mnet, Chord Project. === Examples === == Network-attached storage == Network-attached storage (NAS) provides both storage and a file system, like a shared disk file system on top of a storage area network (SAN). NAS typically uses file-based protocols (as opposed to block-based protocols a SAN would use) such as NFS (popular on UNIX systems), SMB/CIFS (Server Message Block/Common Internet File System) (used with MS Windows systems), AFP (used with Apple Macintosh computers), or NCP (used with OES and Novell NetWare). == Design considerations == === Avoiding single point of failure === The failure of disk hardware or a given storage node in a cluster can create a single point of failure that can result in data loss or unavailability. Fault tolerance and high availability can be provided through data replication of one sort or another, so that data remains intact and available despite the failure of any single piece of equipment. For examples, see the lists of distributed fault-tolerant file systems and distributed parallel fault-tolerant file systems. === Performance === A common performance measurement of a clustered file system is the amount of time needed to satisfy service requests. In conventional systems, this time consists of a disk-access time and a small amount of CPU-processing time. But in a clustered file system, a remote access has additional overhead due to the distributed structure. This includes the time to deliver the request to a server, the time to deliver the response to the client, and for each direction, a CPU overhead of running the communication protocol software. === Concurrency === Concurrency control becomes an issue when more than one person or client is accessing the same file or block and want to update it. Hence updates to the file from one client should not interfere with access and updates from other clients. This problem is more complex with file systems due to concurrent overlapping writes, where different writers write to overlapping regions of the file concurrently. This problem is usually handled by concurrency control or locking which may either be built into the file system or provided by an add-on protocol. == History == IBM mainframes in the 1970s could share physical disks and file systems if each machine had its own channel connection to the drives' control units. In the 1980s, Digital Equipment Corporation's TOPS-20 and OpenVMS clusters (VAX/ALPHA/IA64) included shared disk file systems.

  Read more →
• 

  Big data

  Big data primarily refers to data sets that are too large or complex to be dealt with by traditional data-processing software. Data with many entries (rows) offers greater statistical power, while data with higher complexity (more attributes or columns) may lead to a higher false discovery rate. Big data analysis challenges include capturing data, data storage, data analysis, search, sharing, transfer, visualization, querying, updating, information privacy, and data sources. Big data was originally associated with three key concepts: volume, variety, and velocity. The analysis of big data that have only volume, velocity, and variety can pose challenges in sampling. A fourth concept, veracity, which refers to the level of reliability of data, was thus added. Without sufficient investment in expertise to ensure big data veracity, the volume and variety of data can produce costs and risks that exceed an organization's capacity to create and capture value from big data. Current usage of the term big data tends to refer to the use of predictive analytics, user behavior analytics, or certain other advanced data analytics methods that extract value from big data, and seldom to a particular size of data set. "There is little doubt that the quantities of data now available are indeed large, but that's not the most relevant characteristic of this new data ecosystem." Analysis of data sets can find new correlations to "spot business trends, prevent diseases, combat crime and so on". Scientists, business executives, medical practitioners, advertising and governments alike regularly meet difficulties with large datasets in areas including Internet searches, fintech, healthcare analytics, geographic information systems, urban informatics, and business informatics. Scientists encounter limitations in e-Science work, including meteorology, genomics, connectomics, complex physics simulations, biology, and environmental research. The size and number of available data sets have grown rapidly as data is collected by devices such as mobile devices, cheap and numerous information-sensing Internet of things devices, aerial (remote sensing) equipment, software logs, cameras, microphones, radio-frequency identification (RFID) readers and wireless sensor networks. The world's technological per-capita capacity to store information has roughly doubled every 40 months since the 1980s; as of 2012, every day 2.5 exabytes (2.17×260 bytes) of data are generated. Based on an IDC report prediction, the global data volume was predicted to grow exponentially from 4.4 zettabytes to 44 zettabytes between 2013 and 2020. By 2025, IDC predicts there will be 163 zettabytes of data. According to IDC, global spending on big data and business analytics (BDA) solutions is estimated to reach $215.7 billion in 2021. Statista reported that the global big data market is forecasted to grow to $103 billion by 2027. In 2011 McKinsey & Company reported, if US healthcare were to use big data creatively and effectively to drive efficiency and quality, the sector could create more than $300 billion in value every year. In the developed economies of Europe, government administrators could save more than €100 billion ($149 billion) in operational efficiency improvements alone by using big data. And users of services enabled by personal-location data could capture $600 billion in consumer surplus. One question for large enterprises is determining who should own big-data initiatives that affect the entire organization. Relational database management systems and desktop statistical software packages used to visualize data often have difficulty processing and analyzing big data. The processing and analysis of big data may require "massively parallel software running on tens, hundreds, or even thousands of servers". What qualifies as "big data" varies depending on the capabilities of those analyzing it and their tools. Furthermore, expanding capabilities make big data a moving target. "For some organizations, facing hundreds of gigabytes of data for the first time may trigger a need to reconsider data management options. For others, it may take tens or hundreds of terabytes before data size becomes a significant consideration." == Definition == The term big data has been in use since the 1990s, with some giving credit to John Mashey for popularizing the term. Big data usually includes data sets with sizes beyond the ability of commonly used software tools to capture, curate, manage, and process data within a tolerable elapsed time. Big data philosophy encompasses unstructured, semi-structured and structured data; however, the main focus is on unstructured data. Big data "size" is a constantly moving target; as of 2012 ranging from a few dozen terabytes to many zettabytes of data. Big data requires a set of techniques and technologies with new forms of integration to reveal insights from datasets that are diverse, complex, and of a massive scale. Variability is often included as an additional quality of big data. A 2018 definition states "Big data is where parallel computing tools are needed to handle data", and notes, "This represents a distinct and clearly defined change in the computer science used, via parallel programming theories, and losses of some of the guarantees and capabilities made by Codd's relational model." In a comparative study of big datasets, Kitchin and McArdle found that none of the commonly considered characteristics of big data appear consistently across all of the analyzed cases. For this reason, other studies identified the redefinition of power dynamics in knowledge discovery as the defining trait. Instead of focusing on the intrinsic characteristics of big data, this alternative perspective pushes forward a relational understanding of the object claiming that what matters is the way in which data is collected, stored, made available and analyzed. === Big data vs. business intelligence === The growing maturity of the concept more starkly delineates the difference between "big data" and "business intelligence": Business intelligence uses applied mathematics tools and descriptive statistics with data with high information density to measure things, detect trends, etc. Big data uses mathematical analysis, optimization, inductive statistics, and concepts from nonlinear system identification to infer laws (regressions, nonlinear relationships, and causal effects) from large sets of data with low information density to reveal relationships and dependencies, or to perform predictions of outcomes and behaviors. == Characteristics == Big data can be described by the following characteristics: Volume The quantity of generated and stored data. The size of the data determines the value and potential insight, and whether it can be considered big data or not. The size of big data is usually larger than terabytes and petabytes. Variety The type and nature of the data. Earlier technologies like RDBMSs were capable to handle structured data efficiently and effectively. However, the change in type and nature from structured to semi-structured or unstructured challenged the existing tools and technologies. Big data technologies evolved with the prime intention to capture, store, and process the semi-structured and unstructured (variety) data generated with high speed (velocity), and huge in size (volume). Later, these tools and technologies were explored and used for handling structured data also but preferable for storage. Eventually, the processing of structured data was still kept as optional, either using big data or traditional RDBMSs. This helps in analyzing data towards effective usage of the hidden insights exposed from the data collected via social media, log files, sensors, etc. Big data draws from text, images, audio, video; plus it completes missing pieces through data fusion. Velocity The speed at which the data is generated and processed to meet the demands and challenges that lie in the path of growth and development. Big data is often available in real-time. Compared to small data, big data is produced more continually. Two kinds of velocity related to big data are the frequency of generation and the frequency of handling, recording, and publishing. Veracity The truthfulness or reliability of the data, which refers to the data quality and the data value. Big data must not only be large in size, but also must be reliable in order to achieve value in the analysis of it. The data quality of captured data can vary greatly, affecting an accurate analysis. Value The worth in information that can be achieved by the processing and analysis of large datasets. Value also can be measured by an assessment of the other qualities of big data. Value may also represent the profitability of information that is retrieved from the analysis of big data. Variability The characteristic of the changing formats, structure, or sources of big data. Big data can include structured, unstructured,

  Read more →
• 

  Cypherpunks (book)

  Cypherpunks: Freedom and the Future of the Internet is a 2012 book by Julian Assange, in discussion with Internet activists and cypherpunks Jacob Appelbaum, Andy Müller-Maguhn and Jérémie Zimmermann. Its primary topic is society's relationship with information security. In the book, the authors warn that the Internet has become a tool of the police state, and that the world is inadvertently heading toward a form of totalitarianism. They promote the use of cryptography to protect against state surveillance. In the introduction, Assange says that the book is "not a manifesto [...] [but] a warning". He told Guardian journalist Decca Aitkenhead: A well-defined mathematical algorithm can encrypt something quickly, but to decrypt it would take billions of years – or trillions of dollars' worth of electricity to drive the computer. So cryptography is the essential building block of independence for organisations on the Internet, just like armies are the essential building blocks of states, because otherwise one state just takes over another. There is no other way for our intellectual life to gain proper independence from the security guards of the world, the people who control physical reality. Assange later wrote in The Guardian: "Strong cryptography is a vital tool in fighting state oppression." saying that was the message of his book, Cypherpunks. Cypherpunks is published by OR Books. It is primarily a transcript of World Tomorrow episode eight, a two-part interview between Assange, Jacob Appelbaum, Andy Müller-Maguhn, and Jérémie Zimmermann. In the foreword, Assange said, "the Internet, our greatest tool for emancipation, has been transformed into the most dangerous facilitator of totalitarianism we have ever seen".

  Read more →
• 

  Codebook

  A codebook is a type of document used for gathering and storing cryptography codes. Originally, codebooks were often literally books, but today "codebook" is a byword for the complete record of a series of codes, regardless of physical format. == Cryptography == In cryptography, a codebook is a document used for implementing a code. A codebook contains a lookup table for coding and decoding; each word or phrase has one or more strings which replace it. To decipher messages written in code, corresponding copies of the codebook must be available at either end. The distribution and physical security of codebooks presents a special difficulty in the use of codes compared to the secret information used in ciphers, the key, which is typically much shorter. The United States National Security Agency documents sometimes use codebook to refer to block ciphers; compare their use of combiner-type algorithm to refer to stream ciphers. Codebooks come in two forms, one-part or two-part: In one-part codes, the plaintext words and phrases and the corresponding code words are in the same alphabetical order. They are organized similar to a standard dictionary. Such codes are half the size of two-part codes but are more vulnerable since an attacker who recovers some code word meanings can often infer the meaning of nearby code words. One-part codes may be used simply to shorten messages for transmission or have their security enhanced with superencryption methods, such as adding a secret number to numeric code words. In two-part codes, one part is for converting plaintext to ciphertext, the other for the opposite purpose. They are usually organized similarly to a language translation dictionary, with plaintext words (in the first part) and ciphertext words (in the second part) presented like dictionary headwords. The earliest known use of a codebook system was by Gabriele de Lavinde in 1379 working for the Antipope Clement VII. Two-part codebooks go back as least as far as Antoine Rossignol in the 1800s. From the 15th century until the middle of the 19th century, nomenclators (named after nomenclator) were the most used cryptographic method. Codebooks with superencryption were the most used cryptographic method of World War I. The JN-25 code used in World War II used a codebook of 30,000 code groups superencrypted with 30,000 random additives. The book used in a book cipher or the book used in a running key cipher can be any book shared by sender and receiver and is different from a cryptographic codebook. == Social sciences == In social sciences, a codebook is a document containing a list of the codes used in a set of data to refer to variables and their values, for example locations, occupations, or clinical diagnoses. == Data compression == Codebooks were also used in 19th- and 20th-century commercial codes for the non-cryptographic purpose of data compression. Codebooks are used in relation to precoding and beamforming in mobile networks such as 5G and LTE. The usage is standardized by 3GPP, for example in the document TS 38.331, NR; Radio Resource Control (RRC); Protocol specification.

  Read more →