AI Data Privacy Concerns

AI Data Privacy Concerns — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  Software engine

  A software engine is a core component of a complex software system. The word "engine" is a metaphor of a car's engine. Thus a software engine is a complex subsystem; not unlike how a car engine functions. Software engines work in conjunction with other components of a process or system. They typically have an input and an output, and the productivity is usually linear to running speed. There is no formal guideline for what should be called an engine, but the term has become widespread in the software industry. == Notable examples == === Multi-engine systems === Mainstream web browsers have both a browser engine and a JavaScript engine. Video games are often based on a game engine. Some of these also have specialized physics or graphics engines.

  Read more →
• 

  NYSERNet

  NYSERNet, Inc. (New York State Education and Research Network), is a non-profit Internet service provider in New York State. It mainly provides Internet access to universities, colleges, museums, health care facilities, primary and secondary schools, and research institutions. == History == NYSERNet was founded in 1986 in Troy, New York. Its founders compared NYSERNet's network with the Erie Canal and considered it the next step in two centuries to draw the country together. NYSERNet's network reaches from Buffalo to New York City. Completed in 1987, it was the first statewide regional IP network in the United States.[1] Initial speed of 56 kbps was upgraded to T1 in 1989 and T3 in 1994. It was the original assignee of AS174 according to RFC1117. This ASN is used today by Cogent Communications for their global network.

  Read more →
• 

  Cryptographic bill of materials

  Cryptographic bill of materials (CBOM—also cryptography bill of materials) is a structured inventory of all cryptographic assets present in a software, firmware, device, or system. It enumerates algorithms (and parameters such as key sizes and modes), cryptographic libraries or modules, digital certificates, keys and related material, and protocols in use, and maps their relationships to the components that implement or invoke them. CBOMs are used to improve security analysis, compliance, and cryptographic agility, and are increasingly referenced in guidance for post‑quantum cryptography (PQC) migration. == Definition and scope == A CBOM inventories cryptographic primitives and materials—such as encryption and signature algorithms (with specific variants and modes), key sizes, cryptographic libraries/modules, digital certificates (e.g., X.509), keys and other related cryptographic material, and security protocols (e.g., TLS, IPsec). It also documents dependencies (for example, an application uses an algorithm provided by a library; a protocol uses several algorithms) and can capture certificate lifecycles, cryptographic module certifications (e.g., FIPS 140‑3), and policy conformance metadata. In common practice, a CBOM may be embedded within an SBOM format (such as CycloneDX) or exported as a separate, linked artifact. === Typical CBOM fields === The exact schema varies by implementation, but common fields are summarized below (see CycloneDX CBOM guide and NIST SP 1800‑38B). == Relation to SBOM == A CBOM is complementary to, but distinct from, a software bill of materials (SBOM). Whereas an SBOM lists software components and their versions, a CBOM focuses specifically on the cryptography present and how it is configured and used. For example, an SBOM might enumerate inclusion of a library such as OpenSSL, while the CBOM would identify which algorithms and parameters that library enables (e.g., RSA‑2048, ECDH P‑256, AES‑GCM) and list relevant keys and certificates. The pairing enables both supply‑chain transparency and cryptographic transparency. == History == The term and practice emerged in the early–mid 2020s alongside software‑supply‑chain transparency and PQC planning. The OWASP CycloneDX standard introduced native CBOM support (v1.6 and later), modeling algorithms, keys, certificates, and protocols as first‑class “cryptographic assets” and providing dependency semantics (uses/implements) between software and cryptography. Open tooling from industry and researchers (e.g., IBM's CBOMkit and related generators/viewers) appeared to automate discovery and representation of cryptographic use in the CycloneDX CBOM schema. == Regulatory and policy context == In the United States, policy has emphasized cryptographic inventories as a prerequisite to PQC migration. The White House's National Security Memorandum 10 (2022) directed a government‑wide transition to quantum‑resistant cryptography; the Office of Management and Budget's M‑23‑02 (November 2022) operationalized this by requiring agencies to submit a prioritized inventory of cryptographic systems (with algorithm and key details) by 4 May 2023 and annually thereafter, and tasked CISA/NSA/NIST to develop automated discovery and inventory strategies. A 2024 Office of the National Cyber Director report reiterated that a “comprehensive cryptographic inventory” is the baseline for PQC planning and must be maintained iteratively with both automated and manual discovery. NIST's NCCoE practice guide (SP 1800‑38B, preliminary draft) provides concrete methods for cryptographic discovery and documentation across enterprises, aligning with CBOM‑style representations. CISA later published a strategy to migrate federal agencies to automated cryptography discovery and inventory tools to support continuous reporting. Separately, NSA, CISA, and NIST issued joint guidance encouraging all organisations to prepare cryptographic inventories and roadmaps for PQC, beyond government environments. == Role in quantum readiness and cryptographic agility == Because large‑scale quantum computing threatens widely used public‑key algorithms (e.g., RSA, ECC), organisations are planning multi‑year transitions to post-quantum cryptography. CBOMs enable that planning by identifying where quantum‑vulnerable algorithms appear, prioritising high‑impact systems, and tracking replacements over time. A machine‑readable CBOM also supports cryptographic agility and incident response: if an algorithm, library, or certificate lifecycle becomes non‑compliant or vulnerable, the CBOM indicates which products and systems are affected and where mitigations must be applied first. == Standards and tooling == CycloneDX (OWASP): Native CBOM modelling (v1.6+) for algorithms, certificates, keys/related material, and protocols, with dependency semantics and examples. The project publishes a CBOM guide and use‑case profiles (e.g., certificate and algorithm inventories). NIST NCCoE SP 1800‑38 series: Practice guides for PQC migration include enterprise cryptographic discovery methods that produce CBOM‑like inventories and integrate multiple discovery tools. Government automation initiatives: Following M‑23‑02, CISA issued a strategy to migrate to automated cryptography discovery and inventory tools to support agency reporting and continuous inventory management. Open‑source and vendor tools: IBM's CBOMkit and related components generate, analyse, and visualise CBOMs; the IBM CBOM specification work was upstreamed into CycloneDX 1.6. === Data model and interchange (example) === CycloneDX provides machine‑readable encodings (JSON/XML) for CBOM content. The example below (subset) shows an application depending on a crypto library that provides the AES‑256‑GCM algorithm, and the application also depends on a leaf X.509 certificate. See the CycloneDX CBOM guide, JSON reference, and the “Implementation details” use‑case for the semantics of `dependsOn` and `provides`. == Relationship to cybersecurity supply chain initiatives == CBOMs complement SBOM‑focused supply‑chain transparency introduced by U.S. Executive Order 14028 and NTIA/NIST SBOM work. SBOMs document software components; CBOMs add detail on embedded cryptography to support risk management, policy compliance (e.g., disallowing deprecated algorithms), and PQC transition planning.

  Read more →
• 

  Change data capture

  In databases, change data capture (CDC) is a set of software design patterns used to determine and track the data that has changed (the "deltas") so that action can be taken using the changed data. The result is a delta-driven dataset. CDC is an approach to data integration that is based on the identification, capture and delivery of the changes made to enterprise data sources. For instance it can be used for incremental update of data loading. CDC occurs often in data warehouse environments since capturing and preserving the state of data across time is one of the core functions of a data warehouse, but CDC can be utilized in any database or data repository system. == Methodology == System developers can set up CDC mechanisms in a number of ways and in any one or a combination of system layers from application logic down to physical storage. In a simplified CDC context, one computer system has data believed to have changed from a previous point in time, and a second computer system needs to take action based on that changed data. The former is the source, the latter is the target. It is possible that the source and target are the same system physically, but that would not change the design pattern logically. Multiple CDC solutions can exist in a single system. === Timestamps on rows === Tables whose changes must be captured may have a column that represents the time of last change. Names such as LAST_UPDATE, LAST_MODIFIED, etc. are common. Any row in any table that has a timestamp in that column that is more recent than the last time data was captured is considered to have changed. Timestamps on rows are also frequently used for optimistic locking so this column is often available. === Version numbers on rows === Database designers give tables whose changes must be captured a column that contains a version number. Names such as VERSION_NUMBER, etc. are common. One technique is to mark each changed row with a version number. A current version is maintained for the table, or possibly a group of tables. This is stored in a supporting construct such as a reference table. When a change capture occurs, all data with the latest version number is considered to have changed. Once the change capture is complete, the reference table is updated with a new version number. (Do not confuse this technique with row-level versioning used for optimistic locking. For optimistic locking each row has an independent version number, typically a sequential counter. This allows a process to atomically update a row and increment its counter only if another process has not incremented the counter. But CDC cannot use row-level versions to find all changes unless it knows the original "starting" version of every row. This is impractical to maintain.) === Status indicators on rows === This technique can either supplement or complement timestamps and versioning. It can configure an alternative if, for example, a status column is set up on a table row indicating that the row has changed (e.g., a boolean column that, when set to true, indicates that the row has changed). Otherwise, it can act as a complement to the previous methods, indicating that a row, despite having a new version number or a later date, still shouldn't be updated on the target (for example, the data may require human validation). === Time/version/status on rows === This approach combines the three previously discussed methods. As noted, it is not uncommon to see multiple CDC solutions at work in a single system, however, the combination of time, version, and status provides a particularly powerful mechanism and programmers should utilize them as a trio where possible. The three elements are not redundant or superfluous. Using them together allows for such logic as, "Capture all data for version 2.1 that changed between 2005-06-01 00:00 and 2005-07-01 00:00 where the status code indicates it is ready for production." === Triggers on tables === May include a publish/subscribe pattern to communicate the changed data to multiple targets. In this approach, triggers log events that happen to the transactional table into another queue table that can later be "played back". For example, imagine an Accounts table, when transactions are taken against this table, triggers would fire that would then store a history of the event or even the deltas into a separate queue table. The queue table might have schema with the following fields: Id, TableName, RowId, Timestamp, Operation. The data inserted for our Account sample might be: 1, Accounts, 76, 2008-11-02 00:15, Update. More complicated designs might log the actual data that changed. This queue table could then be "played back" to replicate the data from the source system to a target. Data capture offers a challenge in that the structure, contents and use of a transaction log is specific to a database management system. Unlike data access, no standard exists for transaction logs. Most database management systems do not document the internal format of their transaction logs, although some provide programmatic interfaces to their transaction logs (for example: Oracle, DB2, SQL/MP, SQL/MX and SQL Server 2008). Other challenges in using transaction logs for change data capture include: Coordinating the reading of the transaction logs and the archiving of log files (database management software typically archives log files off-line on a regular basis). Translation between physical storage formats that are recorded in the transaction logs and the logical formats typically expected by database users (e.g., some transaction logs save only minimal buffer differences that are not directly useful for change consumers). Dealing with changes to the format of the transaction logs between versions of the database management system. Eliminating uncommitted changes that the database wrote to the transaction log and later rolled back. Dealing with changes to the metadata of tables in the database. CDC solutions based on transaction log files have distinct advantages that include: minimal impact on the database (even more so if one uses log shipping to process the logs on a dedicated host). no need for programmatic changes to the applications that use the database. low latency in acquiring changes. transactional integrity: log scanning can produce a change stream that replays the original transactions in the order they were committed. Such a change stream include changes made to all tables participating in the captured transaction. no need to change the database schema == Confounding factors == As often occurs in complex domains, the final solution to a CDC problem may have to balance many competing concerns. === Unsuitable source systems === Change data capture both increases in complexity and reduces in value if the source system saves metadata changes when the data itself is not modified. For example, some Data models track the user who last looked at but did not change the data in the same structure as the data. This results in noise in the Change Data Capture. === Tracking the capture === Actually tracking the changes depends on the data source. If the data is being persisted in a modern database then Change Data Capture is a simple matter of permissions. Two techniques are in common use: Tracking changes using database triggers Reading the transaction log as, or shortly after, it is written. If the data is not in a modern database, CDC becomes a programming challenge. === Push versus pull === Push: the source process creates a snapshot of changes within its own process and delivers rows downstream. The downstream process uses the snapshot, creates its own subset and delivers them to the next process. Pull: the target that is immediately downstream from the source, prepares a request for data from the source. The downstream target delivers the snapshot to the next target, as in the push model. === Alternatives === Sometimes the slowly changing dimension is used as an alternative method. CDC and SCD are similar in that both methods can detect changes in a data set. The most common forms of SCD are type 1 (overwrite), type 2 (maintain history) or 3 (only previous and current value). SCD 2 can be useful if history is needed in the target system. CDC overwrites in the target system (akin to SCD1), and is ideal when only the changed data needs to arrive at the target, i.e. a delta-driven dataset.

  Read more →
• 

  CityEngine

  ArcGIS CityEngine is a commercial 3D modeling program. Developed by Esri R&D Center Zurich (formerly Procedural Inc.), it specializes in the generation of 3D urban environments to support the creation of detailed large-scale 3D city models. Unlike traditional 3D modeling methodology, which uses computer-aided design (CAD) tools and techniques, CityEngine takes a procedural modeling approach which shapes generation via a rules-based system. Due to its integration with the wider ArcGIS platform, CityEngine can also be used with geographic information system (GIS) datasets. CityEngine can be used for urban planning and architecture, graphics visualization, game development, entertainment, and archeology. CityEngine can be used to visualize the building information modeling (BIM) data of buildings in a larger urban context, making for more realistic construction projects. == History and releases == === Software history === ArcGIS CityEngine, originally named Esri CityEngine, was developed at Swiss technology university ETH Zurich by Pascal Mueller, the co-founder and CEO of Procedural Inc. While researching for his PhD at the ETH Computer Vision Lab, Mueller invented a number of techniques for procedural modeling of 3D architecture that make up the foundation of CityEngine. CityEngine publically debuted at the 2001 SIGGRAPH conference; since then, additional research papers have been published that have contributed to CityEngine and its features. The first commercial version of CityEngine was released in 2008. In 2007, Procedural Inc. was founded and separated from ETH Zurich, the top-ranking technology university in Switzerland. In the summer of 2011, Procedural Inc. was acquired by Esri Inc., becoming Esri R&D Center Zurich. Esri CityEngine was renamed to ArcGIS CityEngine in June 2020 to offically make it a part of the ArcGIS software suite. === Releases === === Licensing and pricing === ArcGIS CityEngine is included in the Professional and Professional Plus tiers of ArcGIS Online. Pricing may vary by region and distributors. In the US, the professional tier costs US$2,200 per year; in the UK, it is £4,200 per year (excluding VAT). CityEngine can be purchased elsewhere via a local Esri partner. . Once purchased, users can download and obtain license details from the MyEsri portal. == Features == CGA (computer generated architecture) parametric modeling rules to control mass, geometry assets, proportions, or texturing of buildings or streets on a citywide scale Select a target location and import geo-referenced satellite imagery and 3D terrain of the location to more quickly build accurate urban environments through OpenStreetMap integration Interactively control specific street or building parameters, such as height or age Import/export geo-spatial/vector data with industry-standard formats such as Esri Shapefile, File Geodatabase, and OpenStreetMap, as well as file formats for WebGL, KMZ, Collada, Autodesk FBX, Autodesk Maya, 3DS, Wavefront OBJ, RenderMan RIB, Alembic, e-on software's Vue, Universal Scene Description USD, Khronos Group GLTF, Unreal Engine, and Unreal Datasmith Script and generate rules-based reports to show socioeconomic figures (e.g., Gross Floor Area (GFA) and Floor Area Ratio (FAR)) to analyze their urban design proposals. VR viewing of modeled environments with Samsung Gear VR Use a variety of materials through the Esri materials library == Procedural modeling == ArcGIS CityEngine uses a procedural modeling approach to automatically generate models through a predefined rule set. The rules are defined through a CGA shape grammar system, enabling the creation of complex parametric models. Users can change or add the shape grammar as needed. Urban environments can be modeled within CityEngine by starting with creating a street network (either from the street drawing tool or with data imported from map data). Then, lots may be subdivided as many times as specified, resulting in a map of multiple lots and streets. CityEngine can then be instructed to start generating the buildings using defined procedural modeling rules. At this point, the city model can be re-designed and adjusted by changing the parameters or the shape grammar. === Geodesign === Though CityEngine is not an analytical tool like GIS, discussions about geodesign often mention the use of ArcGIS CityEngine. As it can be used to enhance 3D shape generation in ArcGIS, ArcGIS CityEngine is a critical product to improve the applicability of geodesign by using geospatial information to design or analyze a city. == Applications == === Urban design and planning === Garsdale Design used ArcGIS CityEngine in the creation of city master plans in Iraq before 2013, both to model existing historic areas and also model future plans. Larger companies like Foster+Partners and HOK Architects have also used CityEngine in their urban planning projects. === Urban and environmental studies === Because its primary feature is building informative city models, some urban researchers use CityEngine to compare land-use planning schemes, for example in very dense global cities such as Hong Kong and Seoul. Environmental scientists can also utilize the instant 3D model generation in CityEngine, which can make for more convenient informative research than modeling a city by creating each building individually. === Game development === CityEngine can be used as a tool in the creation of video games that require detailed 3D environments to assign interactive scripts. === Movie industry === Zootopia (also known outside of the US as Zootopolis), which won the 2016 Academy Award for Best Animated Feature Film, used CityEngine to model the city in its movie. multi-scaling city, the designers used CityEngine due to its rule-based system. CityEngine was also used to create Big Hero 6's San-Fransokyo. === Military === Due to its integration with the Esri product suite and its ability to process geospatial data to create 3D scenes/maps, CityEngine can be used within military/defense organizations. == List of movies and TV shows using CityEngine == Studios and companies rarely state what software they use in their pipelines. When CityEngine is mentioned as a tool in production, it's often in a small reference in a larger article. Movies only claimed to use CityEngine by a single Esri employee Presented at FMX 2025 workshop == Ports == ArcGIS CityEngine is built on top of Eclipse IDE, and has therefore able to be used on Windows and Linux operating systems. Support for macOS was stopped in March 2021. == Plugins and extensions == ArcGIS CityEngine currently works with a number of third party 3D modeling, rendering, and analytical software products via its SDK and API; these currently are: ArcGIS CityEngine for ArcGIS Urban: ArcGIS Urban Suite Puma: ArcGIS CityEngine for Rhinoceros 3D Palladio: ArcGIS CityEngine for Houdini Serlio: ArcGIS CityEngine for Maya PyPRT: ArcGIS CityEngine for Python ArcGIS CityEngine provides a Python scripting interface built on Jython (current version 2.7.0) which allows users to create their own tools and functionality. == Publications ==

  Read more →
• 

  Cryptographic Module Testing Laboratory

  Cryptographic Module Testing Laboratory (CMTL) is an information technology (IT) computer security testing laboratory that is accredited to conduct cryptographic module evaluations for conformance to the FIPS 140-2 U.S. Government standard. The National Institute of Standards and Technology (NIST) National Voluntary Laboratory Accreditation Program (NVLAP) accredits CMTLs to meet Cryptographic Module Validation Program (CMVP) standards and procedures. This has been replaced by FIPS 140-2 and the Cryptographic Module Validation Program (CMVP). == CMTL requirements == These laboratories must meet the following requirements: NIST Handbook 150, NVLAP Procedures and General Requirements NIST Handbook 150-17 Information Technology Security Testing - Cryptographic Module Testing NVLAP Specific Operations Checklist for Cryptographic Module Testing == FIPS 140-2 in relation to the Common Criteria == A CMTL can also be a Common Criteria (CC) Testing Laboratory (CCTL). The CC and FIPS 140-2 are different in the abstractness and focus of evaluation. FIPS 140-2 testing is against a defined cryptographic module and provides a suite of conformance tests to four FIPS 140 security levels. FIPS 140-2 describes the requirements for cryptographic modules and includes such areas as physical security, key management, self tests, roles and services, etc. The standard was initially developed in 1994 - prior to the development of the CC. The CC is an evaluation against a Protection Profile (PP), or security target (ST). Typically, a PP covers a broad range of products. A CC evaluation does not supersede or replace a validation to either FIPS 140-1, FIPS140-2 or FIPS 140-3. The four security levels in FIPS 140-1 and FIPS 140-2 do not map directly to specific CC EALs or to CC functional requirements. A CC certificate cannot be a substitute for a FIPS 140-1 or FIPS 140-2 certificate. If the operational environment is a modifiable operational environment, the operating system requirements of the Common Criteria are applicable at FIPS Security Levels 2 and above. FIPS 140-1 required evaluated operating systems that referenced the Trusted Computer System Evaluation Criteria (TCSEC) classes C2, B1 and B2. However, TCSEC is no longer in use and has been replaced by the Common Criteria. Consequently, FIPS 140-2 now references the Common Criteria. FIPS 140-2 or FIPS 140-3 validation efforts can be in some parts reused in Common Criteria evaluations, specifically in areas related to entropy source and cryptographic algorithms.

  Read more →
• 

  Virtual collective consciousness

  Virtual collective consciousness (VCC) is a term rebooted and promoted by two behavioral scientists, Yousri Marzouki and Olivier Oullier in their 2012 Huffington Post article titled: "Revolutionizing Revolutions: Virtual Collective Consciousness and the Arab Spring", after its first appearance in 1999-2000. VCC is now defined as an internal knowledge catalyzed by social media platforms and shared by a plurality of individuals driven by the spontaneity, the homogeneity, and the synchronicity of their online actions. VCC occurs when a large group of persons, brought together by a social media platform think and act with one mind and share collective emotions. Thus, they are able to coordinate their efforts efficiently, and could rapidly spread their word to a worldwide audience. When interviewed about the concept of VCC that appeared in the book - Hyperconnectivity and the Future of Internet Communication - he edited, Professor of Pervasive Computing, Adrian David Cheok mentioned the following: "The idea of a global (collective) virtual consciousness is a bottom-up process and a rather emergent property resulting from a momentum of complex interactions taking place in social networks. This kind of collective behaviour (or intelligence) results from a collision between a physical world and a virtual world and can have a real impact in our life by driving collective action." == Etymology == In 1999-2000, Richard Glen Boire provided a cursory mention and the only occurrence of the term "Virtual collective consciousness" in his text as follows: The trend of technology is to overcome the limitations of the human body. And, the Web has been characterized as a virtual collective consciousness and unconsciousness The recent definition of VCC evolved from the first empirical study that provided a cyberpsychological insight into the contribution of Facebook to the 2011 Tunisian revolution. In this study, the concept was originally called "collective cyberconsciousness". The latter is an extension of the idea of "collective consciousness" coupled with "citizen media" usage. The authors of this study also made a parallel between this original definition of VCC and other comparable concepts such as Durkheim's collective representation, Žižek's "collective mind" or Boguta's "new collective consciousness" that he used to describe the computational history of the Internet shutdown during the Egyptian revolution. Since VCC is the byproduct of the network's successful actions, then these actions must be timely, acute, rapid, domain-specific, and purpose-oriented to successfully achieve their goal. Before reaching a momentum of complexity, each collective behavior starts by a spark that triggers a chain of events leading to a crystallized stance of a tremendous amount of interactions. Thus, VCC is an emergent global pattern from these individual actions. In 2012, the term virtual collective consciousness resurfaced and was brought to light after extending its applications to the Egyptian case and the whole social networking major impact on the success of the so-called Arab Spring. Moreover, the acronym VCC was suggested to identify the theoretical framework covering on-line behaviors leading to a virtual collective consciousness. Hence, online social networks have provided a new and faster way of establishing or modifying "collective consciousness" that was paramount to the 2011 uprisings in the Arab world. == Theoretical underpinnings of VCC == Various theoretical references in fields ranging from sociology to computer science were mentioned in order to account for the key features that render the framework for a virtual collective consciousness. The following list is not exhaustive, but the references it contains are often highlighted: Émile Durkheim's collective representations are at the heart of VCC since collectivity taken decisions according to Durkheim's assumptions will approve or disapprove individuals' actions and help them eventually reach their final goal. Marshall McLuhan's global village: The shrinking of our big world to a small place called cyberspace is made possible by technological extensions of human consciousness. Carl Jung's collective unconscious: When a society witnesses significant changes, the anchoring of archetypal images (e.g., political leaders) seems to be deeply rooted in individuals' collective unconscious that is likely to bias their political choices. Individual memories of public events were also supposed to convey a "collective awareness" that can be subconsciously altered by the instantaneous spread of information through social networking around the world. Daniel Wegner's transactive memory (TM): social-networking platforms such as Facebook during the Tunisian revolution or Twitter during the Egyptian revolution served as placeholders of a VCC where information can be harnessed and steered to the highly specific revolutionary purpose. Although research on TM was originally limited to couples, small groups, and organizations, recent studies strongly suggest that an effective TM can operate on a very large scale too. James Surowiecki's wisdom of crowds Collective influence algorithm: The CI (Collective influence) algorithm is effective in finding influential nodes in a variety of networks, including social networks, communication networks, and biological networks. It has been used to identify influencers on social-media platforms, to identify key nodes in transportation networks, and to identify potential drug-targets in biological networks. == Some illustrations of VCC == Besides the studied effect of social networking on the Tunisian and Egyptian revolutions, the former via Facebook and the latter via Twitter other applications were studied under the prism of VCC framework: The Whitacre's virtual choir: A compelling example of the degree of autonomy and self-identity members of a spontaneously created network through a VCC is Eric Whitacre's unique musical project that involved a collection of singers performing remotely to create a virtual Choir. The effect of all the voices illustrated a genuine virtual collective empathy merging the artist's mind with all the singers through his silent conducting gestures. The Harlem Shake dance: The Bitcoin protocol: It was questioned whether or not the Bitcoin protocol can morph into virtual collective consciousness. The Byzantine generals problem was used as an analogy to understand the behavioral complexity of the community of Bitcoin's users. Artificial Social Networking Intelligence (ASNI): refers to the application of artificial intelligence within social networking services and social media platforms. It encompasses various technologies and techniques used to automate, personalize, enhance, improve, and synchronize users' interactions and experiences within social networks. ASNI is expected to evolve rapidly, influencing how we interact online and shaping our digital experiences. Transparency, ethical considerations, media influence bias, and user control over data will be crucial to ensure responsible development and positive impact.

  Read more →
• 

  WhoSay

  WhoSay was an American social media service and branding platform for celebrities and their fans. Founded in Los Angeles in 2010, with financing by Creative Artists Agency (CAA), Amazon.com and other investors, it is notable for allowing its users to retain ownership rights over the content that they post to their accounts, through copyright branding, and for enabling users to post content to other social media sites like Twitter, Facebook, Instagram and Tumblr simultaneously. WhoSay describes itself as a "social celebrity magazine" whose editorial team keeps its users informed about the latest celebrity and entertainment news. Clients such as Dylan McDermott and Chris Rock lauded the service for its ability to add content to multiple social network sites easily. Rock in particular has commented on its ease of use for those who are not part of a tech-savvy demographic, commenting, "It's perfect for someone that's not 25." WhoSay's competitors included theAudience, which is operated by the William Morris Endeavor. == History == WhoSay was founded in March 2010, by Steve Ellis and the Los Angeles-based talent agency Creative Artists Agency (CAA). It was financed through investments Amazon.com (who along with CAA, holds a minority stake in the company), Comcast, Greylock Partners, and High Peak Ventures. The company's main headquarters are in The New York Times Building in Manhattan, with additional headquarters in CAA's office building in the Silicon Beach area of Los Angeles, and in London. The company was founded to protect celebrities' intellectual property and enable the celebrities themselves to profit themselves from their own content through copyright branding. Its chief executive is co-founder Steve Ellis, who, after leaving Getty Images, was contacted by CAA, who were looking to resolve the issue of celebrities losing the rights to their own photos and videos when uploading them to social network sites. Ellis explained WhoSay's mission thus: "We work with people who are constantly being utilized by third parties for the wrong reasons. [The company was formed] to give celebrities and other influential people a set of tools to allow them to manage and control their presence in the digital world." In this way, WhoSay is likened by Ellis to "a People magazine by the people themselves who are in it." The company started slowly, until CAA client Tom Hanks signed onto WhoSay three months after the service's launch. The company continued to maintain a low profile for the first three years of operation, during which it accumulated a client list of 1,500 actors, musicians and artists. Clients are accepted by the service on an invitation-only basis, although they are not restricted to Creative Artists clients. Among them are Kelly Clarkson, Julia Louis-Dreyfus, Paula Patton, Kevin Spacey, Jim Carrey, John Cusack, Bill Maher, Johnny Knoxville, Chelsea Handler, Eva Longoria, Spike Lee, Enrique Iglesias and Katie Couric. Clients are not charged for the service, and are given a share of any revenue that is generated by advertisements. They are also given the ability share in the database of e-mail addresses that come with registration, in order to communicate directly with fans. Actor Dylan McDermott was introduced to WhoSay by his agent, as a way of easily posting content to Facebook, Twitter, Tumblr and even China's Tencent social network with relative ease. McDermott comments, "When you put something out there, you can hit everything at one time. It makes it easy for me." Comedian Chris Rock has commented that WhoSay is ideal for people like him have developed difficulty in keeping track of different websites as they get older, saying, "It's perfect for someone that's not 25." In September 2013 WhoSay introduced a mobile application for consumers. By October 2013, the company's website attracted 12 million monthly visitors. In July 2014 Rob Gregory left his role as president of Newsweek's The Daily Beast to become WhoSay's chief revenue officer. Among his responsibilities are developing ways to monetize WhoSay's web and mobile products, such as premium advertising strategies and brand partnerships. WhoSay does not allow consumers to create accounts, nor does it include search features, making it difficult to access a celebrity's account unless a user is directed there from one of their other social pages. According to Ellis, consumers have enough social media choices, saying, "Frankly they don't really need the services that we provide, and there are a lot of very specific features built into our service that really only benefit someone who is of a high profile." By February 2015, WhoSay had amassed 4.8 million unique users, and expanded its accounts to companies that employ celebrities for branded content. Such companies include Lexus, which partnered with the company to promote a campaign in which actress Rosario Dawson, during the lead up to the 87th Academy Awards, released five short videos on her social media accounts. The videos feature her driving through Los Angeles in preparation for the grand opening of her pop-up store, which sells Studio One Eighty Nine, a clothing line tied to her foundation promoting African culture and content. That April, WhoSay partnered with Chevrolet's #BestDayEver social media campaign for April Fool's Day, enlisting Olivia Wilde, Norman Reedus, Alec Baldwin, Ian Somerhalder, and Nikki Reed to surprise students in four U.S. classrooms as their substitute teachers. For example, Baldwin, dressed as Abraham Lincoln, surprised students in an Occidental College class on U.S. Culture and Society. Other companies that WhoSay has partnered with include KFC, JCPenney, Dunkin' Donuts and Crest. In January 2018, the website was acquired by Viacom (now Paramount Global).

  Read more →
• 

  Pixlr

  Pixlr is a group of SaaS creative tools including Pixlr.com, Designs.ai and Vectr.com. Pixlr.com is a cloud-based set of image editing tools and utilities, including AI image generation and enhancements. The Pixlr suite targets users who require subjectively simple, or more advanced, photo editing as well as graphic design. It features a freemium business model with subscription plans—Plus, Premium and Teams. The platform can be used on desktop and also smartphones and tablets. Pixlr is compatible with various image formats such as JPEG, PNG, WEBP, GIF, PSD (Photoshop Document) and PXZ (native Pixlr document format). Designs.ai lets users create content using AI, with a goal of being within two minutes, across different media types including videos, text, banners and audio. Vectr.com was acquired in 2017 before being spun out into Pixlr Group in 2023. == History == Pixlr was founded in 2008 and built on Macromedia Flash. On 19 July 2011, Autodesk announced that they had acquired the Pixlr suite. In 2013, Time listed Pixlr as one of the top 50 websites of the year. In 2017, Pixlr was acquired from Autodesk. It was subsequently rebuilt and relaunched in HTML5 in 2019. In September 2023, Pixlr was awarded as the Top 13 GenAi Web Product by the world's top venture firm Andreessen Horowitz. In November 2023, Pixlr, Designs.ai and Vectr were combined as a new business group named Pixlr Group focusing on generative AI and creative software solutions. In May 2024, Pixlr was featured as one of the top 18 progressive web applications highlighted on Google I/O. == Versions == Pixlr.com rebranded itself as a full creative suite in 2019 by introducing Pixlr X, Pixlr E and Pixlr M. The platform introduced more features in December 2021 with a new logo and added tools which included: Brushes, the 'Heal tool', Animation, and Batch upload. The brush feature enables the creation of hand-drawn effects. The Heal tool allows users to remove unwanted objects from their images whereas the Animation feature can be used to include movements into their edits. Users can also utilize Batch upload to edit up to 50 images simultaneously. In November 2022, Pixlr 2023 was launched, adding more tools such as "AI smart resize", colorization, text wrapping and other additional effects. In November 2023, Pixlr 2024 was launched with Pixlr Designer and new AI-powered updates which includes AI image generation, AI infill, AI inpainting and more.

  Read more →
• 

  Personal network

  A personal network is a set of human contacts known to an individual, with whom that individual would expect to interact at intervals to support a given set of activities. In other words, a personal network is a group of caring, dedicated people who are committed to maintain a relationship with a person in order to support a given set of activities. Having a strong personal network requires being connected to a network of resources for mutual development and growth. Personal networks can be understood by: who knows you what you know about them what they know about you what are you learning together how you work at that Personal networks are intended to be mutually beneficial, extending the concept of teamwork beyond the immediate peer group. The term is usually encountered in the workplace, though it could apply equally to other pursuits outside work. Personal networking is the practice of developing and maintaining a personal network, which is usually undertaken over an extended period. The concept is related to business networking and is often encouraged by large organizations, in the hope of improving productivity, and so a number of tools exist to support the maintenance of networks. Many of these tools are IT-based, and use Web 2.0 technologies. == History of networking and business success == In the second half of the twentieth century, U.S. advocates for workplace equity popularized the term and concept of networking as part of a larger social capital lexicon—which also includes terms such as glass ceiling, role model, mentoring, and gatekeeper—serving to identify and address the problems barring non-dominant groups from professional success. Mainstream business literature subsequently adopted the terms and concepts, promoting them as pathways to success for all career climbers. In 1970 these terms were not in the general American vocabulary; by the mid-1990s they had become part of everyday speech. Before the mid-twentieth century, what we call networking today was framed in the language of family and friendship. These close personal relationships provided a range of opportunities to preferred subsets of people, such as access to job opportunities, information, credit, and partnerships. Family networks and nepotism have proven particularly strong throughout history. However, other common bonds—from ethnicity and religion to school ties and club memberships—can connect subsets of people as well. Of course people whom insiders consider undesirable have been barred from such networks, with important consequences. Those who tap into influential networks can be nurtured toward success. Those who are shut out from networks can lose hope of success. Numerous business heroes of the past—such as Benjamin Franklin, Andrew Carnegie, Henry Ford, and John D. Rockefeller—exploited networks to great effect. The business networks that seemed natural and transparent to these white men were a closed book to women and minorities for much of American history. Drawing on work from the social sciences, these outsider groups had to identify and then harness the mechanisms behind networking's power. A prominent early example of this process was the formation of corporate caucuses by black men at Xerox starting in 1969. Groups of black salesmen met regularly to share information about Xerox's culture and strategies for navigating it most effectively. Through confrontation and collaboration with a relatively accommodating upper management, the caucuses helped open opportunities for high-performing black employees. The popular and business press began using the terms "network" and "networking" in the mid-1970s in the context of businesswomen consciously pursuing this strategy. Authors encouraged female workers to recognize and exploit the informal workplace systems that provided advancement. They urged women to identify mentors, use social contacts, and build peer and authority networks. The push for networking drew on ideas and relationships from the era's feminist movement, and dictionaries of the time explicitly linked business networking to women's efforts to succeed in the workplace. Since the closing decades of the twentieth century, networking has become a pervasive term and concept in American society. People now invoke networking in relation to everything from business to child rearing to science. While ambitious careerists seek networks as an indispensable talisman, companies purposefully encourage networking among their employees to boost performance and gain competitive advantage. At the same time, Americans are forgetting the workplace activism that first illuminated the power of networking. Unfortunately, this loss of historical context can fuel a backlash against outsider groups who still seek to synthesize networks so they can access the same opportunities enjoyed by insiders. == Characteristics of networks == Broadly speaking, all networks have the following characteristics: Purpose – A network can be established for learning, mission, business, idea, and family or personal reasons. Structure – A network is a group of interlinked entities that form a cluster. Most social structures tend to be characterized by dense clusters of strong connections. Style – The place, space, pace and style of interaction of the networks give an understanding of the style of the networks. Namkee Park, Seungyoon Lee and Jang Hyun Kim examined the relations between personal network characteristics and Facebook use. According to their study, personal networks are investigated through several structural characteristics, which can be categorized into three major dimensions according to the level of analysis: Dyadic tie attributes which include the characteristics of ego-alter ties such as duration, multiplexity, and proximity. Ego-alter tie attributes represent various dimensions of relationships between the focal person and their close contacts. First, tie duration refers to the length of time since the tie was originally initiated, which indicates the duration of relationships. Second, multiplexity includes a focal individual's degree of involvement in various types of interactions with network members. The third dimension is the physical proximity between ego and alter. Theories of proximity suggest that physical proximity between people affects their interaction and subsequently, their formation of network ties. The characteristics of alter-alter ties including personal network density. When moving to ties at the alter-alter level, ego-network density, which refers to the extent to which one's alters are connected with each other, is an important dimension of personal networks. Dense personal network structure indicates close interpersonal contacts among alters, and consequently, is considered to promote the sharing of resources. On the other hand, loose connections, or structural holes in ego-networks, have been found to facilitate the flow of information and to provide advantages in searching and obtaining resources (e.g., getting a job). The composition of alter attributes centered on the heterogeneity of alters in one's personal network. The heterogeneity of alters in one's personal network is associated with access to diverse resources and information It is expected, thus, that the heterogeneity attributes may enhance the focal actor's social activities. Each of these characteristics represents unique aspects of individuals' network relationships. == Types of personal networks == Personal networks can be used for two main reasons: social and professional. In 2012, LinkedIn along with TNS conducted a survey of 6,000 social network users to understand the difference between personal social networks and personal professional networks. The "Mindset Divide" of users of these networks was compared as follows: Emotions: Personal social networks: Nostalgia, fun, distraction. Personal professional networks: Achievement, success, aspiration. Use: Personal social networks: Users are in a casual mindset often just passing time. They use social networks to socialize, stay in touch, be entertained and kill time. Personal professional networks: In this purposeful mindset, users invest time to improve themselves and their future. These networks are used to maintain professional identity, make useful contacts, search for opportunities and stay in touch. Content: Personal professional networks: These provide information about career, brand updates and current affairs. Professional development: Personal development networks: These provide access to those who can provide information, knowledge, advice, support, expertise, guidance, and concrete resources to learn and work effectively—thus those who support the continuing professional development. == Personal network management == Personal network management (PNM) is a crucial aspect of personal information management and can be understood as the practice of managing the links and connections for social and profession

  Read more →
• 

  Time-lock puzzle

  A time-lock puzzle, or time-released cryptography, encrypts a message that cannot be decrypted until a specified amount of time has passed. The concept was first described by Timothy C. May, and a solution first introduced by Ron Rivest, Adi Shamir, and David A. Wagner in 1996. Time-lock puzzle are useful in cases where confidentiality of information is determined by time, such as a diarist who does not want their views released until 50 years after their death, an auction where bids are sealed until the bidding period is closed, electronic voting, and contract signing. They can additionally be used in creating further cryptographic primitives, such as verifiable delay functions and zero knowledge proofs. Time-released cryptography can be achieved through several different mechanisms. Use mathematical problems requiring sequential calculations to solve, and cannot be solved with parallelization. Thus, adding more computers to a problem will not help solve the problem faster. Use of a trusted agent, or multiple agents who each hold a part of the message and cryptographic keys, who release the message after a specified time period has passed. Distribute public encryption keys to users, and place private cryptographic keys with a trusted agent in an offline location, to be released at a later date.

  Read more →
• 

  Factorization of polynomials over finite fields

  In mathematics and computer algebra the factorization of a polynomial consists of decomposing it into a product of irreducible factors. This decomposition is theoretically possible and is unique for polynomials with coefficients in any field, but rather strong restrictions on the field of the coefficients are needed to allow the computation of the factorization by means of an algorithm. In practice, algorithms have been designed only for polynomials with coefficients in a finite field, in the field of rationals or in a finitely generated field extension of one of them. All factorization algorithms, including the case of multivariate polynomials over the rational numbers, reduce the problem to this case; see polynomial factorization. It is also used for various applications of finite fields, such as coding theory (cyclic redundancy codes and BCH codes), cryptography (public key cryptography by the means of elliptic curves), and computational number theory. As the reduction of the factorization of multivariate polynomials to that of univariate polynomials does not have any specificity in the case of coefficients in a finite field, only polynomials with one variable are considered in this article. == Background == === Finite field === The theory of finite fields, whose origins can be traced back to the works of Gauss and Galois, has played a part in various branches of mathematics. Due to the applicability of the concept in other topics of mathematics and sciences like computer science there has been a resurgence of interest in finite fields and this is partly due to important applications in coding theory and cryptography. Applications of finite fields introduce some of these developments in cryptography, computer algebra and coding theory. A finite field or Galois field is a field with a finite order (number of elements). The order of a finite field is always a prime or a power of prime. For each prime power q = pr, there exists exactly one finite field with q elements, up to isomorphism. This field is denoted GF(q) or Fq. If p is prime, GF(p) is the prime field of order p; it is the field of residue classes modulo p, and its p elements are denoted 0, 1, ..., p−1. Thus a = b in GF(p) means the same as a ≡ b (mod p). === Irreducible polynomials === Let F be a finite field. As for general fields, a non-constant polynomial f in F[x] is said to be irreducible over F if it is not the product of two polynomials of positive degree. A polynomial of positive degree that is not irreducible over F is called reducible over F. Irreducible polynomials allow us to construct the finite fields of non-prime order. In fact, for a prime power q, let Fq be the finite field with q elements, unique up to isomorphism. A polynomial f of degree n greater than one, which is irreducible over Fq, defines a field extension of degree n which is isomorphic to the field with qn elements: the elements of this extension are the polynomials of degree lower than n; addition, subtraction and multiplication by an element of Fq are those of the polynomials; the product of two elements is the remainder of the division by f of their product as polynomials; the inverse of an element may be computed by the extended GCD algorithm (see Arithmetic of algebraic extensions). It follows that, to compute in a finite field of non prime order, one needs to generate an irreducible polynomial. For this, the common method is to take a polynomial at random and test it for irreducibility. For sake of efficiency of the multiplication in the field, it is usual to search for polynomials of the shape xn + ax + b. Irreducible polynomials over finite fields are also useful for pseudorandom number generators using feedback shift registers and discrete logarithm over F2n. The number of irreducible monic polynomials of degree n over Fq is the number of aperiodic necklaces, given by Moreau's necklace-counting function Mq(n). The closely related necklace function Nq(n) counts monic polynomials of degree n which are primary (a power of an irreducible); or alternatively irreducible polynomials of all degrees d which divide n. === Example === The polynomial P = x4 + 1 is irreducible over Q but not over any finite field. On any field extension of F2, P = (x + 1)4. On every other finite field, at least one of −1, 2 and −2 is a square, because the product of two non-squares is a square and so we have If − 1 = a 2 , {\displaystyle -1=a^{2},} then P = ( x 2 + a ) ( x 2 − a ) . {\displaystyle P=(x^{2}+a)(x^{2}-a).} If 2 = b 2 , {\displaystyle 2=b^{2},} then P = ( x 2 + b x + 1 ) ( x 2 − b x + 1 ) . {\displaystyle P=(x^{2}+bx+1)(x^{2}-bx+1).} If − 2 = c 2 , {\displaystyle -2=c^{2},} then P = ( x 2 + c x − 1 ) ( x 2 − c x − 1 ) . {\displaystyle P=(x^{2}+cx-1)(x^{2}-cx-1).} === Complexity === Polynomial factoring algorithms use basic polynomial operations such as products, divisions, gcd, powers of one polynomial modulo another, etc. A multiplication of two polynomials of degree at most n can be done in O(n2) operations in Fq using "classical" arithmetic, or in O(nlog(n) log(log(n)) ) operations in Fq using "fast" arithmetic. A Euclidean division (division with remainder) can be performed within the same time bounds. The cost of a polynomial greatest common divisor between two polynomials of degree at most n can be taken as O(n2) operations in Fq using classical methods, or as O(nlog2(n) log(log(n)) ) operations in Fq using fast methods. For polynomials h, g of degree at most n, the exponentiation hq mod g can be done with O(log(q)) polynomial products, using exponentiation by squaring method, that is O(n2log(q)) operations in Fq using classical methods, or O(nlog(q)log(n) log(log(n))) operations in Fq using fast methods. In the algorithms that follow, the complexities are expressed in terms of number of arithmetic operations in Fq, using classical algorithms for the arithmetic of polynomials. == Factoring algorithms == Many algorithms for factoring polynomials over finite fields include the following three stages: Square-free factorization Distinct-degree factorization Equal-degree factorization An important exception is Berlekamp's algorithm, which combines stages 2 and 3. === Berlekamp's algorithm === Berlekamp's algorithm is historically important as being the first factorization algorithm which works well in practice. However, it contains a loop on the elements of the ground field, which implies that it is practicable only over small finite fields. For a fixed ground field, its time complexity is polynomial, but, for general ground fields, the complexity is exponential in the size of the ground field. === Square-free factorization === The algorithm determines a square-free factorization for polynomials whose coefficients come from the finite field Fq of order q = pm with p a prime. This algorithm firstly determines the derivative and then computes the gcd of the polynomial and its derivative. If it is not one then the gcd is again divided into the original polynomial, provided that the derivative is not zero (a case that exists for non-constant polynomials defined over finite fields). This algorithm uses the fact that, if the derivative of a polynomial is zero, then it is a polynomial in xp, which is, if the coefficients belong to Fp, the pth power of the polynomial obtained by substituting x by x1/p. If the coefficients do not belong to Fp, the pth root of a polynomial with zero derivative is obtained by the same substitution on x, completed by applying the inverse of the Frobenius automorphism to the coefficients. This algorithm works also over a field of characteristic zero, with the only difference that it never enters in the blocks of instructions where pth roots are computed. However, in this case, Yun's algorithm is much more efficient because it computes the greatest common divisors of polynomials of lower degrees. A consequence is that, when factoring a polynomial over the integers, the algorithm which follows is not used: one first computes the square-free factorization over the integers, and to factor the resulting polynomials, one chooses a p such that they remain square-free modulo p. Algorithm: SFF (Square-Free Factorization) Input: A monic polynomial f in Fq[x] where q = pm Output: Square-free factorization of f R ← 1 # Make w be the product (without multiplicity) of all factors of f that have # multiplicity not divisible by p c ← gcd(f, f′) w ← f/c # Step 1: Identify all factors in w i ← 1 while w ≠ 1 do y ← gcd(w, c) fac ← w / y R ← R · faci w ← y; c ← c / y; i ← i + 1 end while # c is now the product (with multiplicity) of the remaining factors of f # Step 2: Identify all remaining factors using recursion # Note that these are the factors of f that have multiplicity divisible by p if c ≠ 1 then c ← c1/p R ← R·SFF(c)p end if Output(R) The idea is to identify the product of all irreducible factors of f with the same multiplicity. This is done in two steps. The first step uses the formal d

  Read more →
• 

  Lenny (chatbot)

  Lenny is a chatbot designed to scam bait telemarketers, scammers, and other unwanted incoming calls using messages. == Background == Telemarketers may be perceived by some as annoying and wasting people's time, and some deliberately attempt to scam or defraud people. In April 2018, stats published by YouMail estimated the United States received over three billion robocalls that month. Attempts to block the callers have been hindered by Caller ID spoofing. == Features == The bot was written in 2011, and development taken over by an Alberta-based programmer known as "Mango" two years later. It is driven by sixteen pre-recorded audio clips, spoken in a soft and slow Australian accent in the manner of an elderly man. The bot's original creator stated on Reddit that in building the character he asked himself the question "What would be a telemarketer's worst nightmare?" He answered with this being a lonely old man who is up for a chat, proud of his family and can't focus on the telemarketer's goal. There is no speech recognition or artificial intelligence, and the bot's software is simple and straightforward. The first four clips are played sequentially in order to grab the telemarketer's interest and begin their sales pitch to Lenny, then the remaining twelve are played sequentially on loop until the telemarketer hangs up. The program waits for a gap of 1.5 seconds of silence before playing the next audio clip, to simulate natural breaks in the conversation. The messages are purposefully vague and open-ended so they can be applied to as many conversations as possible. They include references to Lenny's children, the state of the economy, and being interrupted by some ducks outside. According to research into the bot, around 75% of callers realise they are talking to a computer program within two minutes; however, some calls have lasted around an hour. == Distribution == Though other chatbots had been developed earlier, Lenny was the first one to be released for free on a public server and could be accessed by anyone. Recordings of conversations with the bot are widely shared online on websites such as Reddit and YouTube. Though "Mango" only intended Lenny to be used against dishonest telemarketers, such as scammers, he does not mind it being used against callers who are merely annoying. The bot has also been used against political campaigners, such as a supporter of Pierre Poilievre in the 2015 Canadian federal election.

  Read more →
• 

  Data integration

  Data integration is the process of combining, sharing, or synchronizing data from multiple sources to provide users with a unified view. There are a wide range of possible applications for data integration, from commercial (such as when a business merges multiple databases) to scientific (combining research data from different bioinformatics repositories). The decision to integrate data tends to arise when the volume, complexity (that is, big data) and need to share existing data explodes. It has become the focus of extensive theoretical work, and numerous open problems remain unsolved. Data integration encourages collaboration between internal as well as external users. The data being integrated must be received from a heterogeneous database system and transformed to a single coherent data store that provides synchronous data across a network of files for clients. A common use of data integration is in data mining when analyzing and extracting information from existing databases that can be useful for Business information. == History == Issues with combining heterogeneous data sources, often referred to as information silos, under a single query interface have existed for some time. In the early 1980s, computer scientists began designing systems for interoperability of heterogeneous databases. The first data integration system driven by structured metadata was designed in 1991 at the University of Minnesota for the Integrated Public Use Microdata Series (IPUMS). IPUMS used a data warehousing approach, which extracts, transforms, and loads data from heterogeneous sources into a unique view schema so data from different sources become compatible. By making thousands of population databases interoperable, IPUMS demonstrated the feasibility of large-scale data integration. The data warehouse approach offers a tightly coupled architecture because the data are already physically reconciled in a single queryable repository, so it usually takes little time to resolve queries. The data warehouse approach is less feasible for data sets that are frequently updated, requiring the extract, transform, load (ETL) process to be continuously re-executed for synchronization. Difficulties also arise in constructing data warehouses when one has only a query interface to summary data sources and no access to the full data. This problem frequently emerges when integrating several commercial query services like travel or classified advertisement web applications. A trend began in 2009 favoring the loose coupling of data and providing a unified query-interface to access real time data over a mediated schema (see Figure 2), which allows information to be retrieved directly from original databases. This is consistent with the SOA approach popular in that era. This approach relies on mappings between the mediated schema and the schema of original sources, and translating a query into decomposed queries to match the schema of the original databases. Such mappings can be specified in two ways: as a mapping from entities in the mediated schema to entities in the original sources (the "Global-as-View" (GAV) approach), or as a mapping from entities in the original sources to the mediated schema (the "Local-as-View" (LAV) approach). The latter approach requires more sophisticated inferences to resolve a query on the mediated schema, but makes it easier to add new data sources to a (stable) mediated schema. As of 2010, some of the work in data integration research concerns the semantic integration problem. This problem addresses not the structuring of the architecture of the integration, but how to resolve semantic conflicts between heterogeneous data sources. For example, if two companies merge their databases, certain concepts and definitions in their respective schemas like "earnings" inevitably have different meanings. In one database it may mean profits in dollars (a floating-point number), while in the other it might represent the number of sales (an integer). A common strategy for the resolution of such problems involves the use of ontologies which explicitly define schema terms and thus help to resolve semantic conflicts. This approach represents ontology-based data integration. On the other hand, the problem of combining research results from different bioinformatics repositories requires bench-marking of the similarities, computed from different data sources, on a single criterion such as positive predictive value. This enables the data sources to be directly comparable and can be integrated even when the natures of experiments are distinct. As of 2011, it was determined that current data modeling methods were imparting data isolation into every data architecture in the form of islands of disparate data and information silos. This data isolation is an unintended artifact of the data modeling methodology that results in the development of disparate data models. Disparate data models, when instantiated as databases, form disparate databases. Enhanced data model methodologies have been developed to eliminate the data isolation artifact and to promote the development of integrated data models. One enhanced data modeling method recasts data models by augmenting them with structural metadata in the form of standardized data entities. As a result of recasting multiple data models, the set of recast data models will now share one or more commonality relationships that relate the structural metadata now common to these data models. Commonality relationships are a peer-to-peer type of entity relationships that relate the standardized data entities of multiple data models. Multiple data models that contain the same standard data entity may participate in the same commonality relationship. When integrated data models are instantiated as databases and are properly populated from a common set of master data, then these databases are integrated. Since 2011, data hub approaches have been of greater interest than fully structured (typically relational) Enterprise Data Warehouses. Since 2013, data lake approaches have risen to the level of Data Hubs. (See all three search terms popularity on Google Trends.) These approaches combine unstructured or varied data into one location, but do not necessarily require an (often complex) master relational schema to structure and define all data in the Hub. In recent times, as the number of applications being used have increased many fold and application to application integration have become critical and this has given rise to [Unified APIs] that help application developers integrate their apps with other apps and more recently with [MCP - Model Context Protocol] taking it a step further for AI Agents. Data integration plays a big role in business regarding data collection used for studying the market. Converting the raw data retrieved from consumers into coherent data is something businesses try to do when considering what steps they should take next. Organizations are more frequently using data mining for collecting information and patterns from their databases, and this process helps them develop new business strategies to increase business performance and perform economic analyses more efficiently. Compiling the large amount of data they collect to be stored in their system is a form of data integration adapted for Business intelligence to improve their chances of success. == Example == Consider a web application where a user can query a variety of information about cities (such as crime statistics, weather, hotels, demographics, etc.). Traditionally, the information must be stored in a single database with a single schema. But any single enterprise would find information of this breadth somewhat difficult and expensive to collect. Even if the resources exist to gather the data, it would likely duplicate data in existing crime databases, weather websites, and census data. A data-integration solution may address this problem by considering these external resources as materialized views over a virtual mediated schema, resulting in "virtual data integration". This means application-developers construct a virtual schema—the mediated schema—to best model the kinds of answers their users want. Next, they design "wrappers" or adapters for each data source, such as the crime database and weather website. These adapters simply transform the local query results (those returned by the respective websites or databases) into an easily processed form for the data integration solution (see figure 2). When an application-user queries the mediated schema, the data-integration solution transforms this query into appropriate queries over the respective data sources. Finally, the virtual database combines the results of these queries into the answer to the user's query. This solution offers the convenience of adding new sources by simply constructing an adapter or an application software blade for them. It contrasts with ETL systems or with a si

  Read more →
• 

  Commit (data management)

  In computer science and data management, a commit is a behavior that marks the end of a transaction and provides Atomicity, Consistency, Isolation, and Durability (ACID) in transactions. The submission records are stored in the submission log for recovery and consistency in case of failure. In terms of transactions, the opposite of committing is giving up tentative changes to the transaction, which is rolled back. Due to the rise of distributed computing and the need to ensure data consistency across multiple systems, commit protocols have been evolving since their emergence in the 1970s. The main developments include the Two-Phase Commit (2PC) first proposed by Jim Gray, which is the fundamental core of distributed transaction management. Subsequently, the Three-phase Commit (3PC), Hypothesis Commit (PC), Hypothesis Abort (PA), and Optimistic Commit protocols gradually emerged, solving the problems of blocking and fault recovery. Today, new fields such as e-commerce payment and blockchain technology are emerging, and submission protocols play a significant role in various business areas. By effectively handling transactions, resolving faults and recovering problems, the commit protocol becomes crucial in ensuring the reliability and consistency of data management. == History == The concept of Commit originated in the late 1960s and early 1970s, when computer technology was rapidly advancing and data management was becoming an important requirement in business and finance. Enterprises have gradually replaced the traditional paper records with computers, which has fully improved the work efficiency. The reliability and consistency of data have become a necessary requirement. Transaction management at this stage is relatively simple, limited to using a single computer for processing. It merely effectively records the changes in data to ensure that the data remains stable after the transaction is completed or terminated. In the late 1970s, as database systems moved from a single calculator operation to multiple distributed collaborations, ensuring data consistency and reliability became a new challenge. In 1978, computer scientist Jim Gray proposed the famous two-phase Commit Protocol (2PC), which became an effective solution for distributed transaction management, successfully managing data synchronization problems between multiple nodes. However, this commit protocol has some potential transaction blocking problems when nodes fail. In the early 1980s, researchers discovered that although the two-step commit protocol was effective at synchronizing data, there could be long waits and even system crashes, with limitations. To improve this problem, people have begun to explore new and effective methods, including enhancing efficiency by reducing message communication during the protocol process. IBM's R database introduced the Assumed Commit and Assumed abort protocols, which contributed significantly to transaction management efficiency. These two protocols have greatly improved the processing efficiency of distributed transactions by reducing communication overhead and have become an important breakthrough in the technology of transaction commit protocols. By the early 1990s, with the increase in business demands and the complexity of transactions, enterprises required higher efficiency in distributed transaction processing. In order to adapt to the needs of different environments, the scientific community has gradually developed various variants of commit protocols to provide more flexible transaction management options for different needs. For example, the three-phase commit protocol promotes the commit of transactions more effectively and reduces the occurrence of blocking problems by adding a pre-commit protocol and a timeout mechanism. In the 21st century, with the popularization of mobile Internet and wireless technology, the commit protocol has been further developed, and researchers have begun to pay attention to how to reduce the blocking in the transaction process to solve the problem of broadband limitation, battery life and network instability in the mobile environment. The proposal of optimistic commit protocol marks the extension of commit technology from traditional database to the emerging mobile data field. This protocol allows transactions to temporarily use unconfirmed data, improving the user experience in cases of poor network conditions. In recent years, with the rise of blockchain and decentralized technologies, submission protocols and consensus mechanisms have gradually merged. These consensus algorithms play a role in tamper-proofing and preventing malicious attacks on node pairs in a decentralized environment. This enables commit to no longer be confined to the scope of traditional database management, but to become the core technology of trust computing and distributed ledgers, further expanding the application field of commit in the digital age. This integration has brought about extensive application impacts. Each transaction can achieve the effect of tracking global submissions through the verification of the consensus mechanism, becoming an important technical foundation for promoting the circulation of digital assets, the operation of cryptocurrencies and decentralized applications. == Commit Protocol Types == In the world of data management, a transaction is a series of database operations, such as bank transfers and order submission. In order to ensure the accuracy, consistency, and security of the data, transactions are usually completed completely, or cancelled completely, leaving no partially completed results. Commit protocol is the method used to coordinate this process. Different protocols are applicable to different submission scenarios and have their own advantages and disadvantages. There are four major commit protocols. === Two-Phase Commit (2PC) === The two-phase commit protocol is the most classic and broadest approach to distributed transactions, which includes both a preparation phase and a commit phase. This commit protocol is designed to allow the database coordinator to determine if all participating nodes agree. The preparation phase is the phase in which the coordination node sends a ready to commit request to all nodes participating in the transaction. The commit phase is a global commit after all participating nodes are ready, and if no agreement is reached, all nodes roll back the transaction and undo all previous operations. Although the two-phase commit protocol is the easiest to operate and widely used, its obvious drawback is that it can cause transactions to be blocked for a long time when nodes fail, resulting in a decline in system performance and making it difficult to terminate or continue immediately. === Three-Phase Commit (3PC) === The three-phase commit protocol is an improved non-blocking protocol based on 2PC, which is divided into three stages: preparation, pre-commit and commit. Firstly, each node sends a "preparation" request. After confirmation, a "pre-submission" stage is added. At this point, each node has completed most of the preparatory work and is waiting for the final confirmation. Finally, in the formal commit stage, after all nodes send the "commit" request, the transaction is completed and committed. Compared with 2PC, it increases the timeout mechanism, avoids the blocking problem caused by single point of failure, and improves the reliability of the system. The three-phase commit protocol significantly optimizes transaction reliability, but adds additional overhead for message transmission and state maintenance. It is more suitable for distributed application scenarios with high transaction sensitivity and no acceptance of long waiting times. === Presumed Commit (PC) and Presumed Abort (PA) === Presumed Commit (PC) is the default that the transaction will be committed successfully and rollback will be notified unless an anomaly is encountered. This commit reduces the message overhead and logging costs of a normal commits. Presumed Abort (PA) is assumed that the default state of the transaction is a rollback and will only be committed when all nodes have explicitly agreed. This commit is applicable to transactions that are not updated frequently or have a low probability of successful commit. The IBM R Distributed Database management System was the first to propose and practice the PC and PA protocols, handling distributed transaction management very efficiently and becoming a classic case in the field of database transaction management. === Optimistic Commit Protocol === With the rise of the Internet, the previous commit protocols are facing new challenges, especially in mobile scenarios with unstable networks. Excessively long transaction waiting times can affect the user experience. The Optimistic Commit Protocol allows a transaction to temporarily access uncommitted data before committing to avoid wait times. This type of commit is suitable f

  Read more →