AI Art Or Not — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.
SYMAN is an artificial intelligence technology that uses data from social media profiles to identify trends in the job market. SYMAN is designed to organize actionable data for products and services including recruiting, human capital management, CRM, and marketing. SYMAN was developed with a $21 million series B financing round secured by Identified, which was led by VantagePoint Capital Partners and Capricorn Investment Group.
Read more →
IEBus (Inter Equipment Bus) is a communication bus specification "between equipments within a vehicle or a chassis" of Renesas Electronics. It defines OSI model layer 1 and layer 2 specification. IEBus is mainly used for car audio and car navigations, which established de facto standard in Japan, though SAE J1850 is major in United States. IEBus is also used in some vending machines, which major customer is Fuji Electric. Each button on the vending machine has an IEBus ID, i.e. has a controller. Detailed specification is disclosed to licensees only, but protocol analyzers are provided from some test equipment vendors. Its modulation method is PWM (Pulse-Width Modulation) with 6.00 MHz base clock originally, but most of automotive customers use 6.291 MHz, and physical layer is a pair of differential signalling harness. Its physical layer adopts half-duplex, asynchronous, and multi-master communication with carrier-sense multiple access with collision detection (CSMA/CD) for medium access control. It allows for up to fifty units on one bus over a maximum length of 150 meters. Two differential signalling lines are used with Bus+ / Bus− naming, sometimes labeled as Data(+) / Data(−). It is sometimes described as "IE-BUS", "IE-Bus," or "IE Bus," but these are incorrect. In formal, it is "IEBus." IEBus® and Inter Equipment Bus® are registered trademark symbols of Renesas Electronics Corporation, formerly NEC Electronics Corporation, (JPO: Reg. No.2552418 and 2552419, respectively). == History == In the middle of '80s, semiconductor unit of NEC Corporation, currently Renesas Electronics, started the study for increasing demands for automotive audio systems. IEBus is introduced as a solution for the distributed control system. In the late 1980s, several similar specifications, including the Domestic Digital Bus (D2B), the Japanese Home Bus (HBS), and the European Home System (EHS) are proposed by different companies or organizations. These were once discussed as IEC 61030, but it was withdrawn in 2006. IEBus is also a similar specification (refer to "Transfer signal format" section), but not listed in these criteria. As the result, IEBus becomes a de facto standard of car audio in Japan. Regarding the Domestic Digital Bus (D2B), it is re-defined as D2B Optical by Mercedes-Benz independently. As for Japanese Home Bus System (HBS), it is defined in 1988 as Home Bus System Standard Specification, ET-2101 by JEITA and REEA (Radio Engineering & Electronics Assiation) in Japan. It is being used by several Japanese air conditioner manufacturers (for example, M-Net from Mitsubishi and the P1/P2 or F1/F2 bus from Daikin). Fujitsu provided HBPC (Home Bus Protocol Controller) chip as MB86046B. But it is unclear whether Fujitsu (currently, Cypress) still manufactures this HBPC LSI as of 2018. Mitsumi Electric provides the MM1007 and MM1192 driver ICs for HBS. The HBS specification is also discussed in the Echonet Consortium. In 2014, a utility model patent for protocol converter from HBS to RS-485 is granted in China as "CN204006496U." Regarding the replacement of IEBus, a paper by Hyundai Autonet, currently Hyundai Mobis, describes as follows. "In communication methods for digital input capable amplifiers, Inter Equipment Bus (IEBus) was used in early times, but for now, Controller Area Network (CAN) is mainly used." == Protocol overview == A master talks to a slave. Each unit has a master and a slave address register. Only one device can talk on the bus at any given time. There is a pecking order for the types of communications which will take precedence over another. Each communication from master to slave must be replied to by the slave going back to the master with acknowledge bits each of those show ACK or NAK. If the master does not receive the ACK within a predefined time allowance for a mode, it drops the communication and returns to its standby (listen) mode. Detailed specification of OSI model layer 2 is disclosed to licensees only, but protocol analyzers are provided from some test equipment vendors. In 2012, one of Chinese manufacturer's patent is granted as "CN202841169U". An open-source software emulator called "IEBus Studio" exists on a repository of SourceForge, but the last update was on 2008-02-24. Another open-source analyzer software called "IEBusAnalyzer" is available on GitHub repository. Some hobbyist made some tools also. === Physical layer (OSI model layer 1) specification overview === From μPD6708 data sheet. and μPD78098B Subseries user's manual, hardware. Communication system Half-duplex asynchronous communication Multi-master system All the units connected to the IEBus can transfer data to the other units. Broadcast communication function (communication between one unit and multiple units) Normally, communication is individually carried out from one unit to another. By using the broadcast communication function, however, communication can be executed from one unit to plural units as follows: Group broadcast communication: Broadcast communication to group units Simultaneous broadcast communication: Broadcast communication to all units Effective transmission rate The effective transmission rate can be selected from the following three communication modes: Mixture of the plural of modes in the same bus line is not allowed. Correct communication between different base clock is not possible. Access control CSMA/CD (Carrier Sense Multiple Access with Collision Detection) The priority of occupying IEBus is as follows: «1» Broadcast communication takes precedence over individual communication. «2» The lower the master address, the higher the priority. Communication scale Number of units: 50 MAX. Cable length: 150 m MAX. (when a twisted pair cable is used) Load capacity: MAX. 8000 pF; between Bus+ and Bus−, (6.000000 MHz base clock) MAX. 7100 pF; between Bus+ and Bus−, (6.291456 MHz base clock) Terminating resistor: 120 Ω Logic level Logic 1: Low level. Voltage difference between Bus+ and Bus− is under 20mV Logic 0: High Level. Voltage difference between Bus+ and Bus− is over 120mV In-phase input voltage high: Bus+ ≤ (VDD-1.0) V, Bus− ≥ 1.0 V === Transfer signal format === From μPD6708 data sheet. and μPD78098B Subseries user's manual, hardware. This frame format is much similar to that of Domestic Digital Bus (D2B). All fields are MSB first. ==== Functions of Control bits ==== === Bit format === Each IEBus bit consists of four periods. Preparation period: The first or subsequent low-level (logic "1") period Synchronization period: Next high-level (logic "0") period Data period: Period indicating value of bit; ether low-level (logic "1") or high-level (logic "0") Stop period: The last low-level (logic "1") period Synchronization is done by each bit. Time lengths of the synchronization period and data period are almost the same. The time of the entire bits' and each bit's specification, related to the time of each period allocated to it, differ depending both on the type of the transmit bit and on whether the unit is the master or a slave unit. == Automotive manufacturers using IEBus == Each manufacturer has its own name, but it is not an alias of IEBus. Those are specifications of wire harness which comprise control cables based on IEBus, OSI model layer 3 and above communication protocol, audio cables, interconnection couplers, and so on. === Pioneer === Pioneer Corporation employed IEBus for its original branded car audio in early '90s. In its earlier stage, it was used just for control bus between the head unit in dashboard and the CD changer usually placed in trunk room. Nowadays, the specification includes connection between head units, navigation systems, rear speaker systems, and so on. IP-Bus: Wire harness specification. === Toyota === Pioneer Corporation pushed Toyota Motor Corporation to adopt IEBus as the genuine parts. In 1994, Toyota decided to employ IEBus for its genuine specification, but it is slightly different from that of Pioneer. It is named as AVC-LAN. AVC-LAN: Wire harness specification, based on mode 2. === Honda/Acura === Pioneer Corporation also pushed Honda Motor. Honda also decided to adopt IEBus as its genuine parts specification just after Toyota do so. GA-NET II: Wire harness specification. Honda Music Link: Honda genuine gadget to connect Apple Inc. products. A hobbyist made touch screen controller on Acura TSX for a Car PC installed in the trunk. === Sirius XM Satellite Radio === Sirius XM Satellite Radio is a satellite broadcasting radio operator in US. Its digital media receiver equipment utilizes IEBus. == Evaluation boards == === SAKURA board === GR-SAKUKRA board and GR-SAKURA-FULL board are Renesas official promotion boards of RX63N chip, which enables IEBus mode 0 and 1, but not mode 2, i.e. not available for Toyota AVC-LAN. They are an Arduino pin compatible low-price ones, suitable for hobbyists. Their color of printed circuit board is SAKURA in Japanese, which means cherry blossom. To e
Read more →
"Pivot to video" is a phrase referring to the trend, starting in 2015, of media publishing companies cutting staff resources for written content (generally published on their own web sites) in favor of short-form video content (often published on third-party platforms such as Facebook, Instagram, Twitter, YouTube, Snapchat, and TikTok). These moves were generally presented by publishers as a response to changes in social media traffic or to changes in the media consumption habits of younger audiences. However, many media commentators have argued that this shift was primarily motivated by advertising revenue, and that only advertisers, not consumers, prefer video over text. The pivot's contribution to job loss in the media industry has given the phrase "pivot to video" an association with decline, especially in a business context. Commentators have also noted a lack of transparency and accuracy in the viewership metrics reported by platforms such as Facebook, pointing out that abrupt shifts in platforms' proprietary algorithms can have devastating effects on publishers' viewership, traffic, and revenue. Following a scandal in which Facebook revealed it had artificially inflated numbers to its advertisers about how long viewers watched ads, many journalists and industry analysts concluded that the shift to video was based on such misleading or inaccurate metrics, which created a false impression that there was customer demand for additional video content. == History == Streaming media technology has been available since the early 1990s, though it was relatively low-fidelity and not widely available until the mid-2000s. In 2007, traditional media publishers including the New York Times, Washington Post and Time Inc. created new divisions to develop web videos, and Facebook launched its video platform. Twitter purchased micro-video service Vine in October 2012, began adding native video streaming in late 2014, and acquired video-streaming service Periscope in January 2015. An August 2014 profile on BuzzFeed noted the publisher's large investment into video production, and observed that "the future of BuzzFeed may not even be on BuzzFeed.com. One of the company’s nascent ideas, BuzzFeed Distributed, will be a team of 20 people producing content that lives entirely on other popular platforms, like Tumblr, Instagram or Snapchat." On 7 January 2015, Facebook issued a statement about "the shift to video," reporting that "since June 2014, Facebook has averaged more than 1 billion video views every day." Media critic John Herrman argued that "What the shift to Facebook video means is that Facebook is more interested in hosting the things media companies make than just spreading them, that it views links to outside pages as a problem to be solved, and that it sees Facebook-hosted video as an example of the solution." In February 2015, the digital video-journalism publisher NowThis announced that it would operate without a home page, producing content to be published directly on social media platforms. In April 2016, Mashable fired much of its editorial staff, attempting to pivot away from hard news coverage while "growing Mashable across every platform" and doubling down on branded content and video. By December 2017, following a sale to Ziff Davis, Mashable retreated from this focus on video; Bernard Gershon, president of GershonMedia, said that the announcement of many such "pivots" were actually aimed primarily at investors. By 2017, "advertiser interest in video [was] insatiable... Any CFO is going to say 'How can we get more video?'" according to an executive of the publishers' trade association Digital Content Next. Publishers such as Vanity Fair, the Washington Post, and Sports Illustrated began adapting their own articles into cheap video content, either dictated by a newsreader or animated as a slideshow with captions, which could be shared on social platforms or even played alongside the articles themselves. June 2017 saw numerous high-profile pivots to video. Vocativ laid off at least 20 staff, including its entire newsroom, explaining that "as the industry evolves, we are undertaking a strategic shift to focus exclusively on video content that will be distributed via social media and other platforms." Fox Sports eliminated its entire writing staff to focus on creating "premium video across all platforms." And MTV News announced a restructuring that would cut its writing team. Less than two years earlier, MTV News had hired Grantland co-founder Dan Fierman to lead a significant investment in "longform" political and cultural reporting, but Fierman left in April 2017, and in June MTV announced it was "shifting resources into short-form video content more in line with young people's media consumption habits." In July, Vice Media laid off at least 60 employees, including the editor-in-chief of Vice Sports, while expanding video production. August 2017 saw Mic cut ten writers and directed the remainder of the newsroom to generate videos for social platforms. CEO Chris Altchek said "When you think about how many hours people spend watching video versus reading, the audience has already spoken." The move was ultimately unsuccessful, and Mic laid off the majority of its staff a year later before being sold to Bustle Media Group for a fraction of its former value. In September 2017, the for-profit wiki-hosting company Fandom began adding commercially produced videos to its otherwise user-generated wiki subdomains, explicitly citing the need to "keep up with user and advertiser expectations" by "diversifying our content," claiming without substantiation that "consumer patterns are changing," necessitating the addition of "complementary video" to accommodate that supposed need. Objection to the content in these videos and its sharp contrast against the content of the wiki sites to which they were applied led to vocal user backlash, leading Fandom CCO Dorth Raphaely to offer the following non-committal response: "I agree that with these videos in particular we did not deliver the right type of content experience." Movie Pilot CEO Tobi Bauckhage explained his company's fall 2017 layoffs as part of moving "from a text-based publishing model to video... a reaction to the fact that Facebook has changed their algorithms in favor of video instead of referral traffic over the last 12 months and we were losing money in the publishing bit of our business." As part of the company's change in direction, the majority of its staff was laid off and its parent company was sold to Webedia. In November 2017, magazine publisher Condé Nast cut jobs, reduced the frequency of several magazines, and shut down the print edition of Teen Vogue, then invested significant new resources in video production, with a senior executive saying "In the next 24 months, I hope that video is half our business... It’s critical. It’s the macro trend of content consumption." In February 2018, Vox Media cut approximately 50 employees, primarily those assigned to "social video," as Vox CEO Jim Bankoff admitted that those efforts were not "viable audience or revenue growth drivers." In August 2020, Facebook Inc. (now Meta Platforms) pivoted Instagram to video in an effort to replicate the success of TikTok and appeal to a younger audience, introducing "reels" as a form of video and promoting them aggressively. Reels accounted more than half the 20 most-viewed posts on Facebook; however, most of these reels were anonymous aggregations of content from TikTok. Elon Musk declared in early 2024 that X (formerly Twitter) was now a "video-first platform", which has been described by critics as a "pivot to video". == As euphemism == In 2017, Journalist Brian Feldman said that "'Pivoting to video' has become a business strategy for digital publishers common enough in recent months to be a kind of cliché — a slick way to describe something else: layoffs." In response, writers use the phrase as gallows humor shorthand for death or cancellation, as in "how do i tell my bf i want our relationship to pivot to video" (SkyNews' Mollie Goodfellow) or "Horse broke its leg, so we had to take it out back and help it 'pivot to video'" (blogger Anil Dash). == Facebook metrics controversy == In September 2016, Facebook admitted that it had reported artificially inflated numbers to its advertisers about how long viewers watched ads leading to an overestimation of 60-80%. Plaintiffs in a later court case allege the discrepancy was as high as 150-900%. Facebook apologized in an official statement and in multiple staff appearances at New York Advertising Week. Two months later, Facebook disclosed additional discrepancies in audience metrics. In October 2018, a California federal court unsealed the text of a class action lawsuit filed by advertisers against Facebook, alleging that Facebook had known since 2015 that its viewership numbers were highly inflated, that internal records showed it "was far from an hon
Read more →
The ACTS Gigabit Satellite Network was a pioneering, high-speed communications satellite network in the years 1993-2004, created as a prototype system to explore high-speed networking of digital endpoints. The system was jointly sponsored by NASA and ARPA, implemented by BBN Technologies and Motorola, and was inducted into the Space Technology Hall of Fame in April 1997. The Advanced Communications Technology Satellite (ACTS) network was designed to provide fiber-compatible SONET service to remote nodes and networks through a wideband satellite system, and provided long-haul, point-to-point and point-to-multipoint full-duplex SONET services, at rates up to 622 Mbit/s, over NASA's Advanced Communication Technology Satellite (ACTS). The Advanced Communications Technology Satellite itself, built and operated by Lockheed Martin, was launched on STS-51 on September 12, 1993, by the Space Shuttle Discovery, and occupied a geostationary orbit at 100° west longitude. It was the first communication satellite to operate in the 20–30 GHz frequency band (Ka band), with 30 GHz uplink and 20 GHz downlink signals. The satellite incorporated advanced on-board switching and multiple dynamically-hopping spot-beam antennas for selected areas of the United States including Hawaii. Up to 3 uplink and 3 downlink antenna beams could be active simultaneously. The ACTS network ground terminals were transportable Gigabit Earth Stations (GES) with fiber-optic SONET interfaces (OC-3 and OC-12), which also supported the Asynchronous Transfer Mode (ATM) protocol suite. The network control and management functions are distributed in the various Gigabit Earth Stations, with the operator's interface being centralized in a Network Management Terminal (NMT), which could be collocated at a GES, or anywhere in the Internet. The system was operational and used for experiments for 127 months, instead of the originally planned 24–48 months. In all, 53 terminals were built and used by more than 100 experimenters to test ACTS abilities. In Nov. 1997 a record data rate of 520 Mbit/s TCP/IP throughput was achieved using ATM between several ground stations via ACTS. On May 31, 2000 the ACTS experiments program officially came to a close, but the system continued to support experiments until it was deactivated on April 28, 2004.
Read more →
Immuni was an open-source COVID-19 contact tracing app used for digital contact tracing in Italy, dismissed on 31 December 2022, after a long and debated criticism for having been a failure due to the lack of trust placed by citizens. Immuni COVID-19 contact-tracing app had in fact been downloaded only by 12% of Italians between 14 and 75 years old (the government had previously stated that, in order for the app to work properly, it should have been downloaded by at least 60% of Italians). It makes use of the Apple/Google Exposure Notification system. == Development == It was developed by Bending Spoons and released by the Italian Ministry of Health on 1 June 2020. After a testing phase in 4 Italian regions (Abruzzo, Apulia, Liguria, Marche), the app started being active in the whole country on 15 June 2020. The app was initially released on App Store and Google Play, and since 1 February 2021 it is available on the Huawei AppGallery as well. === Source code === The source code was published on GitHub on the 25 May. The app only works in Italy, but compatibility with other European contact tracing apps was a goal. Since 19 October 2020 the app supports key-exchanges with the EU Interoperability Gateway and is therefore able to communicate with contact tracing apps of other EU countries. == Shutdown == As of 16 December 2020, the app was downloaded more than 10 million times, a number which increased to 21.882.502 downloads the day before the app's shutdown. On 27 December 2022 the Italian Ministry of Health announced that the app and its infrastructures will be dismissed on the 31 December of the same year.
Read more →
Embedded analytics enables organisations to integrate analytics capabilities into their own, often software as a service, applications, portals, or websites. This differs from embedded software and web analytics (also commonly known as product analytics). This integration typically provides contextual insights, quickly, easily and conveniently accessible since these insights should be present on the web page right next to the other, operational, parts of the host application. Insights are provided through interactive data visualisations, such as charts, diagrams, filters, gauges, maps and tables often in combination as dashboards embedded within the system. This setup enables easier, in-depth data analysis without the need to switch and log in between multiple applications. Embedded analytics is also known as customer facing analytics. Embedded analytics is the integration of analytic capabilities into a host, typically browser-based, business-to-business, software as a service, application. These analytic capabilities would typically be relevant and contextual to the use-case of the host application. == History == The term "embedded analytics" was first used by Howard Dresner: consultant, author, former Gartner analyst and inventor of the term "business intelligence" said Howard Dresner while he was working for Hyperion Solutions, a company that Oracle bought in 2007. Oracle started then to use the term "embedded analytics" at their press release for Oracle Rapid Planning on 2009 . == Considerations with embedded analytics == When evaluating embedding analytics, consideration would normally be given to integration at various levels, these would likely include: security integration, data integration, application logic integration, business rules integration, and user experience integration. This is in contrast to traditional BI, which expects users to leave their workflow applications to look at data insights in a separate set of tools. This immediacy makes embedded analytics much more intuitive and likely to be valued by users. A December 2016 report from Nucleus Research found that using BI tools, which require toggling between applications, can take up as much as 1–2 hours of an employee's time each week, whereas embedded analytics eliminate the need to toggle between apps.
Read more →
In cryptography, a pepper is a secret added to an input such as a password during hashing with a cryptographic hash function. This value differs from a salt in that it is not stored alongside a password hash, but rather the pepper is kept separate using another meachanism, such as a Hardware Security Module. Note that the National Institute of Standards and Technology refers to this value as a secret key rather than a pepper. A pepper is similar in concept to a salt or an encryption key. It is like a salt in that it is a randomized value that is added to a password hash, and it is similar to an encryption key in that it should be kept secret. A pepper performs a comparable role to a salt or an encryption key, but while a salt is not secret (merely unique) and can be stored alongside the hashed output, a pepper is secret and must not be stored with the output. The hash and salt are usually stored in a database, but, if stored, a pepper must be stored separately to prevent it from being obtained by the attacker in case of a database breach. == History == The idea of a site- or service-specific salt (in addition to a per-user salt) has a long history, with Steven M. Bellovin proposing a local parameter in a Bugtraq post in 1995. In 1996 Udi Manber also described the advantages of such a scheme, terming it a secret salt. However, he suggested not storing the value of the secret salt, but instead rediscovering it by trial and error at password verification time. The term pepper has been used, by analogy to salt, but with a variety of meanings. For example, when discussing a challenge-response scheme, pepper has been used for a salt-like quantity, though not used for password storage; it has been used for a data transmission technique where a pepper must be guessed; and even as a part of jokes. The term pepper was proposed for a secret or local parameter stored separately from the password in a discussion of protecting passwords from rainbow table attacks. This usage did not immediately catch on: for example, Fred Wenzel added support to Django password hashing for storage based on a combination of bcrypt and HMAC with separately stored nonces, without using the term. Usage has since become more common. == Types == There are multiple different types of pepper: A shared secret that is common to all users. A randomly-selected number that must be re-discovered on every password input. These mechanisms could be combined with password salting, iterated hashing or even one another. == Shared-secret pepper == Bellovin and Webster suggest prepend a shared secret to the password before hashing, which allows easy use of existing hash functions. For example, consider two users to be added to a database. This table contains two combinations of username and password. The password is not saved, and the 8-byte (64-bit) 44534C70C6883DE2 pepper is saved in a safe place separate from the output values of the hash, in this case SHA256. Unlike the salt, the pepper does not provide protection to users who use the same password, but protects against dictionary attacks, unless the attacker has the pepper value available. Since the same pepper is not shared between different applications, an attacker is unable to reuse the hashes of one compromised database to another. A complete scheme for saving passwords may include both salt and pepper use. For example, it has been suggested to combine the pepper by encrypting salted password hashes, which allows rotation of the pepper. In the case of a shared-secret pepper, a single compromised password (via password reuse or other attack) along with a user's salt can lead to an attack to discover the pepper, rendering it ineffective. If an attacker knows a plaintext password and a user's salt, as well as the algorithm used to hash the password, then discovering the pepper can be a matter of brute forcing the values of the pepper. This is why NIST recommends the secret value be at least 112 bits, so that discovering it by exhaustive search is prohibitively expensive. The pepper must be generated anew for every application it is deployed in, otherwise a breach of one application would result in lowered security of another application. Without knowledge of the pepper, other passwords in the database will be far more difficult to extract from their hashed values, as the attacker would need to guess the password as well as the pepper. A pepper adds security to a database of salts and hashes because unless the attacker is able to obtain the pepper, cracking even a single hash is intractable, no matter how weak the original password. Even with a list of (salt, hash) pairs, an attacker must also guess the secret pepper in order to find the password which produces the hash. The NIST specification for a secret salt suggests using a Password-Based Key Derivation Function (PBKDF) with an approved Pseudorandom Function such as HMAC with SHA-3 as the hash function of the HMAC. The NIST recommendation is also to perform at least 1000 iterations of the PBKDF, and a further minimum 1000 iterations using the secret salt in place of the non-secret salt. == Randomly-selected pepper that must be re-discovered == The aim of this mechanism is to slow down password the password verification step, thus slowing attacks. The aim is similar increasing the iteration count on bcrypt or Argon2, but the mechanism is different. The secret salt or pepper must be rediscovered by the verifier or attacker each time by guessing. In this situation, the password hashing function is calculated using both the password and the pepper. At password storage time, the pepper is chosen randomly from a range between 1 and R, the hash output is calculated using the password and the pepper. The hash output is stored with the username. The pepper is then discarded. At password verification time, the verifier is provided with a username and password to verify. The originally calculated hash is retrieved for the given username, and then the hash of the password and each value between 1 and R is calculated. If any of these hash values match the stored password hash, the password is considered valid. Note, the possible values of the pepper should not be tested in a fixed order known to an attacker, otherwise a timing attack may reveal the pepper. If the password is correct, the correct pepper will be found in R/2 hash evaluations on average. If the password is incorrect, all R values must be tested before the password can be rejected.
Read more →
Data marketplace is an online platform for sharing and consuming data in the form of data assets or data products. Part of the data management stack, it aims to bring together data producers and data consumers (including business users and AI) in a single space, with the objective of increasing access to understandable, high-quality data. Included within its Data Marketplaces and Exchange (DME) category by Gartner, data marketplaces can provide data internally within an organization, externally with partners, or as open data. == Concept == Digitization has dramatically increased data volumes within organizations, with IDC predicting that by 2025 the world will contain 175 zettabytes of data. This has created a need to both manage this data and provide access to it to enable business intelligence and data analysis. However, data is often scattered within multiple systems (such as data warehouses and data lakes), and is in formats that are only understandable by technical experts, such as data scientists. According to IDC, 81% of IT leaders cite data silos as a major barrier to digital transformation. This means that data is not freely available to business users or external audiences such as partners or citizens, limiting its value, and holding back AI deployments. Data marketplaces solve this issue, providing seamless, self-service access to high-quality data in an understandable, secure and auditable manner. They break down data silos, reduce friction in data access, and enable a broader range of users, including non-technical profiles, to find, understand, and consume data autonomously. Data assets on the marketplace can be raw data, data visualizations or data products. Data marketplaces combine data management functions such as data governance with the user-friendly experience offered by e-commerce marketplaces in order to increase the usage of data. These include features such as powerful search engines, feedback, ratings, subscriptions and product description sheets. According to Gartner, data marketplaces provide infrastructure, transactional capabilities, and services for both consumers and providers of data assets. == History and timeline == Data marketplaces have evolved since they first emerged in terms of both their scope and usage. === 2000s === With the rise of the internet, data brokers began collecting, aggregating, distributing and selling personal, financial and marketing data to third parties online. Data marketplaces were deployed to monetize this data, making it discoverable and accessible to users, either through subscriptions or one-off purchases. At the same time, regulations, such as the US Open Government Initiative of 2009 and others around the world mandated greater transparency and data sharing with the public. Data sharing portals were created by public and government bodies to make this information available through self-service to all users. === 2010s === Due to the growth of big data and cloud platforms, cloud-based data exchange platforms emerged. These were offered by major infrastructure providers, and included Amazon Web Services (AWS) Data Exchange, Snowflake Data Marketplace, and the Google Cloud Platform. These platforms moved beyond simple data brokerage or open data by providing structured, catalogued data sharing between organizations. === 2020s === Driven by a need to increase internal data sharing with both business users and AI, organizations are now looking to adopt internal data marketplaces. These aim to democratize data consumption by providing seamless access for all employees and AI to trusted data, including data products, through an intuitive, e-commerce style experience. According to Gartner analyst Richa Jha, "by providing a single, governed platform for discovering, sharing, and scaling data products, data marketplaces drive productivity, collaboration, and ROI across the enterprise." == Data marketplaces within the overall data architecture == Data marketplaces provide a consumption and collaboration layer for data. That means they complement and integrate with other parts of the overall data architecture, including: === Data warehouses and data lakes === Data marketplaces connect to data sources, such as data warehouses or data lakes, to provide intuitive access to the data stored within them, enabling data to be shared and distributed to non-technical audiences. Access can be direct, with data and data products stored within the data marketplace or virtualized. === Data catalog === A data catalog provides a technical inventory of an organization's data estate. It collects technical information on all available data assets within an organization, based on metadata descriptions. This ensures traceability, and supports compliance and governance requirements. Unlike a data marketplace, a data catalog does not provide access to data, and is designed to be used by data professionals, rather than the business. This means it lacks an intuitive, understandable interface and is consequently not easily accessible by business users. === Data mesh === Data mesh is an architecture and framework for data management, first defined by Zhamak Dehghani in 2019. It aims to decentralize data ownership to delegate responsibility, empowering teams and focusing on delivering data to users in the form of self-service data products. The data marketplace is a central pillar of data mesh, providing intuitive access to these data products, and creating a collaboration space for data owners and data consumers. === Data product === Data products are high-value, consumable data assets that package high-quality data and associated tools to enable seamless usage by business users at scale. First defined by McKinsey in 2022, they have an identified owner, a service level agreement (SLA), and a reusability logic. == Core components of a data marketplace == A data marketplace typically includes specific core components: === E-commerce style interface === An e-commerce style experience that engages non-technical users, minimizes the need for training and builds confidence and trust in data. Look and feel should be customizable to incorporate corporate design guidelines to ensure consistency with other organizational applications. === Built-in data catalog === As in a standalone data catalog, this indexes all available data, based on metadata that includes type, source, owner, freshness, and quality level. === Discovery and search engine === This enables users to search, filter, explore and discover available data intuitively. As in an e-commerce marketplace, it should be intelligent, and provide relevant results based on natural language queries. === Access control and security management === Data marketplaces will contain data that needs to be protected under regulations such as the General Data Protection Regulation (GDPR) in Europe, the California Consumer Privacy Act (CCPA) in the United States, and sector-specific frameworks in industries such as finance and healthcare. To ensure both security and compliance while maximizing data consumption, the data marketplace should include granular access management and a full audit trail. === Semantic layer and business glossary === Different parts of the business are likely to use different terms to describe data. This leads to inconsistencies and an inability to share data across systems and teams. The semantic layer and business glossary standardize a shared vocabulary and common definitions of business indicators and concepts, providing a single language for data across the business and for AI agents. === Data governance mechanisms === These enforce corporate data governance policies, ensuring data traceability through data lineage, quality certification, usage monitoring, and continuous improvement through user feedback loops. === Collaboration features === As on an e-commerce website, a data marketplace should provide collaboration features that bring together data users and data owners. This includes the ability to rate data products, share use cases, and provide feedback to data owners, creating a community around data and supporting a data-driven culture. == Types of data marketplace == While they share the same underlying technology, data marketplaces can be deployed in three broad ways: === Internal data marketplaces === These bring together data from across an organization and make it available via self-service to employees from across the business. They aim to widen access to data and consequently to improve decision-making and reporting, increase performance and maximize efficiency. === Ecosystem data marketplaces === These extend sharing beyond a single organization, enabling multiple partners (public institutions, industry players, research bodies) to share and consume data within a governed framework. Data can be provided by all parties or simply by one organization and consumed by others. Ecosystem data marketplaces are particularly relevant in
Read more →
Polynomial texture mapping (PTM), also known as Reflectance Transformation Imaging (RTI), is a technique of imaging and interactively displaying objects under varying lighting conditions to reveal surface phenomena. The data acquisition method is single camera multi light (SCML). == Origins == The method was originally developed by Tom Malzbender of HP Labs in order to generate enhanced 3D computer graphics and it has since been adopted for cultural heritage applications. == Methodology == A series of images is captured in a darkened environment with the camera in a fixed position and the object lit from different angles (Single Camera Multi Light). Interactive software processes and combines the set of images to enable the user inspecting the object to control a virtual light source. The virtual light source may be manipulated to simulate light from different angles and of different intensity or wavelengths to illuminate the surface of artefacts and reveal details. Open-source tools for processing the captured images and publishing the resulting relightable images on the web are freely available. == Applications == Polynomial texture mapping may be used for detailed recording and documentation, 3D modeling, edge detection, and to aid the study of inscriptions, rock art and other artefacts. It has been applied to hundreds of the Vindolanda tablets by the Centre for the Study of Ancient Documents at the University of Oxford in conjunction with the British Museum. It has also been deployed, by Ben Altshuler of the Institute for Digital Archaeology, to scan the Philae obelisk at Kingston Lacy and the Parian Chronicle at the Ashmolean Museum; in both cases scans revealed significant, previously illegible text. Method was also used for identifying microscopic worked antler from Star Carr and recording ancient rock art in Armenia. A 'dome' supporting twenty-four lights has been used to image paintings in the National Gallery and produce polynomial texture maps, providing information on condition phenomena for conservation purposes. Studies of the technique at the National Gallery and Tate concluded that it is an effective tool for documenting changes in the condition of paintings, more easily repeatable than raking light photography, and therefore could be used to assess paintings during structural treatment and before and after loan. Twelve dome-based systems built by the University of Southampton have been used to capture thousands of cuneiform tablets at various museums. The technique is now also finding uses in the field of forensic science, for example in imaging footprints, tyre marks, and indented writing.
Read more →
In cryptography, format-preserving encryption (FPE), refers to encrypting in such a way that the output (the ciphertext) is in the same format as the input (the plaintext). The meaning of "format" varies. Typically only finite sets of characters are used; numeric, alphabetic or alphanumeric. For example: Encrypting a 16-digit credit card number so that the ciphertext is another 16-digit number. Encrypting an English word so that the ciphertext is another English word. Encrypting an n-bit number so that the ciphertext is another n-bit number (this is the definition of an n-bit block cipher). For such finite domains, and for the purposes of the discussion below, the cipher is equivalent to a permutation of N integers {0, ... , N−1} where N is the size of the domain. == Motivation == === Restricted field lengths or formats === One motivation for using FPE comes from the problems associated with integrating encryption into existing applications, with well-defined data models. A typical example would be a credit card number, such as 1234567812345670 (16 bytes long, digits only). Adding encryption to such applications might be challenging if data models are to be changed, as it usually involves changing field length limits or data types. For example, output from a typical block cipher would turn credit card number into a hexadecimal (e.g.0x96a45cbcf9c2a9425cde9e274948cb67, 34 bytes, hexadecimal digits) or Base64 value (e.g. lqRcvPnCqUJc3p4nSUjLZw==, 24 bytes, alphanumeric and special characters), which will break any existing applications expecting the credit card number to be a 16-digit number. Apart from simple formatting problems, using AES-128-CBC, this credit card number might get encrypted to the hexadecimal value 0xde015724b081ea7003de4593d792fd8b695b39e095c98f3a220ff43522a2df02. In addition to the problems caused by creating invalid characters and increasing the size of the data, data encrypted using the CBC mode of an encryption algorithm also changes its value when it is decrypted and encrypted again. This happens because the random seed value that is used to initialize the encryption algorithm and is included as part of the encrypted value is different for each encryption operation. Because of this, it is impossible to use data that has been encrypted with the CBC mode as a unique key to identify a row in a database. FPE attempts to simplify the transition process by preserving the formatting and length of the original data, allowing a drop-in replacement of plaintext values with their ciphertexts in legacy applications. == Comparison to truly random permutations == Although a truly random permutation is the ideal FPE cipher, for large domains it is infeasible to pre-generate and remember a truly random permutation. So the problem of FPE is to generate a pseudorandom permutation from a secret key, in such a way that the computation time for a single value is small (ideally constant, but most importantly smaller than O(N)). == Comparison to block ciphers == An n-bit block cipher technically is a FPE on the set {0, ..., 2n-1}. If an FPE is needed on one of these standard sized sets (for example, n = 64 for DES and n = 128 for AES) a block cipher of the right size can be used. However, in typical usage, a block cipher is used in a mode of operation that allows it to encrypt arbitrarily long messages, and with an initialization vector as discussed above. In this mode, a block cipher is not an FPE. == Definition of security == In cryptographic literature (see most of the references below), the measure of a "good" FPE is whether an attacker can distinguish the FPE from a truly random permutation. Various types of attackers are postulated, depending on whether they have access to oracles or known ciphertext/plaintext pairs. == Algorithms == In most of the approaches listed here, a well-understood block cipher (such as AES) is used as a primitive to take the place of an ideal random function. This has the advantage that incorporation of a secret key into the algorithm is easy. Where AES is mentioned in the following discussion, any other good block cipher would work as well. === The FPE constructions of Black and Rogaway === Implementing FPE with security provably related to that of the underlying block cipher was first undertaken in a paper by cryptographers John Black and Phillip Rogaway, which described three ways to do this. They proved that each of these techniques is as secure as the block cipher that is used to construct it. This means that if the AES algorithm is used to create an FPE algorithm, then the resulting FPE algorithm is as secure as AES because an adversary capable of defeating the FPE algorithm can also defeat the AES algorithm. Therefore, if AES is secure, then the FPE algorithms constructed from it are also secure. In all of the following, E denotes the AES encryption operation that is used to construct an FPE algorithm and F denotes the FPE encryption operation. ==== FPE from a prefix cipher ==== One simple way to create an FPE algorithm on {0, ..., N-1} is to assign a pseudorandom weight to each integer, then sort by weight. The weights are defined by applying an existing block cipher to each integer. Black and Rogaway call this technique a "prefix cipher" and showed it was provably as good as the block cipher used. Thus, to create an FPE on the domain {0,1,2,3}, given a key K apply AES(K) to each integer, giving, for example, weight(0) = 0x56c644080098fc5570f2b329323dbf62 weight(1) = 0x08ee98c0d05e3dad3eb3d6236f23e7b7 weight(2) = 0x47d2e1bf72264fa01fb274465e56ba20 weight(3) = 0x077de40941c93774857961a8a772650d Sorting [0,1,2,3] by weight gives [3,1,2,0], so the cipher is F(0) = 3 F(1) = 1 F(2) = 2 F(3) = 0 This method is only useful for small values of N. For larger values, the size of the lookup table and the required number of encryptions to initialize the table gets too big to be practical. ==== FPE from cycle walking ==== If there is a set M of allowed values within the domain of a pseudorandom permutation P (for example P can be a block cipher like AES), an FPE algorithm can be created from the block cipher by repeatedly applying the block cipher until the result is one of the allowed values (within M). CycleWalkingFPE(x) { if P(x) is an element of M then return P(x) else return CycleWalkingFPE(P(x)) } The recursion is guaranteed to terminate. (Because P is one-to-one and the domain is finite, repeated application of P forms a cycle, so starting with a point in M the cycle will eventually terminate in M.) This has the advantage that the elements of M do not have to be mapped to a consecutive sequence {0,...,N-1} of integers. It has the disadvantage, when M is much smaller than P's domain, that too many iterations might be required for each operation. If P is a block cipher of a fixed size, such as AES, this is a severe restriction on the sizes of M for which this method is efficient. For example, an application may want to encrypt 100-bit values with AES in a way that creates another 100-bit value. With this technique, AES-128-ECB encryption can be applied until it reaches a value which has all of its 28 highest bits set to 0, which will take an average of 228 iterations to happen. ==== FPE from a Feistel network ==== It is also possible to make a FPE algorithm using a Feistel network. A Feistel network needs a source of pseudo-random values for the sub-keys for each round, and the output of the AES algorithm can be used as these pseudo-random values. When this is done, the resulting Feistel construction is good if enough rounds are used. One way to implement an FPE algorithm using AES and a Feistel network is to use as many bits of AES output as are needed to equal the length of the left or right halves of the Feistel network. If a 24-bit value is needed as a sub-key, for example, it is possible to use the lowest 24 bits of the output of AES for this value. This may not result in the output of the Feistel network preserving the format of the input, but it is possible to iterate the Feistel network in the same way that the cycle-walking technique does to ensure that format can be preserved. Because it is possible to adjust the size of the inputs to a Feistel network, it is possible to make it very likely that this iteration ends very quickly on average. In the case of credit card numbers, for example, there are 1015 possible 16-digit credit card numbers (accounting for the redundant check digit), and because the 1015 ≈ 249.8, using a 50-bit wide Feistel network along with cycle walking will create an FPE algorithm that encrypts fairly quickly on average. === The Thorp shuffle === A Thorp shuffle is like an idealized card-shuffle, or equivalently a maximally-unbalanced Feistel cipher where one side is a single bit. It is easier to prove security for unbalanced Feistel ciphers than for balanced ones. === VIL mode === For domain sizes that are a power of two, and an existing block cipher with a smaller bl
Read more →
Honey encryption is a type of data encryption that "produces a ciphertext, which, when decrypted with an incorrect key as guessed by the attacker, presents a plausible-looking yet incorrect plaintext." == Creators == Ari Juels and Thomas Ristenpart of the University of Wisconsin, the developers of the encryption system, presented a paper on honey encryption at the 2014 Eurocrypt cryptography conference. == Method of protection == A brute-force attack involves repeated decryption with random keys; this is equivalent to picking random plaintexts from the space of all possible plaintexts with a uniform distribution. This is effective because even though the attacker is equally likely to see any given plaintext, most plaintexts are extremely unlikely to be legitimate i.e. the distribution of legitimate plaintexts is non-uniform. Honey encryption defeats such attacks by first transforming the plaintext into a space such that the distribution of legitimate plaintexts is uniform. Thus an attacker guessing keys will see legitimate-looking plaintexts frequently and random-looking plaintexts infrequently. This makes it difficult to determine when the correct key has been guessed. In effect, honey encryption "[serves] up fake data in response to every incorrect guess of the password or encryption key." The security of honey encryption relies on the fact that the probability of an attacker judging a plaintext to be legitimate can be calculated (by the encrypting party) at the time of encryption. This makes honey encryption difficult to apply in certain applications e.g. where the space of plaintexts is very large or the distribution of plaintexts is unknown. It also means that honey encryption can be vulnerable to brute-force attacks if this probability is miscalculated. For example, it is vulnerable to known-plaintext attacks: if the attacker has a crib that a plaintext must match to be legitimate, they will be able to brute-force even Honey Encrypted data if the encryption did not take the crib into account. == Example == An encrypted credit card number is susceptible to brute-force attacks because not every string of digits is equally likely. The number of digits can range from 13 to 19, though 16 is the most common. Additionally, it must have a valid IIN and the last digit must match the checksum. An attacker can also take into account the popularity of various services: an IIN from MasterCard is probably more likely than an IIN from Diners Club Carte Blanche. Honey encryption can protect against these attacks by first mapping credit card numbers to a larger space where they match their likelihood of legitimacy. Numbers with invalid IINs and checksums are not mapped at all (i.e. have probability 0 of legitimacy). Numbers from large brands like MasterCard and Visa map to large regions of this space, while less popular brands map to smaller regions, etc. An attacker brute-forcing such an encryption scheme would only see legitimate-looking credit card numbers when they brute-force, and the numbers would appear with the frequency the attacker would expect from the real world. == Application == Juels and Ristenpart aim to use honey encryption to protect data stored on password manager services. Juels stated that "password managers are a tasty target for criminals," and worries that "if criminals get a hold of a large collection of encrypted password vaults they could probably unlock many of them without too much trouble." Hristo Bojinov, CEO and founder of Anfacto, noted that "Honey Encryption could help reduce their vulnerability. But he notes that not every type of data will be easy to protect this way. … Not all authentication or encryption system yield themselves to being honeyed."
Read more →
In cryptography, the branch number is a numerical value that characterizes the amount of diffusion introduced by a vectorial Boolean function F that maps an input vector a to output vector F ( a ) {\displaystyle F(a)} . For the (usual) case of a linear F the value of the differential branch number is produced by: applying nonzero values of a (i.e., values that have at least one non-zero component of the vector) to the input of F; calculating for each input value a the Hamming weight W {\displaystyle W} (number of nonzero components), and adding weights W ( a ) {\displaystyle W(a)} and W ( F ( a ) ) {\displaystyle W(F(a))} together; selecting the smallest combined weight across for all nonzero input values: B d ( F ) = min a ≠ 0 ( W ( a ) + W ( F ( a ) ) ) {\displaystyle B_{d}(F)={\underset {a\neq 0}{\min }}(W(a)+W(F(a)))} . If both a and F ( a ) {\displaystyle F(a)} have s components, the result is obviously limited on the high side by the value s + 1 {\displaystyle s+1} (this "perfect" result is achieved when any single nonzero component in a makes all components of F ( a ) {\displaystyle F(a)} to be non-zero). A high branch number suggests higher resistance to the differential cryptanalysis: the small variations of input will produce large changes on the output and in order to obtain small variations of the output, large changes of the input value will be required. The term was introduced by Daemen and Rijmen in early 2000s and quickly became a typical tool to assess the diffusion properties of the transformations. == Mathematics == The branch number concept is not limited to the linear transformations, Daemen and Rijmen provided two general metrics: differential branch number, where the minimum is obtained over inputs of F that are constructed by independently sweeping all the values of two nonzero and unequal vectors a, b ( ⊕ {\displaystyle \oplus } is a component-by-component exclusive-or): B d ( F ) = min a ≠ b ( W ( a ⊕ b ) + W ( F ( a ) ⊕ F ( b ) ) {\displaystyle B_{d}(F)={\underset {a\neq b}{\min }}(W(a\oplus b)+W(F(a)\oplus F(b))} ; for linear branch number, the independent candidates α {\displaystyle \alpha } and β {\displaystyle \beta } are independently swept; they should be nonzero and correlated with respect to F (the L A T ( α , β ) {\displaystyle LAT(\alpha ,\beta )} coefficient of the linear approximation table of F should be nonzero): B l ( F ) = min α ≠ 0 , β , L A T ( α , β ) ≠ 0 ( W ( α ) + W ( β ) ) {\displaystyle B_{l}(F)={\underset {\alpha \neq 0,\beta ,LAT(\alpha ,\beta )\neq 0}{\min }}(W(\alpha )+W(\beta ))} .
Read more →
A personal cloud is a collection of digital content and services that are accessible from any device through the Internet. It is not a tangible entity, but a place that gives users the ability to store, synchronize, stream and share content on a relative core, moving from one platform, screen and location to another. Created on connected services and applications, it reflects and sets consumer expectations for how next-generation computing services will work. The four primary types of personal cloud in use today are: Online cloud, NAS device cloud, server device cloud, and home-made clouds. == Online cloud == The online cloud is sometimes referred to as the public cloud. It is the cloud computing model where online resources like software and data storage are made available over the Internet. Typically, an individual or organization has little control over the ecosystem in which the online cloud is hosted, and the core infrastructure is shared between many individuals and organizations. The data and applications provided by the service provider are logically segregated so that only those authorized are allowed access. == NAS device cloud == A network-attached storage (NAS) device is a computer connected to a network that provides only file-based data storage services to other devices on the network. Although it may technically be possible to run other software on a NAS device, it is not designed to be a general purpose server. Cloud NAS is remote storage that is accessed over the Internet as if it were local. A cloud NAS is often used for backups and archiving. One of the benefits of NAS Cloud is that data in the cloud can be accessed at any time from anywhere. The main drawback, however, is that the speed of the transfer rate is only as fast as the network connection the data is accessed over and can therefore be fairly slow. == Server device cloud == In many ways cloud servers work in the same way as physical servers but the functions they perform can be very different. Typically, the cloud server is an on-premises device that is connected to the Internet and gives users the functions available on the online cloud but with the added benefit and security of the files being in their control on their premises. The server cloud has been historically enterprise-based deployed by businesses needing an in-house cloud. However, there are also in-house options available for individual users. == Home-made clouds == For the more technologically proficient user a common solution for using a personal cloud is to create a home-made cloud system by connecting an external USB hard drive to a Wi-Fi router. This enables both wired and wireless computers to access the USB hard drive and use it for storage or for retrieving files a user needs to share on the network thereby acting like a cloud. Setting up a personal cloud requires a user to have particular skills in technology and network setup. One of the risks associated with improper setup is security, and leaving the files accessible to anyone with technical knowledge. Not every router supports this type of access and modification.
Read more →
In computing, a data definition specification (DDS) is a guideline to ensure comprehensive and consistent data definition. It represents the attributes required to quantify data definition. A comprehensive data definition specification encompasses enterprise data, the hierarchy of data management, prescribed guidance enforcement and criteria to determine compliance. == Overview == A data definition specification may be developed for any organization or specialized field, improving the quality of its products through consistency and transparency. It eliminates redundancy (since all contributing areas are referencing the same specification) and provides standardization and degrees of compliance, making it easier and more efficient to create, modify, verify, analyze and share information across the enterprise. To understand how a data definition specification works in an enterprise, we must look at the elements of a DDS. Writing data definitions, defining business terms (or rules) in the context of a particular environment, provides structure for an organization's data architecture. In developing these definitions, the words used must be traceable to clearly defined data. A data definition specification may be used in the following activities: Business intelligence Business process modeling Business rules management Data analysis and modeling Information architecture Metadata modeling Data mastering Report generation == Criteria == A data definition specification requires data definitions to be: Atomic – singular, describing only one concept. Commonly used and ambiguous terms should be defined. While a term refers to one concept, several words may be used in a term: File – A concept identifiable with one word File extension – A concept identifiable with more than one word Traceable – Mapped to a specific data element. In business, a term may be traced to an entity (for example, a customer) or an attribute (such as a customer's name). A term may be a value in a data set (such as gender), or designate the data set itself. Traceability indicates relationships in the data hierarchy. Consistent - Used in a standard syntax; if used in a specific context, the context is noted Accurate - Precise, correct and unambiguous, stating what the term is and is not Clear - Readily understood by the reader Complete - With the term, its description and contextual references Concise - To avoid circular references == Applications == === Enterprise data === A data definition specification was produced by the Open Mobile Alliance to document charging data. The document, the centralized catalog of data elements defined for interfaces, specifies the mapping of these data elements to protocol fields in the interfaces. Created for the exchange of financial data, Market Data Definition Language (MDDL) is an XML specification designed to enable the interchange of information necessary to account, to analyze, and to trade financial instruments of the world's markets. It defines an XML-based interchange format and common data dictionary on the fields needed to describe: (1) financial instruments, (2) corporate events affecting value and tradability, and (3) market-related, economic and industrial indicators. The principal function of MDDL is to allow entities to exchange market data by standardizing formats and definitions. MDDL provides a common format for market data so that it can be efficiently passed from one processing system to another and provides a common understanding of market data content by standardizing terminology and by normalizing the relationships of various data elements to one another ... From the user perspective, the goal of MDDL is to enable users to integrate data from multiple sources by standardizing both the input feeds used for data warehousing (i.e., define what's being provided by vendors) and the output methods by which client applications request the data (i.e., ensure compatibility on how to get data in and out of applications)." === Clinical submissions === The Clinical Data Interchange Standards Consortium, a global, multidisciplinary, non-profit organization, has established standards to support the acquisition, exchange, submission and archiving of clinical research data and metadata. CDISC standards are vendor-neutral, platform-independent and freely available from the CDISC website. The Case Report Tabulation Data Definition Specification (define.xml) draft version 2.0, the oldest data definition specification, is part of the evolution from the 1999 FDA electronic submission (eSub) guidance and electronic Common Technical Document (eCTD) documents specifying that a document describing the content and structure of included data be included in a submission. Define.xml was developed to automate the review process by generating a machine-readable data-definition document. Define.xml has standardized submissions to the Food and Drug Administration, reducing review times from over two years to several months. === Archival data === A data definition specification is the foundation of metadata for scientific data archiving. The Metadata Encoding and Transmission Standard (METS) uses one principle of a DDS: consistent use of key terms to catalog digital objects for global use. The METS schema is a flexible mechanism for encoding descriptive, administrative and structural metadata for a digital library object and expressing complex links between metadata, and can provide a useful standard for the exchange of digital-library objects between repositories. A similar effort is underway to preserve complex data associated with video-game archiving. Preserving Virtual Worlds attempted to address archival-format deficiencies, citing the lack of suitable documentation for interactive fiction and games at the bit level: specifically, the absence of "representation information" needed to map raw bits into higher-level data constructs. Preserving Virtual Worlds 2 is a research project expanding on initial efforts in this field.
Read more →
Knapsack cryptosystems are cryptosystems whose security is based on the hardness of solving the knapsack problem. They remain quite unpopular because simple versions of these algorithms have been broken for several decades. However, that type of cryptosystem is a good candidate for post-quantum cryptography. The most famous knapsack cryptosystem is the Merkle-Hellman Public Key Cryptosystem, one of the first public key cryptosystems, published the same year as the RSA cryptosystem. However, this system has been broken by several attacks: one from Shamir, one by Adleman, and the low density attack. However, there exist modern knapsack cryptosystems that are considered secure so far: among them is Nasako-Murakami 2006. Knapsack cryptosystems, when not subject to classical cryptoanalysis, are believed to be difficult even for quantum computers. That is not the case for systems that rely on factoring large integers, like RSA, or computing discrete logarithms, like ECDSA, problems solved in polynomial time with Shor's algorithm.
Read more →