AI Driven Spreadsheet

AI Driven Spreadsheet — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

  • Secure coding

    Secure coding

    Secure coding is the practice of developing computer software in such a way that guards against the accidental introduction of security vulnerabilities. Defects, bugs and logic flaws are consistently the primary cause of commonly exploited software vulnerabilities. Through the analysis of thousands of reported vulnerabilities, security professionals have discovered that most vulnerabilities stem from a relatively small number of common software programming errors. By identifying the insecure coding practices that lead to these errors and educating developers on secure alternatives, organizations can take proactive steps to help significantly reduce or eliminate vulnerabilities in software before deployment. Some scholars have suggested that in order to effectively confront threats related to cybersecurity, proper security should be coded or "baked in" to the systems. With security being designed into the software, this ensures that there will be protection against insider attacks and reduces the threat to application security. Implementing secure coding practices is part of the secure by design approach to security engineering. == Buffer-overflow prevention == Buffer overflows, a common software security vulnerability, happen when a process tries to store data beyond a fixed-length buffer. For example, if there are 8 slots to store items in, there will be a problem if there is an attempt to store 9 items. In computer memory the overflowed data may overwrite data in the next location which can result in a security vulnerability (stack smashing) or program termination (segmentation fault). An example of a C program prone to a buffer overflow is If the user input is larger than the destination buffer, a buffer overflow will occur. To fix this unsafe program, use strncpy to prevent a possible buffer overflow. Another secure alternative is to dynamically allocate memory on the heap using malloc. In the above code snippet, the program attempts to copy the contents of src into dst, while also checking the return value of malloc() to ensure that enough memory was able to be allocated for the destination buffer. == Format-string attack prevention == A Format String Attack is when a malicious user supplies specific inputs that will eventually be entered as an argument to a function that performs formatting, such as printf(). The attack involves the adversary reading from or writing to the stack. The C printf function writes output to stdout. If the parameter of the printf function is not properly formatted, several security bugs can be introduced. Below is a program that is vulnerable to a format string attack. A malicious argument passed to the program could be "%s%s%s%s%s%s%s", which can crash the program from improper memory reads. == Integer-overflow prevention == Integer overflow occurs when an arithmetic operation results in an integer too large to be represented within the available space. A program which does not properly check for integer overflow introduces potential software bugs and exploits. Below is a function in C++ which attempts to confirm that the sum of x and y is less than or equal to a defined value MAX: The problem with the code is it does not check for integer overflow on the addition operation. If the sum of x and y is greater than the maximum possible value of an unsigned int, the addition operation will overflow and perhaps result in a value less than or equal to MAX, even though the sum of x and y is greater than MAX. Below is a function which checks for overflow by confirming the sum is greater than or equal to both x and y. If the sum did overflow, the sum would be less than x or less than y. == Path traversal prevention == Path traversal is a vulnerability whereby paths provided from an untrusted source are interpreted in such a way that unauthorised file access is possible. For example, consider a script that fetches an article by taking a filename, which is then read by the script and parsed. Such a script might use the following hypothetical URL to retrieve an article about dog food: https://www.example.net/cgi-bin/article.sh?name=dogfood.html If the script has no input checking, instead trusting that the filename is always valid, a malicious user could forge a URL to retrieve configuration files from the web server: https://www.example.net/cgi-bin/article.sh?name=../../../../../etc/passwd Depending on the script, this may expose the /etc/passwd file, which on Unix-like systems contains (among others) user IDs, their login names, home directory paths and shells. (See SQL injection for a similar attack.) == Regulatory drivers == Secure coding practices are increasingly mandated by regulatory frameworks governing the development and maintenance of software systems that process sensitive data. The Health Insurance Portability and Accountability Act (HIPAA) Security Rule requires covered entities to protect the integrity of protected health information through technical safeguards under 45 CFR 164.312(c)(1) and to implement mechanisms to authenticate electronic protected health information under 45 CFR 164.312(c)(2). The Payment Card Industry Data Security Standard (PCI DSS) version 4.0 Requirement 6.2 mandates that custom software is developed securely, including training developers in secure coding techniques (6.2.2), reviewing custom code for vulnerabilities before release (6.2.3), and addressing common software attacks in development practices (6.2.4).

    Read more →
  • EdgeRank

    EdgeRank

    EdgeRank is the name commonly given to the algorithm that Facebook uses to determine what articles should be displayed in a user's News Feed. As of 2011, Facebook has stopped using the EdgeRank system and uses a machine learning algorithm that, as of 2013, takes more than 100,000 factors into account. EdgeRank was developed and implemented by Serkan Piantino. == Formula and factors == In 2010, a simplified version of the EdgeRank algorithm was presented as: ∑ e d g e s e u e w e d e {\displaystyle \sum _{\mathrm {edges\,} e}u_{e}w_{e}d_{e}} where: u e {\displaystyle u_{e}} is user affinity. w e {\displaystyle w_{e}} is how the content is weighted. d e {\displaystyle d_{e}} is a time-based decay parameter. User Affinity: The User Affinity part of the algorithm in Facebook's EdgeRank looks at the relationship and proximity of the user and the content (post/status update). Content Weight: What action was taken by the user on the content. Time-Based Decay Parameter: New or old. Newer posts tend to hold a higher place than older posts. Some of the methods that Facebook uses to adjust the parameters are proprietary and not available to the public. A study has shown that it is possible to hypothesize a disadvantage of the "like" reaction and advantages of other interactions (e.g., the "haha" reaction or "comments") in content algorithmic ranking on Facebook. The "like" button can decrease the organic reach as a "brake effect of viral reach". The "haha" reaction, "comments" and the "love" reaction could achieve the highest increase in total organic reach. == Impact == EdgeRank and its successors have a broad impact on what users actually see out of what they ostensibly follow: for instance, the selection can produce a filter bubble (if users are exposed to updates which confirm their opinions etc.) or alter people's mood (if users are shown a disproportionate amount of positive or negative updates). As a result, for Facebook pages, the typical engagement rate is less than 1% (or less than 0.1% for the bigger ones), and organic reach 10% or less for most non-profits. As a consequence, for pages, it may be nearly impossible to reach any significant audience without paying to promote their content.

    Read more →
  • Data drilling

    Data drilling

    Data drilling (also drilldown) refers to any of various operations and transformations on tabular, relational, and multidimensional data. The term has widespread use in various contexts, but is primarily associated with specialized software designed specifically for data analysis. == Common data drilling operations == There are certain operations that are common to applications that allow data drilling. Among them are: Query operations: tabular query pivot query === Tabular query === Tabular query operations consist of standard operations on data tables. Among these operations are: search sort filter (by value) filter (by extended function or condition) transform (e.g., by adding or removing columns) Consider the following example: Fred and Wilma table (Fig 001): gender, fname, lname, home male, fred, chopin, Poland male, fred, flintstone, bedrock male, fred, durst, usa female, wilma, flintstone, bedrock female, wilma, rudolph, usa female, wilma, webb, usa male, fred, johnson, usa The preceding is an example of a simple flat file table formatted as comma-separated values. The table includes first name, last name, gender and home country for various people named fred or wilma. Although the example is formatted this way, it is important to emphasize that tabular query operations (as well as all data drilling operations) can be applied to any conceivable data type, regardless of the underlying formatting. The only requirement is that the data be readable by the software application in use. === Pivot query === A pivot query allows multiple representations of data according to different dimensions. This query type is similar to tabular query, except it also allows data to be represented in summary format, according to a flexible user-selected hierarchy. This class of data drilling operation is formally, (and loosely) known by different names, including crosstab query, pivot table, data pilot, selective hierarchy, intertwingularity and others. To illustrate the basics of pivot query operations, consider the Fred and Wilma table (Fig 001). A quick scan of the data reveals that the table has redundant information. This redundancy could be consolidated using an outline or a tree structure or in some other way. Moreover, once consolidated, the data could have many different alternate layouts. Using a simple text outline as output, the following alternate layouts are all possible with a pivot query: Summarize by gender (Fig 001): female flintstone, wilma rudolph, wilma webb, wilma male chopin, fred flintstone, fred durst, fred johnson, fred (Dimensions = gender; Tabular fields = lname, fname;) Summarize by home, lname (Fig 001): bedrock flintstone fred wilma Poland chopin fred usa ... (Dimensions = home, lname; Tabular fields = fname;) ==== Uses ==== Pivot query operations are useful for summarizing a corpus of data in multiple ways, thereby illustrating different representations of the same basic information. Although this type of operation appears prominently in spreadsheets and desktop database software, its flexibility is arguably under-utilized. There are many applications that allow only a 'fixed' hierarchy for representing data, and this represents a substantial limitation. == Drillup == Drillup is the opposite of drilldown. For example, if you drilldown to see the revenue of one product, then you might want to drillup to see the revenue of all products.

    Read more →
  • Apple Intelligence

    Apple Intelligence

    Apple Intelligence is a generative artificial intelligence system developed by Apple Inc. Relying on a combination of on-device and server processing, it was announced on June 10, 2024, at the 2024 Worldwide Developers Conference, as a built-in feature of Apple's iOS 18, iPadOS 18, and macOS Sequoia, which were announced alongside Apple Intelligence. Apple Intelligence is free for all users with supported devices. On macOS, Apple Intelligence is available only on Apple silicon Mac computers; Intel-based Mac computers are not supported. Features include writing tools that assist users with grammar and proofreading, image generation, summaries of system notifications, AI-assisted image retouching in the Photos app, and integration with ChatGPT, the popular chatbot by OpenAI. As of March 2026, Apple Intelligence is not available yet on devices purchased in mainland China or on any device using an Apple Account set to mainland China, even if the device was bought elsewhere. == History == === Background === Apple first implemented artificial intelligence features in its products with the release of Siri in the iPhone 4S in 2011. In the years after its release, Apple engaged in efforts to ensure its artificial intelligence operations remained covert; according to University of California, Berkeley professor Trevor Darrell, the company's secrecy deterred graduate students. The company started expanding its artificial intelligence team in 2015, opening up its operations by publishing more scientific papers and joining AI industry research groups. Apple reportedly acquired more AI companies from 2016 to 2020. In 2017, Apple released the iPhone 8 and the iPhone X with the A11 Bionic processor, which featured its first dedicated Neural Engine for accelerating common machine learning tasks. Despite its investments in artificial intelligence, Siri was criticized both by reviewers and internally at Apple for lagging behind other AI assistants. The rapid development of generative artificial intelligence and the release of ChatGPT in late 2022 reportedly blindsided Apple executives and forced the company to refocus its efforts on AI. In an interview with Good Morning America, Apple CEO Tim Cook stated that generative AI had "great promise" but had some potential dangers, and that it was "looking closely" at ChatGPT. It was first reported in July 2023 that Apple was creating its own internal large language model, codenamed "Ajax". In October 2023, Apple was reportedly on track to release new generative AI features into its operating systems by 2024, including a significantly redeveloped Siri. In an earnings call in February 2024, Cook stated that the company was spending a "tremendous amount of time and effort" into AI features that would be shared "later that year". === Google deal === In January 2026, Apple and Google announced a multi-year partnership under which Apple’s next-generation foundation models are expected to incorporate Google’s Gemini models and cloud infrastructure. According to the companies, the collaboration is intended to support future Apple Intelligence features, including enhancements to Siri, while Apple Intelligence will continue to operate on Apple devices and through Apple’s Private Cloud Compute system, which Apple states is designed to preserve user privacy. On an earnings call, Apple reported to investors that they were integrating an on-device model of the Google Gemini AI to Siri, as the development of their model was beset with setbacks. Apple has previously tested and used other third-party AI models like ChatGPT, but according to a Bloomberg article by Mark Gurman, Apple pushed forward the proposed Google deal; by using Google's Gemini model possessing 1.2 trillion parameters, Apple would integrate a much larger and more complex model than those it previously developed and used. Of note, comparable AI models from other major companies (including OpenAI and Meta) have also been reported to operate at a similar “trillion-parameter” scale and to compete against Gemini-class systems on benchmarks. == Models == Apple Intelligence consists of an on-device model as well as a cloud model running on servers primarily using Apple silicon. Both models consist of a generic foundation model, as well as multiple adapter models that are more specialized to particular tasks like text summarization and tone adjustment. It was launched for developers and testers on July 29, 2024, in U.S. English, with the developer betas of iOS 18.1, macOS 15.1, and iPadOS 18.1, released partially on October 28, 2024, and will fully launch by 2026. According to a human evaluation done by Apple's machine learning division, the on-device foundation model beat or tied equivalent small models by Mistral AI, Microsoft, and Google, while the server foundation models beat the performance of OpenAI's GPT-3, while roughly matching the performance of GPT-4. Apple's cloud models are built on a Private Cloud Compute platform which is allegedly designed with user privacy and end-to-end encryption in mind. Unlike other generative AI services like ChatGPT which use servers from third parties, Apple Intelligence's cloud models are run entirely on Apple servers with custom Apple silicon hardware built for end-to-end encryption. It was also designed to make sure that the software running on said servers matches the independently verifiable software accessible to researchers. In case of a software mismatch, Apple devices will refuse to connect to the servers. On June 10, 2025, Apple announced that Apple's on-device foundation models will be available to third-party applications as part of the Foundation Models API, with support for structured data response and tool calling. == Features == === Writing tools === Apple Intelligence features writing tools that are powered by LLMs. Selected text can be proofread, rewritten, made more friendly, concise or professional, similar to the AI writing features of the popular online English-language writing assistant tool Grammarly. It can also be used to generate summaries, key points, tables, and lists from an article or piece of writing. In iOS 18.2 and macOS 15.2, a ChatGPT integration was added to Writing Tools through "Compose" and "Describe your change" features. === Real-time Translation === Apple Intelligence enables the real-time translation of messages, photos and videos, and phone calls, through Apple's hardware. For communicating with foreigners, using the Translate app on iPhone to show subtitles in their language or to play back the translated audio naturally in their language, and also by wearing AirPods with Live Translation can now help to understand what someone is saying in users' preferred language in conversation. If both have headphones, simultaneous interpretation can be achieved. === Image Playground === Apple Intelligence can be used to generate images on-device with the Image Playground app. Similarly to OpenAI's DALL-E, it can be used to generate images using AI, using phrases and descriptions to output an image with customizable styles such as Animation and Sketch. In Notes, users can access Image Playground on iPad through the Image Wand tool in the Apple Pencil palette without having to open the Image Playground app. Rough sketches made with Apple Pencil can be transformed into images. As part of iOS, iPadOS, and macOS 26, Image Playground now integrates with the image generation models built into ChatGPT. === Genmoji === Using Apple Intelligence text-to-image models, users can generate unique "Genmoji" images by typing descriptions (prompting). Users can pick people in photos to have Genmoji generate images that resemble them. Similarly to emoji, Genmoji can be added inline to text messages, tapbacks, stickers and can be shared in Messages as well in third-party applications as inline messages or as stickers. === Siri overhaul === Siri, which used to be Apple's virtual assistant, has been updated to be an LLM chatbot, with enhanced capabilities made possible by Apple Intelligence. The latest iteration features an updated user interface, improved natural language processing, and the option to interact via text by double tapping the home bar without enabling the feature in the Accessibility menu, or double-clicking the command key on macOS. In a later update, Apple Intelligence will add the ability for Siri to use personal context from device activities to answer queries. === Mail === Apple Intelligence adds a feature called Priority Messages to the Mail app, which shows urgent emails such as same-day invitations or boarding passes, with AI generated summaries of the email. The Mail app also gains the ability to categorize incoming mail into Primary, Transactions, Updates, and Promotions based on what the email contains, which Apple claims is done all on-device. === Photos === Apple's Photos app includes a feature to create custom memory movies and enhanced search capabilities. Users can describe

    Read more →
  • IMPACT (computer graphics)

    IMPACT (computer graphics)

    IMPACT (sometimes spelled Impact) is a computer graphics architecture for Silicon Graphics computer workstations. IMPACT Graphics was developed in 1995 and was available as a high-end graphics option on workstations released during the mid-1990s. IMPACT graphics gives the workstation real-time 2D and 3D graphics rendering capability similar to that of even high-end PCs made well after IMPACT's introduction. IMPACT graphics systems consist of either one or two Geometry Engines and one or two Raster Engines in various configurations. IMPACT graphics consists of five graphics subsystems: the Command Engine, Geometry Subsystem, Raster Engine, framebuffer and Display Subsystem. IMPACT Graphics can produce resolutions up to 1600 x 1200 pixels with 32-bit color and can also process unencoded NTSC and PAL analog television signals. IMPACT graphics subsystems come in three configurations for SGI Indigo2 IMPACT workstations: Solid IMPACT, High IMPACT, and Maximum IMPACT. The equivalent configurations also exist for the SGI Octane workstation but are referred to as SI, SSI, and MXI (I-series). Later Octane workstations used a similar configuration but with updated ASIC chips and are referred to as SE, SSE, and MXE (E-series). IMPACT uses Rambus RDRAM for texture memory. The IMPACT graphics architecture was superseded by SGI's VPro graphics architecture in 1997.

    Read more →
  • Discoverability

    Discoverability

    Discoverability is the degree to which something, especially a piece of content or information, can be found in a search of a file, database, or other information system. Discoverability is a concern in library and information science, many aspects of digital media, software and web development, and in marketing, since products and services cannot be used if people cannot find it or do not understand what it can be used for. In human-computer interaction the term is further used to describe the discoverability of interactions, features and interactive systems overall . Metadata, or "information about information", such as a book's title, a product's description, or a website's keywords, affects how discoverable something is on a database or online. Adding metadata to a product that is available online can make it easier for end users to find the product. For example, if a song file is made available online, making the title, band name, genre, year of release, and other pertinent information available in connection with this song means the file can be retrieved more easily. The organization of information through the implementation of alphabetical structures or the integration of content into search engines exemplifies strategies employed to enhance the discoverability of information. The concept of discoverability, while related to but distinct from accessibility and usability, which are other qualities that affect the usefulness of a piece of information, is a critical aspect of information retrieval. == Etymology == The concept of "discoverability" in an information science and online context is a loose borrowing from the concept of the similar name in the legal profession. In law, "discovery" is a pre-trial procedure in a lawsuit in which each party, through the law of civil procedure, can obtain evidence from the other party or parties by means of discovery devices such as a request for answers to interrogatories, request for production of documents, request for admissions and depositions. Discovery can be obtained from non-parties using subpoenas. When a discovery request is objected to, the requesting party may seek the assistance of the court by filing a motion to compel discovery. == Purpose == The usability of any piece of information directly relates to how discoverable it is, either in a "walled garden" database or on the open Internet. The quality of information available on this database or on the Internet depends upon the quality of the meta-information about each item, product, or service. In the case of a service, because of the emphasis placed on service reusability, opportunities should exist for reuse of this service. However, reuse is only possible if information is discoverable in the first place. To make items, products, and services discoverable, the process is as follows: Document the information about the item, product or service (the metadata) in a consistent manner. Store the documented information (metadata) in a searchable repository. while technically a human-searchable repository, such as a printed paper list would qualify, "searchable repository" is usually taken to mean a computer-searchable repository, such as a database that a human user can search using some type of search engine or "find" feature. Enable search for the documented information in an efficient manner. supports number 2, because while reading through a printed paper list by hand might be feasible in a theoretical sense, it is not time and cost-efficient in comparison with computer-based searching. Apart from increasing the reuse potential of the services, discoverability is also required to avoid development of solution logic that is already contained in an existing service. To design services that are not only discoverable but also provide interpretable information about their capabilities, the service discoverability principle provides guidelines that could be applied during the service-oriented analysis phase of the service delivery process. === Specific to digital media === In relation to audiovisual content, according to the meaning given by the Canadian Radio-television and Telecommunications Commission (CRTC) for the purpose of its 2016 Discoverability Summit, discoverability can be summed up to the intrinsic ability of given content to "stand out of the lot", or to position itself so as to be easily found and discovered. A piece of audiovisual content can be a movie, a TV series, music, a book (eBook), an audio book or podcast. When audiovisual content such as a digital file for a TV show, movie, or song, is made available online, if the content is "tagged" with identifying information such as the names of the key artists (e.g., actors, directors and screenwriters for TV shows and movies; singers, musicians and record producers for songs) and the genres (for movies genres, music genres, etc.). When users interact with online content, algorithms typically determine what types of content the user is interested in, and then a computer program suggests "more like this", which is other content that the user may be interested in. Different websites and systems have different algorithms, but one approach, used by Amazon (company) for its online store, is to indicate to a user: "customers who bought x also bought y" (affinity analysis, collaborative filtering). This example is oriented around online purchasing behaviour, but an algorithm could also be programmed to provide suggestions based on other factors (e.g., searching, viewing, etc.). Discoverability is typically referred to in connection with search engines. A highly "discoverable" piece of content would appear at the top, or near the top of a user's search results. A related concept is the role of "recommendation engines", which give a user recommendations based on his/her previous online activity. Discoverability applies to computers and devices that can access the Internet, including various console video game systems and mobile devices such as tablets and smartphones. When producers make an effort to promote content (e.g., a TV show, film, song, or video game), they can use traditional marketing (billboards, TV ads, radio ads) and digital ads (pop-up ads, pre-roll ads, etc.), or a mix of traditional and digital marketing. Even before the user's intervention by searching for a certain content or type of content, discoverability is the prime factor which contributes to whether a piece of audiovisual content will be likely to be found in the various digital modes of content consumption. As of 2017, modes of searching include looking on Netflix for movies, Spotify for music, Audible for audio books, etc., although the concept can also more generally be applied to content found on Twitter, Tumblr, Instagram, and other websites. It involves more than a content's mere presence on a given platform; it can involve associating this content with "keywords" (tags), search algorithms, positioning within different categories, metadata, etc. Thus, discoverability enables as much as it promotes. For audiovisual content broadcast or streamed on digital media using the Internet, discoverability includes the underlying concepts of information science and programming architecture, which are at the very foundation of the search for a specific product, information or content. === Human-Computer Interaction === In human–computer interaction (HCI), discoverability refers to the ability of users to perceive and comprehend a system, function, or input method upon encountering it, despite a lack of prior awareness or knowledge, whether through intentional effort or serendipitously . The concept was popularised by Don Norman, who framed it around whether users can determine what actions are possible and how to perform them . Discoverability is considered a precondition for learnability, though the two concepts are frequently conflated in the literature . == Applications == === Within a webpage === Within a specific webpage or software application ("app"), the discoverability of a feature, content or link depends on a range of factors, including the size, colour, highlighting features, and position within the page. When colour is used to communicate the importance of a feature or link, designers typically use other elements as well, such as shadows or bolding, for individuals, who cannot see certain colours. Just as traditional paper printing created other physical locations that stood out, such as being "above the fold" of a newspaper versus "below the fold", a web page or app's screenview may have certain locations that give features additional visibility to users, such as being right at the bottom of the web page or screen. The positional advantages or disadvantages of various locations depend on different cultures and languages (e.g., left to right vs. right to left). Some locations have become established, such as having toolbars at the top of a screen or webpage. Some designers have argued t

    Read more →
  • Synthetic data

    Synthetic data

    Synthetic data are artificially generated data not produced by real-world events. Typically created using algorithms, synthetic data can be deployed to validate mathematical models and to train machine learning models. Data generated by a computer simulation can be seen as synthetic data. This encompasses most applications of physical modeling, such as music synthesizers or flight simulators. The output of such systems approximates the real thing, but is fully algorithmically generated. Synthetic data is used in a variety of fields as a filter for information that would otherwise compromise the confidentiality of particular aspects of the data. In many sensitive applications, datasets theoretically exist but cannot be released to the general public; synthetic data sidesteps the privacy issues that arise from using real consumer information without permission or compensation. == Usefulness == Synthetic data is generated to meet specific needs or certain conditions that may not be found in the original, real data. One of the hurdles in applying up-to-date machine learning approaches for complex scientific tasks is the scarcity of labeled data, a gap effectively bridged by the use of synthetic data, which closely replicates real experimental data. This can be useful when designing many systems, from simulations based on theoretical value, to database processors, etc. This helps detect and solve unexpected issues such as information processing limitations. Synthetic data are often generated to represent the authentic data and allows a baseline to be set. Another benefit of synthetic data is to protect the privacy and confidentiality of authentic data, while still allowing for use in testing systems. Computer security experts claim generated synthetic data "... enables us to create realistic behavior profiles for users and attackers. The data is used to train the fraud detection system itself, thus creating the necessary adaptation of the system to a specific environment." In defense and military contexts, synthetic data is seen as a potentially valuable tool to develop and improve complex AI systems, particularly in contexts where high-quality real-world data is scarce. At the same time, synthetic data together with the testing approach can give the ability to model real-world scenarios. == History == Scientific modelling of physical systems has a long history that runs concurrent with the history of physics. For example, research into synthesis of audio and voice can be traced back to the 1930s and before, driven forward by the developments of the telephone and audio recording technologies. Digitization gave rise to software synthesizers from the 1970s onwards. In the context of privacy-preserving statistical analysis, in 1993, the idea of original fully synthetic data was created by Donald Rubin. Rubin originally designed this to synthesize the Decennial Census long form responses for the short form households. He then released samples that did not include any actual long form records - in this he preserved anonymity of the household. Later that year, the idea of original partially synthetic data was created by Little. Little used this idea to synthesize the sensitive values on the public use file. A 1993 work fitted a statistical model to 60,000 MNIST digits, then it was used to generate over 1 million examples. Those were used to train a LeNet-4 to reach state of the art performance. In 1994, Stephen Fienberg introduced 'critical refinement', in which a parametric posterior predictive distribution (instead of a Bayes bootstrap) is used to do the sampling. Later, other important contributors to the development of synthetic data generation were Trivellore Raghunathan, Jerry Reiter, Donald Rubin, John M. Abowd, and Jim Woodcock. Collectively they came up with a solution for how to treat partially synthetic data with missing data. Similarly, they developed the technique of Sequential Regression Multivariate Imputation. == Calculations == Researchers test the framework on synthetic data, which is "the only source of ground truth on which they can objectively assess the performance of their algorithms". Synthetic data can be generated through the use of random lines, having different orientations and starting positions. Datasets can get fairly complicated. A more complicated dataset can be generated by using a synthesizer build. To create a synthesizer build, first use the original data to create a model or equation that fits the data the best. This model or equation will be called a synthesizer build. This build can be used to generate more data. Constructing a synthesizer build involves constructing a statistical model. In a linear regression line example, the original data can be plotted, and a best fit linear line can be created from the data. This line is a synthesizer created from the original data. The next step will be generating more synthetic data from the synthesizer build or from this linear line equation. In this way, the new data can be used for studies and research, and it protects the confidentiality of the original data. David Jensen from the Knowledge Discovery Laboratory explains how to generate synthetic data: "Researchers frequently need to explore the effects of certain data characteristics on their data model." To help construct datasets exhibiting specific properties, such as auto-correlation or degree disparity, proximity can generate synthetic data having one of several types of graph structure: random graphs that are generated by some random process; lattice graphs having a ring structure; lattice graphs having a grid structure, etc. In all cases, the data generation process follows the same process: Generate the empty graph structure. Generate attribute values based on user-supplied prior probabilities. Since the attribute values of one object may depend on the attribute values of related objects, the attribute generation process assigns values collectively. == Applications == === Fraud detection and confidentiality systems === Testing and training fraud detection and confidentiality systems are devised using synthetic data. Specific algorithms and generators are designed to create realistic data, which then assists in teaching a system how to react to certain situations or criteria. For example, intrusion detection software is tested using synthetic data. This data is a representation of the authentic data and may include intrusion instances that are not found in the authentic data. The synthetic data allows the software to recognize these situations and react accordingly. If synthetic data was not used, the software would only be trained to react to the situations provided by the authentic data and it may not recognize another type of intrusion. === Scientific research === Researchers doing clinical trials or any other research may generate synthetic data to aid in creating a baseline for future studies and testing. Real data can contain information that researchers may not want released, so synthetic data is sometimes used to protect the privacy and confidentiality of a dataset. Using synthetic data reduces confidentiality and privacy issues since it holds no personal information and cannot be traced back to any individual. Beyond privacy protection, synthetic data is also being explored for methodological innovation in drug development. For instance, synthetic data may be used to construct synthetic control arms as an alternative to conventional external control arms based on real-world data (RWD) or randomized controlled trials (RCTs). Collectively, regulatory agencies such as the FDA and EMA appear to be at various stages of recognizing and integrating AI-generated synthetic data into their methodologies. While there is growing consensus on the potential of such data to support model development and the broader lifecycle of medicinal products, to date no drug or medical device has been approved using solely or predominantly synthetic data—particularly not as a comparator arm generated entirely via data-driven algorithms. The quality and statistical handling of synthetic data are expected to become more prominent in future regulatory discussions, particularly in contexts such as predictive modeling (e.g., digital twins), where innovative approaches have already been referenced. === Machine learning === Synthetic data is increasingly being used for machine learning applications: a model is trained on a synthetically generated dataset with the intention of transfer learning to real data. Efforts have been made to enable more data science experiments via the construction of general-purpose synthetic data generators, such as the Synthetic Data Vault. In general, synthetic data has several natural advantages: once the synthetic environment is ready, it is fast and cheap to produce as much data as needed; synthetic data can have perfectly accurate labels, including labeling that may be very expensive or impo

    Read more →
  • Novell Storage Manager

    Novell Storage Manager

    Novell Storage Manager is a system software package released by Novell in 2004 that uses identity, policy and directory events to automate full lifecycle management of file storage for individual users and organizational groups. By tying storage management to an organization's existing identity infrastructure, it has been pointed out, Novell Storage Manager enables the administration of users across all file servers "as a single pool rather than [in] separate independently managed domains." Novell Storage Manager is a component of the Novell File Management Suite. == How It Works == Novell Storage Manager dynamically manages and provisions storage based on user and group events that occur in the directory, including user creations, group assignments, moves, renames, and deletions. When a change happens in the directory that affects a user’s file storage needs or user storage policy, Storage Manager applies the appropriate policy and makes the necessary changes at the file system level to address those storage needs. The following key components comprise Novell Storage Manager's identity and policy-driven state machine architecture: Directory services; Storage policies; Novell Storage Manager event monitors; Novell Storage Manager policy engine; Novell Storage Manager agents; and Action objects. This state machine architecture enables the engine to properly deal with transient waits with directory synchronization issues. It also allows recovery from failures involving network communications, a target server or a server running a component of Storage Manager—including the policy engine itself. If a failure or interruption occurs at any point during operation, Storage Manager will be able to successfully continue the operation from where it was when the interruption occurred. == Reviews == Jon Toigo called Novell Storage Manager "a robust and smart approach to corralling user files... into an organized and efficient management scheme". He also said it was "best in class" of the products he'd reviewed.

    Read more →
  • Association for Computational Linguistics

    Association for Computational Linguistics

    The Association for Computational Linguistics (ACL) is a scientific and professional organization for people working on natural language processing. Its namesake conference is one of the primary high impact conferences for natural language processing research, along with EMNLP. The conference is held each summer in locations where significant computational linguistics research is carried out. It was founded in 1962, originally named the Association for Machine Translation and Computational Linguistics (AMTCL). It became the ACL in 1968. The ACL has a European (EACL), a North American (NAACL), and an Asian (AACL) chapter. == History == The ACL was founded in 1962 as the Association for Machine Translation and Computational Linguistics (AMTCL). The initial membership was about 100. In 1965, the AMTCL took over the journal Mechanical Translation and Computational Linguistics. This journal was succeeded by many other journals: the American Journal of Computational Linguistics (1974–1978, 1980–1983), and then Computational Linguistics (1984–present). Since 1988, the journal has been published for the ACL by MIT Press. The annual meeting was first held in 1963 in conjunction with the Association for Computing Machinery National Conference. The annual meeting was, for a long time, relatively informal and did not publish anything longer than abstracts. By 1968, the society took on its current name, the Association for Computational Linguistics (ACL). The publication of the annual meeting's Proceedings of the ACL began in 1979 and gradually matured into its modern form. Many of the meetings were held in conjunction with the Linguistic Society of America, and a few with the American Society for Information Science and the Cognitive Science Society. The United States government sponsored much research from 1989 to 1994, characterized by an increase in author retention rates and an increase in research in some key topics, such as speech recognition, in ACL. By the 21st century, it was able to maintain authors at a high rate who coalesced in a more stable arrangement around individual research topics. In 1991, the group published a prototype for a text generator based on the universal grammar theory of Noam Chomsky. The system, nicknamed Parrot, relied on a finite set of syntactic transformations and a hand-curated lexicon. Despite some initial success, including experimentation with morpheme syntactics, funding halted after the research team encountered intractable difficulties with inflection and abstract locutions. == Annual Meeting of the ACL == Every year, the ACL holds the Annual Meeting of the ACL. The location lies in Europe in years zero modulo three, North America in years one modulo three, and Asia–Australia in years two modulo three. In 2020, the Annual Meeting received for the first time more submissions from China than the United States. == Activities == The ACL organizes several of the top conferences and workshops in the field of computational linguistics and natural language processing. These include: Annual Meeting of the Association for Computational Linguistics (ACL), the flagship conference of the organization Empirical Methods in Natural Language Processing (EMNLP) International Joint Conference on Natural Language Processing (IJCNLP), held jointly one of the other conferences on a rotating basis Conference on Computational Natural Language Learning (CoNLL) Lexical and Computational Semantics and Semantic Evaluation (SemEval) Joint Conference on Lexical and Computational Semantics (SEM) Workshop on Statistical Machine Translation (WMT) Besides conferences, the ACL also sponsors the journals Computational Linguistics and Transactions of the Association for Computational Linguistics (TACL). Papers and other presentations at ACL and ACL-affiliated venues are archived online in the open-access ACL Anthology. == Special Interest Groups == ACL has a large number of Special Interest Groups (SIGs), focusing on specific areas of natural language processing. Some current SIGs within ACL are: == Presidents == Each year, the ACL elects a distinguished computational linguist who becomes vice-president of the organization in the next calendar year and president one year later. Recent ACL presidents are:

    Read more →
  • BRS/Search

    BRS/Search

    BRS/Search is a full-text database and information retrieval system. BRS/Search uses a fully inverted indexing system to store, locate, and retrieve unstructured data. It was the search engine that in 1977 powered Bibliographic Retrieval Services (BRS) commercial operations with 20 databases (including the first national commercial availability of MEDLINE); it has changed ownership several times during its development and is currently sold as Livelink ECM Discovery Server by Open Text Corporation. == Early development == Development on what was to become BRS began as Biomedical Communications Network (BCN) at the State University of New York at Albany (SUNY). BCN, which went online in 1968, provided on-line access to nine databases, including MEDLINE and BIOSIS Previews, to large universities and medical schools primarily in the Northeast of the USA. State funding for the project was withdrawn in 1975, and Bibliographic Retrieval Services (BRS) was formed as a non-profit concern the following year. It was incorporated in May 1976 as a for-profit corporation with Ron Quake as president, Jan Egeland as vice president in charge of marketing and training, and Lloyd Palmer as vice president of systems. == BRS commercial operations == In December 1976, the First BRS User Meeting was held in Syracuse, New York, and by January 1977 BRS started commercial operations with 20 databases (including the first national commercial availability of MEDLINE) and 9 million records, using modified IBM STAIRS (STorage And Information Retrieval System) software, Telenet for telecommunications, and timesharing mainframe computers of Carrier Corporation. In October 1980 BRS was sold by Egeland and Quake to Indian Head, Inc., a subsidiary of the Dutch company Thyssen-Bornemisza Group. == 1989–1993 == In 1989 Robert Maxwell acquired BRS and the BRS/Search software; he announced the planned incorporation of the ORBIT Search Service and BRS Information Technologies and renamed the whole group Maxwell Online, Inc. At that time BRS Information Technologies was serving the medical and academic library marketplace with over 150 databases. Maxwell later bought the publishing company Macmillan and put Maxwell Online under Macmillan. In the same year BRS/LINK (hypertext connection of databases; first application delivering full text) was announced. The initial BRS/LINK application "relates the citation in a bibliographic database to its full-text article in a second database," and "eliminates the need to re-execute a search strategy in the second database in order to find the corresponding full-text article." Initially BRS/LINK supported linking only selected bibliographic databases: MEDLINE, Health Planning and Administration, and MEDLINE References on AIDS to the full-text Comprehensive Core Medical Library. At the time of Robert Maxwell’s death in 1991, Macmillan brought in Andrew Gregory to represent the company during the 2 years that Maxwell’s affairs were being settled and to prepare Maxwell Online to be able to sell the components. Maxwell Online shortly thereafter underwent yet another name change, this time to InfoPro Technologies. == Dataware Technologies ownership of BRS/SEARCH == Early in 1994, InfoPro Technologies, a subsidiary of MHC Inc. (holding company for Macmillan Inc.), the former Maxwell Online service, sold off all its subsidiaries. ORBIT Search Services went to the French-owned Questel, the dial-up BRS Search Services to CD Plus Technologies (later to become OVID), and BRS Software Products (including BRS/SEARCH) to Dataware Technologies. Almost up to the end of InfoPro Technologies, BRS Software had been the fastest growing segment of the company. At the 14th BRS North American Users Group Conference in 1999, Dave Schubmehl of Dataware Technologies presented a paper in which he stated "The purpose of this presentation is to update BRS users on upcoming releases of BRS/Search, NetAnswer, and other Dataware products. BRS/Search 7.0 will include features specifically requested by customers, as well as other enhancements. Earlier this year, Dataware acquired Sovereign Hill Software, makers of InQuery. In light of that acquisition, and Dataware's other development projects, we'll look at Dataware's plans for all products, including BRS/Search and NetAnswer." == Open Text acquisition of BRS/Search == In 2001 BRS/Search was acquired by Open Text and became LiveLink ECM Discovery Server. It is now referred to as Open Text Discovery Server. Open Text still supports both BRS/Search and NetAnswer. The core BRS/Search technology in the Open Text portfolio was augmented with other capabilities through various acquisitions. For example, Dataware's acquisition of Sovereign-Hill brought InQuery, “a probabilistic information retrieval system using an inference network”, which was developed by the University of Massachusetts Amherst Center for Intelligent Information Retrieval] out of the UMass CIIR and into the marketplace. A product re-branding table shows the range of products, their old names and their new names. InQuery is a concept search engine that uses noun phrases, parts of speech and other co-occurrence relationships in overlapping passages of text rather than single term inverted indexes of single words in documents. Open Text's portfolio has grown to include Hummingbird Content Management, and has always included BASIS. == 2003 == BRS/Search North America User's Group (BRSNAUG) website with a June 8, 2003 date listed the following features for BRS/Search. The BRSNAUG also disincorporated in 2003. Cross-references to BRS/Search on the World Wide Web point to Open Text Livelink. Engine features include: Rapid query response time. Numerical data handling and elementary statistical processing (sum, avg, min, max) Search results weighting and relevancy ranking Left- and right-truncation and expansion of search terms Superior data compression – loaded databases typically use only about 1.5 times the input stream size in disk space Large capacity databases – up to 100 million documents, each with up to 65,000 paragraphs Fine control of indexing and searching – right down to the word, sentence, and paragraph level Fine control over data security. Document access can be controlled at the database, document, and paragraph level International language support for all 7/8 bit characters sets and customizable language tables Flexible and customizable stop word lists ANSI-compatible thesauri Hypertext links within and between documents and databases (R6.x) Support for natural language parsing of queries Automatic document summarization tools Client/Server development Programming interfaces for World-Wide Web (HTTP, HTML) access to databases

    Read more →
  • Gutmann method

    Gutmann method

    The Gutmann method is an algorithm for securely erasing the contents of computer hard disk drives, such as files. Devised by Peter Gutmann and Colin Plumb and presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory in July 1996, it involved writing a series of 35 patterns over the region to be erased. The selection of patterns assumes that the user does not know the encoding mechanism used by the drive, so it includes patterns designed specifically for three types of drives. A user who knows which type of encoding the drive uses can choose only those patterns intended for their drive. A drive with a different encoding mechanism would need different patterns. Most of the patterns in the Gutmann method were designed for older MFM/RLL-encoded disks. Gutmann himself has noted that more modern drives no longer use these older encoding techniques, making parts of the method irrelevant. He said "In the time since this paper was published, some people have treated the 35-pass overwrite technique described in it more as a kind of voodoo incantation to banish evil spirits than the result of a technical analysis of drive encoding techniques". Since about 2001, some ATA IDE and SATA hard drive manufacturer designs include support for the ATA Secure Erase standard, obviating the need to apply the Gutmann method when erasing an entire drive. The Gutmann method does not apply to USB sticks: a 2011 study reports that 71.7% of data remained available. On solid state drives it resulted in 0.8–4.3% recovery. == Background == The delete function in most operating systems simply marks the space occupied by the file as reusable (removes the pointer to the file) without immediately removing any of its contents. At this point the file can be fairly easily recovered by numerous recovery applications. However, once the space is overwritten with other data, there is no known way to use software to recover it. It cannot be done with software alone since the storage device only returns its current contents via its normal interface. Gutmann claims that intelligence agencies have sophisticated tools, including magnetic force microscopes, which together with image analysis, can detect the previous values of bits on the affected area of the media (for example hard disk). This claim however seems to be invalid based on the thesis "Data Reconstruction from a Hard Disk Drive using Magnetic Force Microscopy". == Method == An overwrite session consists of a lead-in of four random write patterns, followed by patterns 5 to 31 (see rows of table below), executed in a random order, and a lead-out of four more random patterns. Each of patterns 5 to 31 was designed with a specific magnetic media encoding scheme in mind, which each pattern targets. The drive is written to for all the passes even though the table below only shows the bit patterns for the passes that are specifically targeted at each encoding scheme. The result should obscure any data on the drive so that only the most advanced physical scanning (e.g., using a magnetic force microscope) of the drive is likely to be able to recover any data. The series of patterns is as follows: Encoded bits shown in bold are what should be present in the ideal pattern, although due to the encoding the complementary bit is actually present at the start of the track. == Criticism == Daniel Feenberg of the National Bureau of Economic Research, an American private nonprofit research organization, criticized Gutmann's claim that intelligence agencies are likely to be able to read overwritten data, citing a lack of evidence for such claims. He finds that Gutmann cites one non-existent source and sources that do not actually demonstrate recovery, only partially-successful observations. The definition of "random" is also quite different from the usual one used: Gutmann expects the use of pseudorandom data with sequences known to the recovering side, not an unpredictable one such as a cryptographically secure pseudorandom number generator. Nevertheless, some published government security procedures consider an overwritten disk to still be sensitive. Human factors and potential limitations in the overwriting software create a residual risk that is not considered acceptable at the highest security levels. Gutmann himself has responded to some of these criticisms and also criticized how his algorithm has been abused in an epilogue to his original paper, in which he states: In the time since this paper was published, some people have treated the 35-pass overwrite technique described in it more as a kind of voodoo incantation to banish evil spirits than the result of a technical analysis of drive encoding techniques. As a result, they advocate applying the voodoo to PRML and EPRML drives even though it will have no more effect than a simple scrubbing with random data. In fact performing the full 35-pass overwrite is pointless for any drive since it targets a blend of scenarios involving all types of (normally-used) encoding technology, which covers everything back to 30+-year-old MFM methods (if you don't understand that statement, re-read the paper). If you're using a drive which uses encoding technology X, you only need to perform the passes specific to X, and you never need to perform all 35 passes. For any modern PRML/EPRML drive, a few passes of random scrubbing is the best you can do. As the paper says, "A good scrubbing with random data will do about as well as can be expected". This was true in 1996, and is still true now. Gutmann's statement has been criticized for not recognizing that PRML/EPRML does not replace RLL, with critics claiming PRML/EPRML to be a signal detection method rather than a data encoding method. Polish data recovery service Kaleron has also claimed that Gutmann's publication contains further factual errors and assumptions that do not apply to actual disks.

    Read more →
  • Information literacy

    Information literacy

    The Association of College and Research Libraries defines information literacy as a "set of integrated abilities encompassing the reflective discovery of information, the understanding of how information is produced and valued and the use of information in creating new knowledge and participating ethically in communities of learning". In the United Kingdom, the Chartered Institute of Library and Information Professionals' definition also makes reference to knowing both "when" and "why" information is needed. The 1989 American Library Association (ALA) Presidential Committee on Information Literacy formally defined information literacy (IL) as attributes of an individual, stating that "to be information literate, a person must be able to recognize when information is needed and have the ability to locate, evaluate and use effectively the needed information". In 1990, academic Lori Arp published a paper asking, "Are information literacy instruction and bibliographic instruction the same?" Arp argued that neither term was particularly well defined by theoreticians or practitioners in the field. Further studies were needed to lessen the confusion and continue to articulate the parameters of the question. The Alexandria Proclamation of 2005 defined the term as a human rights issue: "Information literacy empowers people in all walks of life to seek, evaluate, use and create information effectively to achieve their personal, social, occupational and educational goals. It is a basic human right in a digital world and promotes social inclusion in all nations." The United States National Forum on Information Literacy defined information literacy as "the ability to know when there is a need for information, to be able to identify, locate, evaluate, and effectively use that information for the issue or problem at hand." Meanwhile, in the UK, the library professional body CILIP, define information literacy as "the ability to think critically and make balanced judgements about any information we find and use. It empowers us as citizens to develop informed views and to engage fully with society." A number of other efforts have been made to better define the concept and its relationship to other skills and forms of literacy. Other pedagogical outcomes related to information literacy include traditional literacy, computer literacy, research skills and critical thinking skills. Information literacy as a sub-discipline is an emerging topic of interest and counter measure among educators and librarians with the prevalence of misinformation, fake news, and disinformation. Scholars have argued that in order to maximize people's contributions to a democratic and pluralistic society, educators should be challenging governments and the business sector to support and fund educational initiatives in information literacy. == History == The phrase "information literacy" first appeared in print in a 1974 report written on behalf of the National Commission on Libraries and Information Science by Paul G. Zurkowski, who was at the time president of the Information Industry Association (now the Software and Information Industry Association). Zurkowski used the phrase to describe the "techniques and skills" learned by the information literate "for utilizing the wide range of information tools as well as primary sources in molding information solutions to their problems" and drew a relatively firm line between the "literates" and "information illiterates." The concept of information literacy appeared again in a 1976 paper by Lee Burchina presented at the Texas A&M University library's symposium. Burchina identified a set of skills needed to locate and use information for problem solving and decision making. In another 1976 article in Library Journal, M.R. Owens applied the concept to political information literacy and civic responsibility, stating, "All [people] are created equal but voters with information resources are in a position to make more intelligent decisions than citizens who are information illiterates. The application of information resources to the process of decision-making to fulfill civic responsibilities is a vital necessity." In a literature review published in an academic journal in 2020, Oral Roberts University professor Angela Sample cites several conceptual waves of information literacy definitions as defining information as a way of thinking, a set of skills, and a social practice. The introduction of these concepts led to the adoption of a mechanism called metaliteracy and the creation of threshold concepts and knowledge dispositions, which led to the creation of the ALA's Information Literacy Framework. The American Library Association's Presidential Committee on Information Literacy released a report on January 10, 1989. Titled as the Presidential Committee on Information Literacy: Final Report, the article outlines the importance of information literacy, opportunities to develop it, and the idea of an Information Age School. The recommendations of the Committee led to establishment of the National Forum on Information Literacy, a coalition of more than 90 national and international organizations. In 1998, the American Association of School Librarians and the Association for Educational Communications and Technology published Information Power: Building Partnerships for Learning, which further established specific goals for information literacy education, defining some nine standards in the categories of "information literacy," "independent learning," and "social responsibility." Also in 1998, the Presidential Committee on Information Literacy updated its final report. The report outlined six recommendations from the original report, and examined areas of challenge and progress. In 1999, the Society of College, National and University Libraries (SCONUL) in the UK published The Seven Pillars of Information Literacy to model the relationship between information skills and IT skills, and the idea of the progression of information literacy into the curriculum of higher education. In 2003, the National Forum on Information Literacy, along with UNESCO and the National Commission on Libraries and Information Science, sponsored an international conference in Prague. Representatives from twenty-three countries gathered to discuss the importance of information literacy in a global context. The resulting Prague Declaration described information literacy as a "key to social, cultural, and economic development of nations and communities, institutions and individuals in the 21st century" and declared its acquisition as "part of the basic human right of lifelong learning". In the United States specifically, information literacy was prioritized in 2009 during President Barack Obama's first term. In effort to stress the value information literacy has on everyday communication, he designated October as National Information Literacy Awareness Month in his released proclamation. In 2015, the Association of College and Research Libraries (ACRL) adopted the Framework for Information Literacy for Higher Education, which defines information literacy as "the set of integrated abilities encompassing the reflective discovery of information, the understanding of how information is produced and valued, and the use of information in creating new knowledge and participating ethically in communities of learning".Association of College and Research Libraries (2015-02-09). "Framework for Information Literacy for Higher Education". Association of College and Research Libraries. American Library Association. Retrieved 2026-02-17. == Presidential Committee on Information Literacy == The American Library Association's Presidential Committee on Information Literacy defined information literacy as the ability "to recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information" and highlighted information literacy as a skill essential for lifelong learning and the production of an informed and prosperous citizenry. The committee outlined six principal recommendations. Included were recommendations like "Reconsider the ways we have organized information institutionally, structured information access, and defined information's role in our lives at home in the community, and in the work place"; to promote "public awareness of the problems created by information illiteracy"; to develop a national research agenda related to information and its use; to ensure the existence of "a climate conducive to students' becoming information literate"; to include information literacy concerns in teacher education democracy. In the updated report, the committee ended with an invitation, asking the National Forum and regular citizens to recognize that "the result of these combined efforts will be a citizenry which is made up of effective lifelong learners who can always find the information needed for the issue or decision at hand. This new

    Read more →
  • Color clock

    Color clock

    The color clock, or color timer, is a part of the video circuitry of computer graphics hardware that works with analog color television systems. The clock is timed to match the timing of the color standard it works with, typically NTSC or PAL, ensuring that the data being read from the computer memory to create the image on-screen is in sync with the display. Depending on the speed of the color clock, the product of the resolution and number of colors is defined. Slow color clocks of many early games consoles and home computers resulted in limited color palettes at the highest resolutions.

    Read more →
  • Query rewriting

    Query rewriting

    Query rewriting is a typically automatic transformation that takes a set of database tables, views, and/or queries, usually indices, often gathered data and query statistics, and other metadata, and yields a set of different queries, which produce the same results but execute with better performance (for example, faster, or with lower memory use). Query rewriting can be based on relational algebra or an extension thereof (e.g. multiset relational algebra with sorting, aggregation and three-valued predicates i.e. NULLs as in the case of SQL). The equivalence rules of relational algebra are exploited, in other words, different query structures and orderings can be mathematically proven to yield the same result. For example, filtering on fields A and B, or cross joining R and S can be done in any order, but there can be a performance difference. Multiple operations may be combined, and operation orders may be altered. The result of query rewriting may not be at the same abstraction level or application programming interface (API) as the original set of queries (though often is). For example, the input queries may be in relational algebra or SQL, and the rewritten queries may be closer to the physical representation of the data, e.g. array operations. Query rewriting can also involve materialization of views and other subqueries; operations that may or may not be available to the API user. The query rewriting transformation can be aided by creating indices from which the optimizer can choose (some database systems create their own indexes if deemed useful), mandating the use of specific indices, creating materialized and/or denormalized views, or helping a database system gather statistics on the data and query use, as the optimality depends on patterns in data and typical query usage. Query rewriting may be rule based or optimizer based. Some sources discuss query rewriting as a distinct step prior to optimization, operating at the level of the user accessible algebra API (e.g. SQL). There are other, largely unrelated concepts also named similarly, for example, query rewriting by search engines.

    Read more →
  • Bartels–Stewart algorithm

    Bartels–Stewart algorithm

    In numerical linear algebra, the Bartels–Stewart algorithm is used to numerically solve the Sylvester matrix equation A X − X B = C {\displaystyle AX-XB=C} . Developed by R.H. Bartels and G.W. Stewart in 1971, it was the first numerically stable method that could be systematically applied to solve such equations. The algorithm works by using the real Schur decompositions of A {\displaystyle A} and B {\displaystyle B} to transform A X − X B = C {\displaystyle AX-XB=C} into a triangular system that can then be solved using forward or backward substitution. In 1979, G. Golub, C. Van Loan and S. Nash introduced an improved version of the algorithm, known as the Hessenberg–Schur algorithm. It remains a standard approach for solving Sylvester equations when X {\displaystyle X} is of small to moderate size. == The algorithm == Let X , C ∈ R m × n {\displaystyle X,C\in \mathbb {R} ^{m\times n}} , and assume that the eigenvalues of A {\displaystyle A} are distinct from the eigenvalues of B {\displaystyle B} . Then, the matrix equation A X − X B = C {\displaystyle AX-XB=C} has a unique solution. The Bartels–Stewart algorithm computes X {\displaystyle X} by applying the following steps: 1.Compute the real Schur decompositions R = U T A U , {\displaystyle R=U^{T}AU,} S = V T B T V . {\displaystyle S=V^{T}B^{T}V.} The matrices R {\displaystyle R} and S {\displaystyle S} are block-upper triangular matrices, with diagonal blocks of size 1 × 1 {\displaystyle 1\times 1} or 2 × 2 {\displaystyle 2\times 2} . 2. Set F = U T C V . {\displaystyle F=U^{T}CV.} 3. Solve the simplified system R Y − Y S T = F {\displaystyle RY-YS^{T}=F} , where Y = U T X V {\displaystyle Y=U^{T}XV} . This can be done using forward substitution on the blocks. Specifically, if s k − 1 , k = 0 {\displaystyle s_{k-1,k}=0} , then ( R − s k k I ) y k = f k + ∑ j = k + 1 n s k j y j , {\displaystyle (R-s_{kk}I)y_{k}=f_{k}+\sum _{j=k+1}^{n}s_{kj}y_{j},} where y k {\displaystyle y_{k}} is the k {\displaystyle k} th column of Y {\displaystyle Y} . When s k − 1 , k ≠ 0 {\displaystyle s_{k-1,k}\neq 0} , columns [ y k − 1 ∣ y k ] {\displaystyle [y_{k-1}\mid y_{k}]} should be concatenated and solved for simultaneously. 4. Set X = U Y V T . {\displaystyle X=UYV^{T}.} === Computational cost === Using the QR algorithm, the real Schur decompositions in step 1 require approximately 10 ( m 3 + n 3 ) {\displaystyle 10(m^{3}+n^{3})} flops, so that the overall computational cost is 10 ( m 3 + n 3 ) + 2.5 ( m n 2 + n m 2 ) {\displaystyle 10(m^{3}+n^{3})+2.5(mn^{2}+nm^{2})} . === Simplifications and special cases === In the special case where B = − A T {\displaystyle B=-A^{T}} and C {\displaystyle C} is symmetric, the solution X {\displaystyle X} will also be symmetric. This symmetry can be exploited so that Y {\displaystyle Y} is found more efficiently in step 3 of the algorithm. == The Hessenberg–Schur algorithm == The Hessenberg–Schur algorithm replaces the decomposition R = U T A U {\displaystyle R=U^{T}AU} in step 1 with the decomposition H = Q T A Q {\displaystyle H=Q^{T}AQ} , where H {\displaystyle H} is an upper-Hessenberg matrix. This leads to a system of the form H Y − Y S T = F {\displaystyle HY-YS^{T}=F} that can be solved using forward substitution. The advantage of this approach is that H = Q T A Q {\displaystyle H=Q^{T}AQ} can be found using Householder reflections at a cost of ( 5 / 3 ) m 3 {\displaystyle (5/3)m^{3}} flops, compared to the 10 m 3 {\displaystyle 10m^{3}} flops required to compute the real Schur decomposition of A {\displaystyle A} . == Software and implementation == The subroutines required for the Hessenberg-Schur variant of the Bartels–Stewart algorithm are implemented in the SLICOT library. These are used in the MATLAB control system toolbox. == Alternative approaches == For large systems, the O ( m 3 + n 3 ) {\displaystyle {\mathcal {O}}(m^{3}+n^{3})} cost of the Bartels–Stewart algorithm can be prohibitive. When A {\displaystyle A} and B {\displaystyle B} are sparse or structured, so that linear solves and matrix vector multiplies involving them are efficient, iterative algorithms can potentially perform better. These include projection-based methods, which use Krylov subspace iterations, methods based on the alternating direction implicit (ADI) iteration, and hybridizations that involve both projection and ADI. Iterative methods can also be used to directly construct low rank approximations to X {\displaystyle X} when solving A X − X B = C {\displaystyle AX-XB=C} .

    Read more →