AI Content To Human

AI Content To Human — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

Automated machine learning

Automated machine learning (AutoML) is the process of automating the tasks of applying machine learning to real-world problems. It is the combination of automation and ML. AutoML potentially includes every stage from beginning with a raw dataset to building a machine learning model ready for deployment. AutoML was proposed as an artificial intelligence-based solution to the growing challenge of applying machine learning. The high degree of automation in AutoML aims to allow non-experts to make use of machine learning models and techniques without requiring them to become experts in machine learning. Automating the process of applying machine learning end-to-end additionally offers the advantages of producing simpler solutions, faster creation of those solutions, and models that often outperform hand-designed models. Common techniques used in AutoML include hyperparameter optimization, meta-learning and neural architecture search. == Comparison to the standard approach == In a typical machine learning application, practitioners have a set of input data points to be used for training. The raw data may not be in a form that all algorithms can be applied to. To make the data amenable for machine learning, an expert may have to apply appropriate data pre-processing, feature engineering, feature extraction, and feature selection methods. After these steps, practitioners must then perform algorithm selection and hyperparameter optimization to maximize the predictive performance of their model. If deep learning is used, the architecture of the neural network must also be chosen manually by the machine learning expert. Each of these steps may be challenging, resulting in significant hurdles to using machine learning. AutoML aims to simplify these steps for non-experts, and to make it easier for them to use machine learning techniques correctly and effectively. AutoML plays an important role within the broader approach of automating data science, which also includes challenging tasks such as data engineering, data exploration and model interpretation and prediction. == Targets of automation == Automated machine learning can target various stages of the machine learning process. Steps to automate are: Data preparation and ingestion (from raw data and miscellaneous formats) Column type detection; e.g., Boolean, discrete numerical, continuous numerical, or text Column intent detection; e.g., target/label, stratification field, numerical feature, categorical text feature, or free text feature Task detection; e.g., binary classification, regression, clustering, or ranking Feature engineering Feature selection Feature extraction Meta-learning and transfer learning Detection and handling of skewed data and/or missing values Model selection - choosing which machine learning algorithm to use, often including multiple competing software implementations Ensembling - a form of consensus where using multiple models often gives better results than any single model Hyperparameter optimization of the learning algorithm and featurization Neural architecture search Pipeline selection under time, memory, and complexity constraints Selection of evaluation metrics and validation procedures Problem checking Leakage detection Misconfiguration detection Analysis of obtained results Creating user interfaces and visualizations == Challenges and Limitations == There are a number of key challenges being tackled around automated machine learning. A big issue surrounding the field is referred to as "development as a cottage industry". This phrase refers to the issue in machine learning where development relies on manual decisions and biases of experts. This is contrasted to the goal of machine learning which is to create systems that can learn and improve from their own usage and analysis of the data. Basically, it's the struggle between how much experts should get involved in the learning of the systems versus how much freedom they should be giving the machines. However, experts and developers must help create and guide these machines to prepare them for their own learning. To create this system, it requires labor intensive work with knowledge of machine learning algorithms and system design. Additionally, other challenges include meta-learning and computational resource allocation.
Read more →
Medical data breach

Medical data, including patients' identity information, health status, disease diagnosis and treatment, and biogenetic information, not only involve patients' privacy but also have a special sensitivity and important value, which may bring physical and mental distress and property loss to patients and even negatively affect social stability and national security once leaked. However, the development and application of medical AI must rely on a large amount of medical data for algorithm training, and the larger and more diverse the amount of data, the more accurate the results of its analysis and prediction will be. However, the application of big data technologies such as data collection, analysis and processing, cloud storage, and information sharing has increased the risk of data leakage. In the United States, the rate of such breaches has increased over time, with 176 million records breached by the end of 2017. By 2024, the U.S. Department of Health and Human Services reported 725 large healthcare data breaches affecting approximately 275 million individual records in a single year, marking a significant escalation in both the frequency and scale of incidents. == Black market for health data == In February 2015 an NPR report claimed that organized crime networks had ways of selling health data in the black market. In 2015 a Beazley employee estimated that medical records could sell on the black market for US$40-50. == How data is lost == Theft, data loss, hacking, and unauthorized account access are ways in which medical data breaches happen. Among reported breaches of medical information in the United States networked information systems accounted for the largest number of records breached. There are many data breaches happening in the US health care system, among business associates of the health care providers that continuously gain access to patients' data. == List of data breaches == In February 2024, a ransomware attack on Change Healthcare, a subsidiary of UnitedHealth Group, compromised the protected health information of approximately 100 million individuals, making it the largest healthcare data breach in United States history. The attack disrupted claims processing for healthcare providers nationwide for several weeks. In May 2024, MediSecure suffered a cyberattack involving ransomware in Australia. In May 2021, the Health Service Executive in the Republic of Ireland was the victim of a cyberattack involving ransomware, in the Health Service Executive cyberattack, with admission records and test results present in a sample of the data reviewed by the Financial Times. In October 2018, the Centers for Medicare and Medicaid Services in the US reported that around 75,000 individual records had been affected by a data breach that took place through the ACA Agent and Broker Portal. In 2018, Social Indicators Research published the scientific evidence of 173,398,820 (over 173 million) individuals affected in USA from October 2008 (when the data were collected) to September 2017 (when the statistical analysis took place). In 2015, Anthem Inc. lost data for 37 million people in the Anthem medical data breach In 2014 4.5 million people using Complete Health Systems had their data stolen In 2013-14 1 million people using Montana Department of Public Health and Human Services had their data stolen In 2013 4 million people using Advocate Health and Hospitals Corporation had their data stolen In 2011 4.9 million users of Tricare services had their data stolen due to an employee error by Science Applications International Corporation In 2011 1.9 million people using Health Net had their data stolen In 2011 1 million people using Nemours Foundation had their data stolen In 2010 6800 people using New York-Presbyterian Hospital and Columbia University Medical Center had their data breached. In response, those organizations agreed to pay the United States Department of Health and Human Services a US$4.8 million dollar fine. In 2009 1 million people using BlueCross BlueShield of Tennessee had their data stolen == Regulation == In the United States, the Health Insurance Portability and Accountability Act and Health Information Technology for Economic and Clinical Health Act require companies to report data breaches to affected individuals and the federal government. Under the HIPAA Breach Notification Rule, covered entities must notify affected individuals without unreasonable delay and no later than 60 days after discovering a breach of unsecured protected health information. Breaches affecting 500 or more individuals must also be reported to the HHS Secretary and to prominent media outlets serving the affected state or jurisdiction within the same timeframe; HHS publicly lists these larger breaches on its breach portal, commonly known as the "wall of shame." Breaches affecting fewer than 500 individuals are reported to HHS annually, no later than 60 days after the end of the calendar year in which they were discovered. Health Information Privacy Health Insurance Portability and Accountability Act of 1996 (HIPAA). - 45 CFR Parts 160 and 164, Standards for Privacy of Individually Identifiable Health Information and Security Standards for the Protection of Electronic Protected Health Information. HIPAA includes provisions designed to save health care businesses money by encouraging electronic transactions, as well as regulations to protect the security and confidentiality of patient information. The Privacy Rule became effective April 14, 2001, and most covered entities (health plans, health care clearinghouses, and health care providers that conduct certain financial and administrative transactions electronically) had until April 2003 to comply. This security provision became effective April 21, 2003. The Health Insurance Portability and Accountability Act (HIPAA) is the baseline set of federal regulations governing medical information. It does three things: i. i. i.Establish a structure for how personal health information is disclosed and establish the rights of individuals with respect to health information; ii.Specify security standards for the retention and transmission of electronic patient information; iii.Need a common format and data structure for the electronic exchange of health information. California-Specific Laws California’s medical privacy laws, primarily the Confidentiality of Medical Information Act (CMIA), the data breach sections of the Civil Code, and sections of the Health and Safety Code, provide HIPAA-like protections, although the terminology is different. HIPAA establishes a federal "minimum standard" that applies where there are gaps in California law, and HIPAA also specifies that stricter state laws will override or supersede HIPAA. California's health care privacy laws apply to providers who provide personal health records (PHR), while HIPAA only applies when the provider providing the PHR is a business associate of a covered entity. Federal law does not grant individuals the right to file a lawsuit in the event of a data breach (only the Attorney General can file a lawsuit), but California law does. This means that California law sets a higher standard for medical privacy, and that individuals in California enjoy stronger legal protections and more ways to hold entities that violate their medical privacy accountable. In the UK, the legal framework for how patient data is cared for and processed is the Data Protection Act 2018 (DPA), which incorporates the EU General Data Protection Regulation (GDPR) into law, and the common law duty of confidentiality (CLDC). The data protection legislation requires that the collection and processing of personal data be fair, lawful and transparent. This means that the collection and processing of data as defined by data protection legislation must always have a valid lawful basis and must also meet the requirements of the CLDC. In the China, Article 18 of the "National Health Care Big Data Standards, Security and Services Management Measures (for Trial Implementation)" (National Health Planning and Development (2018) No. 23) promulgated by the National Health Care Commission in 2018 states, "The responsible unit shall adopt measures such as data classification, important data backup, and encryption authentication to guarantee the security of health care big data." However, the scope and definition of important data are not covered. Although the "Information Security Technology-Healthcare Data Security Guide" (the "Guide") issued by the National Standardization Committee also proposes that important data should be evaluated and approved in accordance with the regulations, there is likewise no definition of the connotation and definition of important data.
Read more →
Anderson's rule (computer science)

In the field of computer security, Anderson's rule refers to a principle formulated by Ross J. Anderson: systems that handle sensitive personal information involve a trilemma of security, functionality, and scale, of which you can choose any two. A system that has information on many data subjects and to which many people require access is hard to secure unless its functionality is severely restricted. If it has rich functionality, you may have to restrict the number of people with access, or accept that some information will leak.
Read more →
Blitter object

A blitter object (Bob) is a graphical element (GEL) used by the Amiga computer. Bobs are hardware sprite-like objects, movable on the screen with the help of the blitter coprocessor. == Overview == The AmigaOS GEL system consists of VSprites, Bobs, AnimComps (animation components) and AnimObs (animation objects), each extending the preceding with additional functionality. While VSprites are a virtualization of hardware sprites Bobs are drawn into a playfield by the blitter, saving and restoring the background of the GEL as required. The Bob with the highest video priority is the last one to be drawn, which makes it appear to be in front of all other Bobs. In contrast to hardware sprites Bobs are not limited in size and number. Bobs require more processing power than sprites, because they require at least one DMA memory copy operation to draw them on the screen. Sometimes three distinct memory copy operations are needed: one to save the screen area where the Bob would be drawn, one to actually draw the Bob, and one later to restore the screen background when the Bob moves away. An AnimComp adds animation to a Bob and an AnimOb groups AnimComps together and assigns them velocity and acceleration.
Read more →
Tensor glyph

In scientific visualization a tensor glyph is an object that can visualize all or most of the nine degrees of freedom, such as acceleration, twist, or shear – of a 3 × 3 {\displaystyle 3\times 3} matrix. It is used for tensor field visualization, where a data-matrix is available at every point in the grid. "Glyphs, or icons, depict multiple data values by mapping them onto the shape, size, orientation, and surface appearance of a base geometric primitive." Tensor glyphs are a particular case of multivariate data glyphs. There are certain types of glyphs that are commonly used: Ellipsoid Cuboid Cylindrical Superquadrics According to Thomas Schultz and Gordon Kindlmann, specific types of tensor fields "play a central role in scientific and biomedical studies as well as in image analysis and feature-extraction methods."
Read more →
CityEngine

ArcGIS CityEngine is a commercial 3D modeling program. Developed by Esri R&D Center Zurich (formerly Procedural Inc.), it specializes in the generation of 3D urban environments to support the creation of detailed large-scale 3D city models. Unlike traditional 3D modeling methodology, which uses computer-aided design (CAD) tools and techniques, CityEngine takes a procedural modeling approach which shapes generation via a rules-based system. Due to its integration with the wider ArcGIS platform, CityEngine can also be used with geographic information system (GIS) datasets. CityEngine can be used for urban planning and architecture, graphics visualization, game development, entertainment, and archeology. CityEngine can be used to visualize the building information modeling (BIM) data of buildings in a larger urban context, making for more realistic construction projects. == History and releases == === Software history === ArcGIS CityEngine, originally named Esri CityEngine, was developed at Swiss technology university ETH Zurich by Pascal Mueller, the co-founder and CEO of Procedural Inc. While researching for his PhD at the ETH Computer Vision Lab, Mueller invented a number of techniques for procedural modeling of 3D architecture that make up the foundation of CityEngine. CityEngine publically debuted at the 2001 SIGGRAPH conference; since then, additional research papers have been published that have contributed to CityEngine and its features. The first commercial version of CityEngine was released in 2008. In 2007, Procedural Inc. was founded and separated from ETH Zurich, the top-ranking technology university in Switzerland. In the summer of 2011, Procedural Inc. was acquired by Esri Inc., becoming Esri R&D Center Zurich. Esri CityEngine was renamed to ArcGIS CityEngine in June 2020 to offically make it a part of the ArcGIS software suite. === Releases === === Licensing and pricing === ArcGIS CityEngine is included in the Professional and Professional Plus tiers of ArcGIS Online. Pricing may vary by region and distributors. In the US, the professional tier costs US$2,200 per year; in the UK, it is £4,200 per year (excluding VAT). CityEngine can be purchased elsewhere via a local Esri partner. . Once purchased, users can download and obtain license details from the MyEsri portal. == Features == CGA (computer generated architecture) parametric modeling rules to control mass, geometry assets, proportions, or texturing of buildings or streets on a citywide scale Select a target location and import geo-referenced satellite imagery and 3D terrain of the location to more quickly build accurate urban environments through OpenStreetMap integration Interactively control specific street or building parameters, such as height or age Import/export geo-spatial/vector data with industry-standard formats such as Esri Shapefile, File Geodatabase, and OpenStreetMap, as well as file formats for WebGL, KMZ, Collada, Autodesk FBX, Autodesk Maya, 3DS, Wavefront OBJ, RenderMan RIB, Alembic, e-on software's Vue, Universal Scene Description USD, Khronos Group GLTF, Unreal Engine, and Unreal Datasmith Script and generate rules-based reports to show socioeconomic figures (e.g., Gross Floor Area (GFA) and Floor Area Ratio (FAR)) to analyze their urban design proposals. VR viewing of modeled environments with Samsung Gear VR Use a variety of materials through the Esri materials library == Procedural modeling == ArcGIS CityEngine uses a procedural modeling approach to automatically generate models through a predefined rule set. The rules are defined through a CGA shape grammar system, enabling the creation of complex parametric models. Users can change or add the shape grammar as needed. Urban environments can be modeled within CityEngine by starting with creating a street network (either from the street drawing tool or with data imported from map data). Then, lots may be subdivided as many times as specified, resulting in a map of multiple lots and streets. CityEngine can then be instructed to start generating the buildings using defined procedural modeling rules. At this point, the city model can be re-designed and adjusted by changing the parameters or the shape grammar. === Geodesign === Though CityEngine is not an analytical tool like GIS, discussions about geodesign often mention the use of ArcGIS CityEngine. As it can be used to enhance 3D shape generation in ArcGIS, ArcGIS CityEngine is a critical product to improve the applicability of geodesign by using geospatial information to design or analyze a city. == Applications == === Urban design and planning === Garsdale Design used ArcGIS CityEngine in the creation of city master plans in Iraq before 2013, both to model existing historic areas and also model future plans. Larger companies like Foster+Partners and HOK Architects have also used CityEngine in their urban planning projects. === Urban and environmental studies === Because its primary feature is building informative city models, some urban researchers use CityEngine to compare land-use planning schemes, for example in very dense global cities such as Hong Kong and Seoul. Environmental scientists can also utilize the instant 3D model generation in CityEngine, which can make for more convenient informative research than modeling a city by creating each building individually. === Game development === CityEngine can be used as a tool in the creation of video games that require detailed 3D environments to assign interactive scripts. === Movie industry === Zootopia (also known outside of the US as Zootopolis), which won the 2016 Academy Award for Best Animated Feature Film, used CityEngine to model the city in its movie. multi-scaling city, the designers used CityEngine due to its rule-based system. CityEngine was also used to create Big Hero 6's San-Fransokyo. === Military === Due to its integration with the Esri product suite and its ability to process geospatial data to create 3D scenes/maps, CityEngine can be used within military/defense organizations. == List of movies and TV shows using CityEngine == Studios and companies rarely state what software they use in their pipelines. When CityEngine is mentioned as a tool in production, it's often in a small reference in a larger article. Movies only claimed to use CityEngine by a single Esri employee Presented at FMX 2025 workshop == Ports == ArcGIS CityEngine is built on top of Eclipse IDE, and has therefore able to be used on Windows and Linux operating systems. Support for macOS was stopped in March 2021. == Plugins and extensions == ArcGIS CityEngine currently works with a number of third party 3D modeling, rendering, and analytical software products via its SDK and API; these currently are: ArcGIS CityEngine for ArcGIS Urban: ArcGIS Urban Suite Puma: ArcGIS CityEngine for Rhinoceros 3D Palladio: ArcGIS CityEngine for Houdini Serlio: ArcGIS CityEngine for Maya PyPRT: ArcGIS CityEngine for Python ArcGIS CityEngine provides a Python scripting interface built on Jython (current version 2.7.0) which allows users to create their own tools and functionality. == Publications ==
Read more →
Computer Graphics International

Computer Graphics International (CGI) is one of the oldest annual international conferences on computer graphics. It is organized by the Computer Graphics Society (CGS). Researchers across the whole world are invited to share their experiences and novel achievements in various fields - like computer graphics and human-computer interaction. Former conferences have been held recently in Hong Kong (China), Geneva (Switzerland), Shanghai (China), Geneva (virtually), Calgary (Canada), Bintan (Indonesia) and Yokohama (Japan). == Awards == Starting in the year of 2013, CGI has given yearly a Best Paper Award and a Career Achievement Award. == Venues ==
Read more →
Index locking

In databases an index is a data structure, part of the database, used by a database system to efficiently navigate access to user data. Index data are system data distinct from user data, and consist primarily of pointers. Changes in a database (by insert, delete, or modify operations), may require indexes to be updated to maintain accurate user data accesses. Index locking is a technique used to maintain index integrity. A portion of an index is locked during a database transaction when this portion is being accessed by the transaction as a result of attempt to access related user data. Additionally, special database system transactions (not user-invoked transactions) may be invoked to maintain and modify an index, as part of a system's self-maintenance activities. When a portion of an index is locked by a transaction, other transactions may be blocked from accessing this index portion (blocked from modifying, and even from reading it, depending on lock type and needed operation). Index Locking Protocol guarantees that phantom read phenomenon won't occur. Index locking protocol states: Every relation must have at least one index. A transaction can access tuples only after finding them through one or more indices on the relation A transaction Ti that performs a lookup must lock all the index leaf nodes that it accesses, in S-mode, even if the leaf node does not contain any tuple satisfying the index lookup (e.g. for a range query, no tuple in a leaf is in the range) A transaction Ti that inserts, updates or deletes a tuple ti in a relation r must update all indices to r and it must obtain exclusive locks on all index leaf nodes affected by the insert/update/delete The rules of the two-phase locking protocol must be observed. Specialized concurrency control techniques exist for accessing indexes. These techniques depend on the index type, and take advantage of its structure. They are typically much more effective than applying to indexes common concurrency control methods applied to user data. Notable and widely researched are specialized techniques for B-trees (B-Tree concurrency control) which are regularly used as database indexes. Index locks are used to coordinate threads accessing indexes concurrently, and typically shorter-lived than the common transaction locks on user data. In professional literature, they are often called latches.
Read more →
TiDB

TiDB (; "Ti" stands for Titanium) is an open-source NewSQL database that supports Hybrid Transactional and Analytical Processing (HTAP) workloads. Designed to be MySQL compatible, it is developed and supported primarily by PingCAP and licensed under Apache 2.0. It is also available as a paid product. TiDB drew its initial design inspiration from Google's Spanner and F1 papers. == Release history == See all TiDB release notes. On December 19, 2024, TiDB 8.5 GA was released. On May 24, 2024, TiDB 8.1 GA was released. On December 1, 2023, TiDB 7.5 GA was released. On May 31, 2023, TiDB 7.1 GA was released. On April 7, 2022, TiDB 6.0 GA was released. On April 7, 2021 TiDB 5.0 GA was released. On May 28, 2020, TiDB 4.0 GA was released. On June 28, 2019, TiDB 3.0 GA was released. On April 27, 2018, TiDB 2.0 GA was released. On October 16, 2017, TiDB 1.0 GA was released. == Main features == === Horizontal scalability === TiDB can expand both SQL processing and storage capacity by adding new nodes. === MySQL compatibility === TiDB acts like it is a MySQL 8.0 server to applications. A user can continue to use all of the existing MySQL client libraries. Because TiDB's SQL processing layer is built from scratch, it is not a MySQL fork. === Distributed transactions with strong consistency === TiDB internally shards a table into small range-based chunks that are referred to as "Regions". Each Region defaults to approximately 100 MB in size, and TiDB uses a two-phase commit internally to ensure that regions are maintained in a transactionally consistent way. === Cloud native === TiDB is designed to work in the cloud. The storage layer of TiDB, called TiKV, became a Cloud Native Computing Foundation (CNCF) member project in August 2018, as a Sandbox level project, and became an incubation-level hosted project in May 2019. TiKV graduated from CNCF in September 2020. === Real-time HTAP === TiDB can support both online transaction processing (OLTP) and online analytical processing (OLAP) workloads. TiDB has two storage engines: TiKV, a rowstore, and TiFlash, a columnstore. === High availability === TiDB uses the Raft consensus algorithm to ensure that data is available and replicated throughout storage in Raft groups. In the event of failure, a Raft group will automatically elect a new leader for the failed member, and self-heal the TiDB cluster. === Vector Search === TiDB has a vector data type and vector indexes. This allows TiDB to be used as Vector database in AI Retrieval-augmented generation applications. == Deployment methods == === Kubernetes with Operator === TiDB can be deployed in a Kubernetes-enabled cloud environment by using TiDB Operator. An Operator is a method of packaging, deploying, and managing a Kubernetes application. It is designed for running stateful workloads and was first introduced by CoreOS in 2016. TiDB Operator was originally developed by PingCAP and open-sourced in August, 2018. TiDB Operator can be used to deploy TiDB on a laptop, Google Cloud Platform’s Google Kubernetes Engine, and Amazon Web Services’ Elastic Container Service for Kubernetes. === TiUP === TiDB 4.0 introduces TiUP, a cluster operation and maintenance tool. It helps users quickly install and configure a TiDB cluster with a few commands. == Tools == TiDB has a series of open-source tools built around it to help with data replication and migration for existing MySQL and MariaDB users. === TiDB Data Migration (DM) === TiDB Data Migration (DM) is suited for replicating data from already sharded MySQL or MariaDB tables to TiDB. A common use case of DM is to connect MySQL or MariaDB tables to TiDB, treating TiDB almost as a slave, then directly run analytical workloads on this TiDB cluster in near real-time. === Backup & Restore === Backup & Restore (BR) is a distributed backup and restore tool for TiDB cluster data. === Dumpling === Dumpling is a data export tool that exports data stored in TiDB or MySQL. It lets users make logical full backups or full dumps from TiDB or MySQL. === TiDB Lightning === TiDB Lightning is a tool that supports high speed full-import of a large MySQL dump into a new TiDB cluster. This tool is used to populate an initially empty TiDB cluster with much data, in order to speed up testing or production migration. The import speed improvement is achieved by parsing SQL statements into key-value pairs, then directly generate Sorted String Table (SST) files to RocksDB. === TiCDC === TiCDC is a change data capture tool which streams data from TiDB to other systems like Apache Kafka.
Read more →
Whitelist

A whitelist or allowlist is a list or register of entities that are being provided a particular privilege, service, mobility, access or recognition. Entities on the list will be accepted, approved and/or recognized. Whitelisting is the reverse of blacklisting, the practice of identifying entities that are denied, unrecognized, or ostracized. == Email whitelists == Spam filters often include the ability to "whitelist" certain sender IP addresses, email addresses or domain names to protect their email from being rejected or sent to a junk mail folder. These can be manually maintained by the user or system administrator - but can also refer to externally maintained whitelist services. === Non-commercial whitelists === Non-commercial whitelists are operated by various non-profit organizations, ISPs, and others interested in blocking spam. Rather than paying fees, the sender must pass a series of tests; for example, their email server must not be an open relay and have a static IP address. The operator of the whitelist may remove a server from the list if complaints are received. === Commercial whitelists === Commercial whitelists are a system by which an Internet service provider allows someone to bypass spam filters when sending email messages to its subscribers, in return for a pre-paid fee, either an annual or a per-message fee. A sender can then be more confident that their messages have reached recipients without being blocked, or having links or images stripped out of them, by spam filters. The purpose of commercial whitelists is to allow companies to reliably reach their customers by email. == Advertising whitelist == Many websites rely on ads as a source of revenue, but the use of ad blockers is increasingly common. Websites that detect an adblocker in use often ask for it to be disabled - or their site to be "added to the whitelist" - a standard feature of most adblockers. == Network whitelists == === LAN whitelists === A use for whitelists is in local area network (LAN) security. Many network admins set up MAC address whitelists, or a MAC address filter, to control who is allowed on their networks. This is used when encryption is not a practical solution or in tandem with encryption. However, it's sometimes ineffective because a MAC address can be faked. === IP whitelist === Firewalls can usually be configured to only allow data-traffic from/to certain (ranges of) IP-addresses. === Application whitelists === One approach in combating viruses and malware is to whitelist software which is considered safe to run, blocking all others. This is particularly attractive in a corporate environment, where there are typically already restrictions on what software is approved. Leading providers of application whitelisting technology include Bit9, Velox, McAfee, Lumension, ThreatLocker, Airlock Digital and SMAC. On Microsoft Windows, recent versions include AppLocker, which allows administrators to control which executable files are denied or allowed to execute. With AppLocker, administrators are able to create rules based on file names, publishers or file location that will allow certain files to execute. Rules can apply to individuals or groups. Policies are used to group users into different enforcement levels. For example, some users can be added to a report-only policy that will allow administrators to understand the impact before moving that user to a higher enforcement level. Linux systems typically have AppArmor and SE Linux features available which can be used to effectively block all applications which are not explicitly whitelisted, and commercial products are also available. On HP-UX introduced a feature called "HP-UX Whitelisting" on 11iv3 version. == Controversy regarding name == In 2018, a journal commentary on a report on predatory publishing was released making claims that "white" and "black" are racially charged terms that need to be avoided in instances such as "whitelist" and "blacklist". The premise of the journal is that "black" and "white" have negative and positive connotations respectively. It states that since "blacklisting" was first referred to during "the time of mass enslavement and forced deportation of Africans to work in European-held colonies in the Americas," the word is therefore related to race. There is no mention of "whitelist" and its origin or relation to race. This issue is most widely disputed in computing industries where "whitelist" and "blacklist" are prevalent (e.g. IP whitelisting). Despite the commentary nature of the journal, some companies and individuals in others have taken to replacing "whitelist" and "blacklist" with new alternatives such as "allow list" and "deny list". Those adopting this change consider using the "whitelist"/"blacklist" names as a code smell. Those that oppose these changes question its attribution to race, citing the same etymology quote that the 2018 journal uses. According to the remark, the term "blacklist" evolved from the term "black book" about a century ago. The term "black book" does not appear to have any etymology or sources that support racial associations, instead originating in the 1400s as a reference to "a list of people who had committed crimes or fallen out of favor with leaders", and popularized by King Henry VIII's literal use of a black book. Others also note the prevalence of positive and negative connotations to "white" and "black" in the Bible, predating attributions to skin tone and slavery. It wasn't until the 1960s Black Power movement that "Black" became a widespread word to refer to one's race as a person of color in America (alternate to African-American) lending itself to the argument that the negative connotation behind "black" and "blacklist" both predate attribution to race.
Read more →
T-pose

In computer animation, a T-pose is a default posing for a humanoid 3D model's skeleton before it is animated. It is called so because of its shape: the straight legs and arms of a humanoid model combine to form a capital letter T. When the arms are angled downwards, the pose is sometimes referred to as an A-pose instead. Likewise, if the arms are angled upward, it is called a Y-pose. Generic terms encompassing all these (especially for non-humanoid models) include bind pose, blind pose, and reference pose. == Usage == The T-pose is primarily used as the default armature pose for skeletal animation in 3D software, which is then manipulated to create animation. The purpose of the T-pose relates to the important elements of the body being axis-aligned, thereby making it easier to rig the model for animation, physics, and other controls. Depending on the exact geometry of the model, other poses such as the A-pose may be more suitable for vertex deformation around areas such as the shoulders. Outside of being default poses in animation software, T-poses are typically used as placeholders for animation not yet completed, particularly in 3D animated video games. In some motion capture software, a T-pose must be assumed by the actor in the motion capture suit before motion capturing can begin. There are other poses used, but the T-pose is the most common one. == As an Internet meme == Starting in 2016 and resurfacing in 2017, the T-pose has become a widespread Internet meme due to its bizarre and somewhat comedic appearance, especially in video game glitches where a character's animation is unexpectedly supplanted by a T-pose. In a prerelease video of the game NBA Elite 11, the demo was filled with glitches, notably one unintentionally showing a T-pose in place of the proper animation for the model of player Andrew Bynum. The glitch later gained fame as the "Jesus Bynum glitch". Publisher EA eventually cancelled the game as they found it unsatisfactory. A similar occurrence happened with Cyberpunk 2077. In the 2023 Formula One season, driver George Russell performed a T-pose in the opening credits of the series' TV broadcasts. This quickly became a meme within the motorsports community. Russell repeated the pose after claiming pole position at the 2024 Canadian Grand Prix and winning the 2024 Austrian Grand Prix.
Read more →
Digital curation

Digital curation is the selection, preservation, maintenance, collection, and archiving of digital assets. It is a process that establishes, maintains, and adds value to repositories of digital data for present and future use. The implementation of digital curation is often carried out by archivists, librarians, scientists, historians, and scholars to ensure users have access to reliable, high-quality resources. Enterprises are also starting to adopt digital curation as a means to improve the quality of information and data within their operational and strategic processes. A successful digital curation initiative will help to mitigate digital obsolescence, keeping the information accessible to users indefinitely. Digital curation includes various aspects, including digital asset management, data curation, digital preservation, and electronic records management. == Word History == Much like the word archive has layered meanings and uses, the word curation is both a noun and a verb, used originally in the field of museology to represent a wide range of activities, most often associated with collection care, long-term preservation, and exhibition design. Curation can be a reference to physical repositories that store cultural heritage or natural resource collections (e.g., a curatorial repository) or a representation of varied policies and processes involved with the long-term care and management of heritage collections, digital archives, and research data (e.g, curatorial/collections management plans, curation life-cycle, and data curation). Yet curation is also associated with short-term objectives and processes of selection and interpretation for the purposes of presentation, such as for gallery exhibitions and websites, which contribute to knowledge creation. It has also been applied to interaction with social media including compiling digital images, web links, and movie files. The term curation entered the legal framework through federal historic preservation laws, starting with the National Historic Preservation Act of 1966, and was further defined and coded into federal regulations through 36 CFR Part 79: Curation of Federally-owned and Administered Archaeological Collections. Curation has since permeated into an array of disciplines but remains closely tied to heritage and information management. == Core Principles and Activities == The term "digital curation" was first used in the e-science and biological science fields as a means of differentiating the additional suite of activities ordinarily employed by library and museum curators to add value to their collections and enable its reuse from the smaller subtask of simply preserving the data, a significantly more concise archival task. Additionally, the historical understanding of the term "curator" demands more than simple care of the collection. A curator is expected to command academic mastery of the subject matter as a requisite part of appraisal and selection of assets and any subsequent adding of value to the collection through application of metadata. === Principles === There are five commonly accepted principles that govern the occupation of digital curation: Manage the complete birth-to-retirement life cycle of the digital asset. Evaluate and cull assets for inclusion in the collection. Apply preservation methods to strengthen the asset’s integrity and reusability for future users. Act proactively throughout the asset life cycle to add value to both the digital asset and the collection. Facilitate the appropriate degree of access to users. === Methodology === The Digital Curation Center offers the following step-by-step life cycle procedures for putting the above principles into practice: Sequential Actions: Conceptualize: Consider what digital material you will be creating and develop storage options. Take into account websites, publications, email, among other types of digital output. Create: Produce digital material and attach all relevant metadata, typically the more metadata the more accessible the information. Appraise and select: Consult the mission statement of the institution or private collection and determine what digital data is relevant. There may also be legal guidelines in place that will guide the decision process for a particular collection. Ingest: Send digital material to the predetermined storage solution. This may be an archive, repository or other facility. Preservation action: Employ measures to maintain the integrity of the digital material. Store: Secure data within the predetermined storage facility. Access, use, and reuse: Determine the level of accessibility for the range of digital material created. Some material may be accessible only by password and other material may be freely accessible to the public. Routinely check that material is still accessible for the intended audience and that the material has not been compromised through multiple uses. Transform: If desirable or necessary the material may be transferred into a different digital format. Occasional Actions: Dispose: Discard any digital material that is not deemed necessary to the institution. Reappraise: Reevaluate material to ensure that is it still relevant and is true to its original form. Migrate: Migrate data to another format in order to protect data for using better in the future. == Related terms == The term "digital curation" is sometimes used interchangeably with terms such as "digital preservation" and "digital archiving." While digital preservation does focus a significant degree of energy on optimizing reusability, preservation remains a subtask to the concept of digital archiving, which is in turn a subtask of digital curation. For example, archiving is a part of curation, but so are subsequent tasks such as themed collection-building, which is not considered an archival task. Similarly, preservation is a part of archiving, as are the tasks of selection and appraisal that are not necessarily part of preservation. Data curation is another term that is often used interchangeably with digital curation, however common usage of the two terms differs. While "data" is a more all-encompassing term that can be used generally to indicate anything recorded in binary form, the term "data curation" is most common in scientific parlance and usually refers to accumulating and managing information relative to the process of research. Data-driven research of education request the role of information professional gradually develop tradition of digital service to data curation particularly at the management of digital research data. So, while documents and other discrete digital assets are technically a subset of the broader concept of data, in the context of scientific vernacular digital curation represents a broader purview of responsibilities than data curation due to its interest in preserving and adding value to digital assets of any kind. == Challenges == === Rate of creation of new data and data sets === The ever lowering cost and increasing prevalence of entirely new categories of technology has led to a quickly growing flow of new data sets. These come from well established sources such as business and government, but the trend is also driven by new styles of sensors becoming embedded in more areas of modern life. This is particularly true of consumers, whose production of digital assets is no longer relegated strictly to work. Consumers now create wider ranges of digital assets, including videos, photos, location data, purchases, and fitness tracking data, just to name a few, and share them in wider ranges of social platforms. Additionally, the advance of technology has introduced new ways of working with data. Some examples of this are international partnerships that leverage astronomical data to create "virtual observatories," and similar partnerships have also leveraged data resulting from research at the Large Hadron Collider at CERN and the database of protein structures at the Protein Data Bank. === Storage format evolution and obsolescence === By comparison, archiving of analog assets is notably passive in nature, often limited to simply ensuring a suitable storage environment. Digital preservation requires a more proactive approach. Today’s artifacts of cultural significance are notably transient in nature and prone to obsolescence when social trends or dependent technologies change. This rapid progression of technology occasionally makes it necessary to migrate digital asset holdings from one file format to another in order to mitigate the dangers of hardware and software obsolescence which would render the asset unusable. === Underestimation of human labor costs === Modern tools for program planning often underestimate the amount of human labor costs required for adequate digital curation of large collections. As a result cost-benefit assessments often paint an inaccurate picture of both the amount of work involved and the true cost to the institution for bot
Read more →
Apache Pig

Apache Pig is a high-level platform for creating programs that run on Apache Hadoop. The language for this platform is called Pig Latin. Pig can execute its Hadoop jobs in MapReduce, Apache Tez, or Apache Spark. Pig Latin abstracts the programming from the Java MapReduce idiom into a notation which makes MapReduce programming high level, similar to that of SQL for relational database management systems. Pig Latin can be extended using user-defined functions (UDFs) which the user can write in Java, Python, JavaScript, Ruby or Groovy and then call directly from the language. == History == Apache Pig was originally developed at Yahoo Research around 2006 for researchers to have an ad hoc way of creating and executing MapReduce jobs on very large data sets. In 2007, it was moved into the Apache Software Foundation. === Naming === Regarding the naming of the Pig programming language, the name was chosen arbitrarily and stuck because it was memorable, easy to spell, and for novelty. The story goes that the researchers working on the project initially referred to it simply as 'the language'. Eventually they needed to call it something. Off the top of his head, one researcher suggested Pig, and the name stuck. It is quirky yet memorable and easy to spell. While some have hinted that the name sounds coy or silly, it has provided us with an entertaining nomenclature, such as Pig Latin for the language, Grunt for the shell, and PiggyBank for the CPAN-like shared repository. == Example == Below is an example of a "Word Count" program in Pig Latin: The above program will generate parallel executable tasks which can be distributed across multiple machines in a Hadoop cluster to count the number of words in a dataset such as all the webpages on the internet. == Pig vs SQL == In comparison to SQL, Pig has a nested relational model, uses lazy evaluation, uses extract, transform, load (ETL), is able to store data at any point during a pipeline, declares execution plans, supports pipeline splits, thus allowing workflows to proceed along DAGs instead of strictly sequential pipelines. On the other hand, it has been argued DBMSs are substantially faster than the MapReduce system once the data is loaded, but that loading the data takes considerably longer in the database systems. It has also been argued RDBMSs offer out of the box support for column-storage, working with compressed data, indexes for efficient random data access, and transaction-level fault tolerance. Pig Latin is procedural and fits very naturally in the pipeline paradigm while SQL is instead declarative. In SQL users can specify that data from two tables must be joined, but not what join implementation to use (You can specify the implementation of JOIN in SQL, thus "... for many SQL applications the query writer may not have enough knowledge of the data or enough expertise to specify an appropriate join algorithm."). Pig Latin allows users to specify an implementation or aspects of an implementation to be used in executing a script in several ways. In effect, Pig Latin programming is similar to specifying a query execution plan, making it easier for programmers to explicitly control the flow of their data processing task. SQL is oriented around queries that produce a single result. SQL handles trees naturally, but has no built in mechanism for splitting a data processing stream and applying different operators to each sub-stream. Pig Latin script describes a directed acyclic graph (DAG) rather than a pipeline. Pig Latin's ability to include user code at any point in the pipeline is useful for pipeline development. If SQL is used, data must first be imported into the database, and then the cleansing and transformation process can begin.
Read more →
Czekanowski distance

The Czekanowski distance (sometimes shortened as CZD) is a per-pixel quality metric that estimates quality or similarity by measuring differences between pixels. Because it compares vectors with strictly non-negative elements, it is often used to compare colored images, as color values cannot be negative. This different approach has a better correlation with subjective quality assessment than PSNR. == Definition == Androutsos et al. give the Czekanowski coefficient as follows: d z ( i , j ) = 1 − 2 ∑ k = 1 p min ( x i k , x j k ) ∑ k = 1 p ( x i k + x j k ) {\displaystyle d_{z}(i,j)=1-{\frac {2\sum _{k=1}^{p}{\text{min}}(x_{ik},\ x_{jk})}{\sum _{k=1}^{p}(x_{ik}+x_{jk})}}} Where a pixel x i {\displaystyle x_{i}} is being compared to a pixel x j {\displaystyle x_{j}} on the k-th band of color – usually one for each of red, green and blue. For a pixel matrix of size M × N {\displaystyle M\times N} , the Czekanowski coefficient can be used in an arithmetic mean spanning all pixels to calculate the Czekanowski distance as follows: 1 M N ∑ i = 0 M − 1 ∑ j = 0 N − 1 ( 1 − 2 ∑ k = 1 3 min ( A k ( i , j ) , B k ( i , j ) ) ∑ k = 1 3 ( A k ( i , j ) + B k ( i , j ) ) ) {\displaystyle {\frac {1}{MN}}\sum _{i=0}^{M-1}\sum _{j=0}^{N-1}{\begin{pmatrix}1-{\frac {2\sum _{k=1}^{3}{\text{min}}(A_{k}(i,j),\ B_{k}(i,j))}{\sum _{k=1}^{3}(A_{k}(i,j)+B_{k}(i,j))}}\end{pmatrix}}} Where A k ( i , j ) {\displaystyle A_{k}(i,j)} is the (i, j)-th pixel of the k-th band of a color image and, similarly, B k ( i , j ) {\displaystyle B_{k}(i,j)} is the pixel that it is being compared to. == Uses == In the context of image forensics – for example, detecting if an image has been manipulated –, Rocha et al. report the Czekanowski distance is a popular choice for Color Filter Array (CFA) identification.
Read more →
Unspent transaction output

In cryptocurrencies, an unspent transaction output (UTXO, often capitalized as UTxO) is a distinctive element in a subset of digital currency models. A UTXO represents a certain amount of cryptocurrency that has been authorized by a sender and is available to be spent by a recipient. The utilization of UTXOs in transaction processes is a key feature of many cryptocurrencies, but it primarily characterizes those implementing the UTXO model. UTXOs employ public key cryptography to ascertain and transfer ownership. More specifically, the recipient's public key is formatted into the UTXO, thereby limiting the capability to spend the UTXO to the account that can demonstrate ownership of the corresponding private key. A valid digital signature associated with the public key must be included for the UTXO to be spent. In the UTXO model, each unit of currency is treated as a discrete object. The history of a UTXO is documented only within the blocks where it is transferred. To ascertain the total balance of an account, one must scan each block to find the latest UTXOs linked to that account. While all nodes within a blockchain network must consent to the block history, the blocks relevant to an account's balance are unique to that account. UTXOs constitute a chain of ownership depicted as a series of digital signatures dating back to the coin's inception, regardless of whether the coin was minted via mining, staking, or another procedure determined by the cryptocurrency protocol. The UTXO model was invented for Bitcoin. Cardano uses an extended version of the UTXO model known as EUTXO. == Origins == The conceptual framework of the UTXO model can be traced back to Hal Finney's Reusable Proofs of Work proposal, which itself was based on Adam Back's 1997 Hashcash proposal. Bitcoin, released in 2009, was the first widespread implementation of the UTXO model in practice. == UTXO model vs. account Model == Cryptocurrencies that utilize the UTXO model function differently compared to those using the account model. In the UTXO model, individual units of cryptocurrency, termed as unspent transaction outputs (UTXOs), are transferred between users, analogous to the exchange of physical cash. This model impacts how transactions and ownership are recorded and verified within the blockchain network. The account model preserves a record of each account and its corresponding balance for every block added to the network. This setup enables quicker balance verification without the need to scan historical blocks, but it increases the raw size of each block (though data compression techniques can be utilized to alleviate this). However, both models necessitate the inspection of past blocks to fully authenticate the origin of coins. In the UTXO model, each object is immutable - units of coins cannot be 'edited' in the same way an account balance is modified when a transaction occurs. Rather, the balance is computed from the transaction history dating back to when the coins were first minted. This simplicity enhances security, as a UTXO either exists in its anticipated form or it does not. In contrast, the account model requires meticulous verification of the account's status during transactions, which can lead to oversights if not conducted correctly. In valid blockchain transactions, only unspent outputs (UTXOs) are permissible for funding subsequent transactions. This requirement is critical to prevent double-spending and fraud. Accordingly, inputs in a transaction are removed from the UTXO set, while outputs create new UTXOs that are added to the set. The holders of private keys, such as those with cryptocurrency wallets, can utilize these UTXOs for future transactions.
Read more →