In computing, data transformation is the process of converting data from one format or structure into another format or structure. It is a fundamental aspect of most data integration and data management tasks such as data wrangling, data warehousing, data integration and application integration. Data transformation can be simple or complex based on the required changes to the data between the source (initial) data and the target (final) data. Data transformation is typically performed via a mixture of manual and automated steps. Tools and technologies used for data transformation can vary widely based on the format, structure, complexity, and volume of the data being transformed. A master data recast is another form of data transformation where the entire database of data values is transformed or recast without extracting the data from the database. All data in a well-designed database is directly or indirectly related to a limited set of master database tables by a network of foreign key constraints. Each foreign key constraint is dependent upon a unique database index from the parent database table. Therefore, when the proper master database table is recast with a different unique index, the directly and indirectly related data are also recast or restated. The directly and indirectly related data may also still be viewed in the original form since the original unique index still exists with the master data. Also, the database recast must be done in such a way as to not impact the applications architecture software. When the data mapping is indirect via a mediating data model, the process is also called data mediation. == Data transformation process == Data transformation can be divided into the following steps, each applicable as needed based on the complexity of the transformation required. Data discovery Data mapping Code generation Code execution Data review These steps are often the focus of developers or technical data analysts who may use multiple specialized tools to perform their tasks. The steps can be described as follows: Data discovery is the first step in the data transformation process. Typically the data is profiled using profiling tools or sometimes using manually written profiling scripts to better understand the structure and characteristics of the data and decide how it needs to be transformed. Data mapping is the process of defining how individual fields are mapped, modified, joined, filtered, aggregated etc. to produce the final desired output. Developers or technical data analysts traditionally perform data mapping since they work in the specific technologies to define the transformation rules (e.g. visual ETL tools, transformation languages). Code generation is the process of generating executable code (e.g. SQL, Python, R, or other executable instructions) that will transform the data based on the desired and defined data mapping rules. Typically, the data transformation technologies generate this code based on the definitions or metadata defined by the developers. Code execution is the step whereby the generated code is executed against the data to create the desired output. The executed code may be tightly integrated into the transformation tool, or it may require separate steps by the developer to manually execute the generated code. Data review is the final step in the process, which focuses on ensuring the output data meets the transformation requirements. It is typically the business user or final end-user of the data that performs this step. Any anomalies or errors in the data that are found and communicated back to the developer or data analyst as new requirements to be implemented in the transformation process. == Types of data transformation == === Batch data transformation === Traditionally, data transformation has been a bulk or batch process, whereby developers write code or implement transformation rules in a data integration tool, and then execute that code or those rules on large volumes of data. This process can follow the linear set of steps as described in the data transformation process above. Batch data transformation is the cornerstone of virtually all data integration technologies such as data warehousing, data migration and application integration. When data must be transformed and delivered with low latency, the term "microbatch" is often used. This refers to small batches of data (e.g. a small number of rows or a small set of data objects) that can be processed very quickly and delivered to the target system when needed. === Benefits of batch data transformation === Traditional data transformation processes have served companies well for decades. The various tools and technologies (data profiling, data visualization, data cleansing, data integration etc.) have matured and most (if not all) enterprises transform enormous volumes of data that feed internal and external applications, data warehouses and other data stores. === Limitations of traditional data transformation === This traditional process also has limitations that hamper its overall efficiency and effectiveness. The people who need to use the data (e.g. business users) do not play a direct role in the data transformation process. Typically, users hand over the data transformation task to developers who have the necessary coding or technical skills to define the transformations and execute them on the data. This process leaves the bulk of the work of defining the required transformations to the developer, which often in turn do not have the same domain knowledge as the business user. The developer interprets the business user requirements and implements the related code/logic. This has the potential of introducing errors into the process (through misinterpreted requirements), and also increases the time to arrive at a solution. This problem has given rise to the need for agility and self-service in data integration (i.e. empowering the user of the data and enabling them to transform the data themselves interactively). There are companies that provide self-service data transformation tools. They are aiming to efficiently analyze, map and transform large volumes of data without the technical knowledge and process complexity that currently exists. While these companies use traditional batch transformation, their tools enable more interactivity for users through visual platforms and easily repeated scripts. Still, there might be some compatibility issues (e.g. new data sources like IoT may not work correctly with older tools) and compliance limitations due to the difference in data governance, preparation and audit practices. === Interactive data transformation === Interactive data transformation (IDT) is an emerging capability that allows business analysts and business users the ability to directly interact with large datasets through a visual interface, understand the characteristics of the data (via automated data profiling or visualization), and change or correct the data through simple interactions such as clicking or selecting certain elements of the data. Although interactive data transformation follows the same data integration process steps as batch data integration, the key difference is that the steps are not necessarily followed in a linear fashion and typically don't require significant technical skills for completion. There are a number of companies that provide interactive data transformation tools, including Trifacta, Alteryx and Paxata. They are aiming to efficiently analyze, map and transform large volumes of data while at the same time abstracting away some of the technical complexity and processes which take place under the hood. Interactive data transformation solutions provide an integrated visual interface that combines the previously disparate steps of data analysis, data mapping and code generation/execution and data inspection. That is, if changes are made at one step (like for example renaming), the software automatically updates the preceding or following steps accordingly. Interfaces for interactive data transformation incorporate visualizations to show the user patterns and anomalies in the data so they can identify erroneous or outlying values. Once they've finished transforming the data, the system can generate executable code/logic, which can be executed or applied to subsequent similar data sets. By removing the developer from the process, interactive data transformation systems shorten the time needed to prepare and transform the data, eliminate costly errors in the interpretation of user requirements and empower business users and analysts to control their data and interact with it as needed. == Transformational languages == There are numerous languages available for performing data transformation. Many transformation languages require a grammar to be provided. In many cases, the grammar is structured using something closely resembling Backus–Naur form (BNF). There are numerous languages
Aarogya Setu
Aarogya Setu (lit. 'The bridge to health') is an Indian COVID-19 "contact tracing, syndromic mapping and self-assessment" digital service, primarily a mobile app, developed by the National Informatics Centre under the Ministry of Electronics and Information Technology (MeitY). The app reached more than 100 million installs in 40 days. On 26 May, amid growing privacy and security concerns, the source code of the app was made public. == Full view == The stated purpose of this app is to spread awareness of COVID-19 and to connect essential COVID-19-related health services to the people of India. This app augments the initiatives of the Department of Health to contain COVID-19 and shares best practices and advisories. It is a tracking app which uses the smartphone's GPS and Bluetooth features to track COVID-19 cases. The app is available for Android and iOS mobile operating systems. With Bluetooth, it tries to determine the risk if one has been near (within six feet of) a COVID-19-infected person, by scanning through a database of known cases across India. Using location information, it determines whether the location one is in belongs to one of the infected areas based on the data available. This app is an updated version of an earlier app called Corona Kavach (now discontinued) which was released earlier by the Government of India. == Features and tools == Aarogya Setu has four sections: User Status (tells the risk of getting COVID-19 for the user) Self Assess (helps the users identify COVID-19 symptoms and their risk profile) COVID-19 Updates (gives updates on local and national COVID-19 cases) E-pass integration (if applied for E-pass, it will be available) See Recent Contacts option (allows the users to assess the risk level of their Bluetooth contacts) It tells how many COVID-19 positive cases are likely in a radius of 500 m, 1 km, 2 km, 5 km and 10 km from the user. The app is built on a platform that can provide an application programming interface (API) so that other computer programs, mobile applications, and web services can make use of the features and data available in Aarogya Setu. == Response == Aarogya Setu crossed five million downloads within three days of its launch, making it one of the most popular government apps in India. It became the world's fastest-growing mobile app, beating Pokémon Go, with more than 50 million installs 13 days after launching in India on 2 April 2020. It reached 100 million installs by 13 May 2020, that is in 40 days since its launch. In an order on 29 April 2020 the central government made it mandatory for all employees to download the app and use it – "Before starting for office, they must review their status on Aarogya Setu and commute only when the app shows safe or low risk". The Union Home Ministry also said that the application is mandatory for all living in the COVID-19 containment zone. The government gave the announcement along with the nationwide lockdown extension by two weeks from the 4 May with certain relaxations. On 21 May 2020, the Airport Authority of India issued a Standard Operating Procedure (SOP) stating that all departing passengers must compulsorily be registered with the Aarogya Setu app. It added that the app would not be mandatory for children below 14 years. However, the next day, Civil Aviation Minister Hardeep Singh Puri clarified that the app would not be mandatory for any passengers. On 26 May 2020, the Aarogya Setu app code was made open to developers across the globe to help other countries manage contact tracing in their fight against COVID-19 pandemic. In March 2021, Co-WIN portal was integrated with the app. This allowed users to schedule an appointment through the app for COVID-19 vaccine by registering their phone number and providing relevant documents. == Effectiveness == NITI Aayog CEO revealed that "the app has been able to identify more than 3,000 hotspots in 3–17 days ahead of time." However, users and experts in India and around the world say the app raises huge data security concerns. The app collects name, number, gender, travel history, and uses a phone's Bluetooth and location data to let users know if they have been near a person with COVID-19 by scanning a database of known cases of infection, and also share it with the government simultaneously. This is the major area of concern as the app's constant access to a phone's Bluetooth imposes a form of security threat. But it stood to clarify itself that the informations received are not going to be made public. Amidst all these, the app hits a record of about one-hundred million downloads. == Reception == Rahul Gandhi, leader of the Congress party, termed the Aarogya Setu application a "sophisticated surveillance system" after the government announced that downloading the app would be mandatory for both government and private employees. Following this, others raised the same concerns about the Aarogya Setu app. The Ministry of Electronics and Information Technology (MeitY) responded to these concerns by asserting that Gandhi's claims were false, and that the app was being appreciated internationally. On 5 May, French ethical hacker Robert Baptiste, who goes by the name Elliot Alderson on Twitter, claimed that there were security issues with the app. The Indian government, as well as the app developers, responded to this claim by thanking the hacker for his attention, but dismissed his concerns. The developers of the app stated that the fetching of location data is a documented feature of the app, rather than a flaw, since the app is designed to track the distribution of the virus-infected population. They also asserted that no personal information of any user has been proven to be at risk. On 6 May, Robert Baptiste tweeted that security vulnerabilities in Aarogya Setu allowed hackers to "know who is infected, unwell, [or] made a self assessment in the area of his choice". He also gave details of how many people were unwell and infected at the Prime Minister's Office, the Indian Parliament and the Home Office. The Economic Times pointed out that a clause in the app's Terms and Conditions stated that the user "agrees and acknowledges that the Government of India will not be liable for ... any unauthorised access to your information or modification thereof". In response, several software developers called for the source code to be made public. On 12 May, former Supreme Court Judge Justice B.N. Srikrishna termed the government's push mandating the use of Aarogya Setu app "utterly illegal". He said so far it is not backed by any law and questioned "under what law, government is mandating it on anyone". MIT Technology Review gave 2 out of 5 stars to Aarogya Setu app after analyzing the COVID contact tracing apps launched in 25 countries. The app got stars only for the policy which suggests that data collected is deleted after a period of time and that the data collection, as far as user inputs go, is minimal. It also highlighted that India is the only democracy making its app mandatory for millions of people. The rating was further downgraded from 2 to 1 for collecting more information than the app needs to function. Following this, the MeitY made the source code of the Android app public on GitHub on 26 May, which will be followed by iOS and API documentation. Further, the Government has also launched a "bug bounty program". This was done to "promote transparency and ensure security and integrity of the app". However, experts stated that the server-side code had not yet been publicly released, which meant that public opinion on security and privacy was yet to be completely assuaged. Following this, ZDNet noted that the source code seemed to confirm the government's claim that user location data, if collected, would be anonymised and would be deleted after 45 days, or 60 days for high-risk individuals.
Point-to-point encryption
Point-to-point encryption (P2PE) is a standard established by the PCI Security Standards Council. Payment solutions that offer similar encryption but do not meet the P2PE standard are referred to as end-to-end encryption (E2EE) solutions. The objective of P2PE and E2EE is to provide a payment security solution that instantaneously converts confidential payment card (credit and debit card) data and information into indecipherable code at the time the card is swiped, in order to prevent hacking and fraud. It is designed to maximize the security of payment card transactions in an increasingly complex regulatory environment. == The standard == The P2PE Standard defines the requirements that a "solution" must meet in order to be accepted as a PCI-validated P2PE solution. A "solution" is a complete set of hardware, software, gateway, decryption, device handling, etc. Only "solutions" can be validated; individual pieces of hardware such as card readers cannot be validated. It is also a common mistake to refer to P2PE validated solutions as "certified"; there is no such certification. The determination of whether or not a solution meets the P2PE standard is the responsibility of a P2PE Qualified Security Assessor (P2PE-QSA). P2PE-QSA companies are independent third-party companies who employ assessors that have met the PCI Security Standards Council's requirements for education and experience, and have passed the requisite exam. The PCI Security Standards Council does not validate solutions. == How it works == As a payment card is swiped through a card reading device, referred to as a point of interaction (POI) device, at the merchant location or point of sale, the device immediately encrypts the card information. A device that is part of a PCI-validated P2PE solution uses an algorithmic calculation to encrypt the confidential payment card data. From the POI, the encrypted, indecipherable codes are sent to the payment gateway or processor for decryption. The keys for encryption and decryption are never available to the merchant, making card data entirely invisible to the retailer. Once the encrypted codes are within the secure data zone of the payment processor, the codes are decrypted to the original card numbers and then passed to the issuing bank for authorization. The bank either approves or rejects the transaction, depending upon the card holder's payment account status. The merchant is then notified if the payment is accepted or rejected to complete the process along with a token that the merchant can store. This token is a unique number reference to the original transaction that the merchant can use should they ever be needed to perform research or refund the customer without ever knowing the customer's card information (tokenization). There are also Qualified Integrator and Reseller (QIR) Companies, which are businesses authorized to "implement, configure, and/or support validated" PA-DSS Payment Applications, and perform qualified installations. == Solution providers == According to the PCI Security Standards Council:The P2PE solution provider is a third-party entity (for example, a processor, acquirer, or payment gateway) that has overall responsibility for the design and implementation of a specific P2PE solution, and manages P2PE solutions for its merchant customers. The solution provider has overall responsibility for ensuring that all P2PE requirements are met, including any P2PE requirements performed by third-party organizations on behalf of the solution provider (for example, certification authorities and key-injection facilities). == Benefits == === Customer benefits === P2PE significantly reduces the risk of payment card fraud by instantaneously encrypting confidential cardholder data at the moment a payment card is swiped or "dipped" if it is a chip card at the card reading device (payment terminal) or POI. === Merchant benefits === P2PE significantly facilitates merchant responsibilities: With a P2PE validated solution, merchants save significant time and money as PCI requirements may be greatly reduced. Payment Card Industry Data Security Standard (PCI DSS). For organizations who use a P2PE validated solution provider, the PCI Self Assessment Questionnaire is reduced from 12 sections to 4 sections and the controls are reduced from 329 questions to just 35. In the event of fraud, the P2PE Solution Provider, not the merchant, is held accountable for data loss and resulting fines that may be assessed by the card brands (American Express, Visa, MasterCard, Discover, and JCB). The PCI Security Standards Council does not assess penalties on Solution Providers or Merchants. The payment process with P2PE is quicker than other transaction processes, thus creating simpler and faster customer–merchant transactions. == Point-to-point encryption versus end-to-end encryption == === Point-to-point === A point-to-point connection directly links system 1 (the point of payment card acceptance) to system 2 (the point of payment processing). A true P2PE solution is determined with three main factors: The solution uses a hardware-to-hardware encryption and decryption process along with a POI device that has SRED (Secure Reading and Exchange of Data) listed as a function. The solution has been validated to the PCI P2PE Standard which includes specific POI device requirements such as strict controls regarding shipping, receiving, tamper-evident packaging, and installation. A solution includes merchant education in the form of a P2PE Instruction Manual, which guides the merchant on POI device use, storage, return for repairs, and regular PCI reporting. === End-to-end === End-to-end encryption as the name suggests has the advantage over P2PE that card details are not unencrypted between the two endpoints. If the endpoints are a PCI PED validated PIN pad and a POS acquirer, there is no opportunity for the card details to be intercepted. It is obviously important that the endpoints (the PED and gateway) are provided by PCI accredited organisations. == PCI point-to-point encryption requirements == The requirements include: Secure encryption of payment card data at the point of interaction (POI), P2PE validated application(s) at the point of interaction, Secure management of encryption and decryption devices, Management of the decryption environment and all decrypted account data, Use of secure encryption methodologies and cryptographic key operations, including key generation, distribution, loading/injection, administration, and usage.
Content-oriented workflow models
In data management, a content-oriented workflow model seeks to articulate workflow progression by the presence of content units (like data-records/objects/documents). Most content-oriented workflow approaches provide a life-cycle model for content units, such that workflow progression can be qualified by conditions on the state of the units. Most approaches are research and work in progress and the content models and life-cycle models are more or less formalized. The term content-oriented workflows is an umbrella term for several scientific workflow approaches, namely "data-driven", "resource-driven", "artifact-centric", "object-aware", and "document-oriented". Thus, the meaning of "content" ranges from simple data attributes to self-contained documents; the term "content-oriented workflows" appeared at first in as an umbrella term. Such a general term, independent from a specific approach, is necessary to contrast the content-oriented modelling principle with traditional activity-oriented workflow models (like Petri nets or BPMN) where a workflow is driven by a control flow and where the content production perspective is neglected or even missing. The term "content" was chosen to subsume the different levels in granularity of the content units in the respective workflow models; it was also chosen to make associations with content management. Both terms "artifact-centric" and "data-driven" would also be good candidates for an umbrella term, but each is closely related to a specific approach of a single working group. The "artifact-centric" group itself (i.e. IBM Research) has generalized the characteristics of their approach and has used "information-centric" as an umbrella term in. Yet, the term information is too unspecific in the context of computer science, thus, "content-orientated workflows" is considered as good compromise. == Workflow Model Approaches == === Data-driven === The data-driven process structures provides a sophisticated workflow model being specialized on hierarchical write-and-review-processes. The approach provides interleaved synchronization of sub-processes and extends activity diagrams. Unfortunately, the COREPRO prototype implementation is not publicly available. Research on the project had been ceased. The general idea has been continued by Reichert in form of the #Object-aware approach. Synonyms data-driven process structures / data-driven modeling and coordination Protagonists Dr. Dominic Müller (University of Twente), Joachim Herbst (DaimlerChrysler Research), and Manfred Reichert (at this time Assoc. Prof. at Univ. of Twente, currently Prof. at Ulm Univ.) Organization(s) University of Twente, DaimlerChrysler Period 2005 - 2007 Selected publications Implementation COREPRO === Resource-driven === The resource-driven workflow system is an early approach that considered workflows from a content-oriented perspective and emphasizes on the missing support for plain document-driven processes by traditional activity-oriented workflow engines. The resource-driven approach demonstrated the application of database triggers for handling workflow events. Still the system implementation is centralized and the workflow schema is statically defined. The project appeared in 2005 but many aspects are considered future work by the authors. Research did not continue on the project. Wang completed his PhD thesis in 2009, yet, his thesis does not mention the resource-driven approach to workflow modelling but is about discrete event simulation. Synonyms Resource-based Workflows / Document-Driven Workflow Systems Protagonists Jianrui Wang and Prof. Akhil Kumar Organization Pennsylvania State University Period 2005 - today Selected publications Implementation N/A === Artifact-centric === The artifact-centric approach provides a framework for content-oriented workflows. In this model, the enterprise application landscape includes distributed business services, while the workflow engine is centralized. Process enactment is integrated with database management system infrastructure, and the project is funded by IBM. Synonyms artifact-centric business process models / artifact-based business process (ACP) / artifact-centric workflows Protagonists Richard Hull and Dr. Kamal Bhattacharya as well as Cagdas E. Gerede and Jianwen Su Organization IBM (T.J. Watson Research Center, NY) Period 2007 - today Selected publications Implementation ArtiFact === Object-aware === The object-aware approach manages a set of object types and generates forms for creating object instances. The form completion flow is controlled by transitions between object configurations each describing a progressing set of mandatory attributes. Each object configuration is named by an object state. The data production flow is user-shifting and it is discrete by defining a sequence of object states. The discussion is currently limited to a centralized system, without any workflows across different organizations. However, the approach is of great relevance to many domains like concurrent engineering. Finally, the object-aware approach and its PHILharmonicFlows system are going to provide general-purpose workflow systems for generic enactment of data production processes. Synonyms object-aware process management / datenorientiertes Prozess-Management-System Protagonists Vera Künzle and Prof. Manfred Reichert Organization Ulm University Period 2009 - today Selected publications Implementation PHILharmonicFlows === Distributed Document-oriented === Distributed document-oriented process management (dDPM) enables distributed case handling in heterogeneous system environments and it is based on document-oriented integration. The workflow model reflects the paper-based working practice in inter-institutional healthcare scenarios. It targets distributed knowledge-driven ad hoc workflows, wherein distributed information systems are required to coordinate work with initially unknown sets of actors and activities. The distributed workflow engine supports process planning & process history as well as participant management and process template creation with import/export. The workflow engine embeds a functional fusion of 1) group-based instant messaging 2) with a shared work list editor 3) with version control. The software implementation of dDPM is α-Flow which is available as open source. dDPM and α-Flow provide a content-oriented approach to schema-less workflows. The complete distributed case handling application is provided in form of a single active Document ("α-Doc"). The α-Doc is a case file (as information carrier) with an embedded workflow engine (in form of active properties). Inviting process participants is equivalent to providing them with a copy of an α-Doc, copying it like an ordinary desktop file. All α-Docs that belong to the same case can synchronize each other, based on the participant management, electronic postboxes, store-and-forward messaging, and an offline-capable synchronization protocol. Synonyms distributed document-oriented process management (dDPM), distributed case handling via active documents Protagonists Christoph P. Neumann and Prof. Richard Lenz Organization Friedrich-Alexander-Universität Erlangen-Nürnberg Period 2009 - 2012 Selected Publications and a PhD thesis Implementation α-Flow (open source) == Related Concepts == === Content Management === The bandwidth of Content management systems (CMS) reaches from Web content management systems (WCMS) and Document management system (DMS) to Enterprise Content Management (ECM). Mature DMS products support document production workflows in a basic form, primarily focusing on review cycle workflows concerning a single document. === Groupware and Computer-Supported Cooperative Work === Groupware focuses on messaging (like E-Mail, Chat, and Instant Messaging), shared calendars (e.g. Lotus Notes, Microsoft Outlook with Exchange Server), and conferencing (e.g. Skype). Groupware overlaps with Computer-supported cooperative work (CSCW), that originated from shared multimedia editors (for live drawing/sketching) and synchronous multi-user applications like desktop sharing. The extensive conceptual claim of CSWC must be put into perspective by its actual solution scope, that is available as the CSCW Matrix. === Case Handling === The case handling paradigm stems from Prof. van der Aalst and gained momentum in 2005. The core features are: (a) provide all information available, i.e. present the case as a whole rather than showing bits and pieces, (b) decide about activities on the basis of the information available rather than the activities already executed, (c) separate work distribution from authorization and allow for additional types of roles, not just the execute role, and (d) allow workers to view and add/modify data before or after the corresponding activities have been executed. In healthcare, the flow of a patient between healthcare professionals is considered as a workflow - with activities that inc
Cut, copy, and paste
Cut, copy, and paste are essential commands of modern human–computer interaction and user interface design. They offer an interprocess communication technique for transferring data through a computer's user interface. The cut command removes the selected data from its original position, and the copy command creates a duplicate; in both cases the selected data is kept in temporary storage called the clipboard. Clipboard data is later inserted wherever a paste command is issued. The data remains available to any application supporting the feature, thus allowing easy data transfer between applications. The command names are a (skeuomorphic) interface metaphor based on the physical procedure used in manuscript print editing to create a page layout, like with paper. The commands were pioneered into computing by Xerox PARC in 1974, popularized by Apple Computer in the 1983 Lisa workstation and the 1984 Macintosh computer, and in a few home computer applications such as the 1984 word processor Cut & Paste. This interaction technique has close associations with related techniques in graphical user interfaces (GUIs) that use pointing devices such as a computer mouse (by drag and drop, for example). Typically, clipboard support is provided by an operating system as part of its GUI and widget toolkit. The capability to replicate information with ease, changing it between contexts and applications, involves privacy concerns because of the risks of disclosure when handling sensitive information. Terms like cloning, copy forward, carry forward, or re-use refer to the dissemination of such information through documents, and may be subject to regulation by administrative bodies. == History == === Origins === The term "cut and paste" comes from the traditional practice in manuscript editing, whereby people cut paragraphs from a page with scissors and paste them onto another page. This practice remained standard into the 1980s. Stationery stores sold "editing scissors" with blades long enough to cut an 8½"-wide page. The advent of photocopiers made the practice easier and more flexible. The act of copying or transferring text from one part of a computer-based document ("buffer") to a different location within the same or different computer-based document was a part of the earliest on-line computer editors. As soon as computer data entry moved from punch-cards to online files (in the mid/late 1960s) there were "commands" for accomplishing this operation. This mechanism was often used to transfer frequently-used commands or text snippets from additional buffers into the document, as was the case with the QED text editor. === Early methods === The earliest editors (designed for teleprinter terminals) provided keyboard commands to delineate a contiguous region of text, then delete or move it. Since moving a region of text requires first removing it from its initial location and then inserting it into its new location, various schemes had to be invented to allow for this multi-step process to be specified by the user. Often this was done with a "move" command, but some text editors required that the text be first put into some temporary location for later retrieval/placement. In 1983, the Apple Lisa became the first text editing system to call that temporary location "the clipboard". Earlier control schemes such as NLS used a verb—object command structure, where the command name was provided first and the object to be copied or moved was second. The inversion from verb—object to object—verb on which copy and paste are based, where the user selects the object to be operated before initiating the operation, was an innovation crucial for the success of the desktop metaphor as it allowed copy and move operations based on direct manipulation. === Popularization === Inspired by early line and character editors, such as Pentti Kanerva's TV-Edit, that broke a move or copy operation into two steps—between which the user could invoke a preparatory action such as navigation—Lawrence G. "Larry" Tesler proposed the names "cut" and "copy" for the first step and "paste" for the second step. Beginning in 1974, he and colleagues at Xerox PARC implemented several text editors that used cut/copy-and-paste commands to move and copy text. Apple Computer popularized this paradigm with its Lisa (1983) and Macintosh (1984) operating systems and applications. The functions were mapped to key combinations using the ⌘ Command key as a special modifier, which is held down while also pressing X for cut, C for copy, or V for paste. These few keyboard shortcuts allow the user to perform all the basic editing operations, and the keys are clustered at the left end of the bottom row of the standard QWERTY keyboard. These are the standard shortcuts: Control-Z (or ⌘ Command+Z) to undo Control-X (or ⌘ Command+X) to cut Control-C (or ⌘ Command+C) to copy Control-V (or ⌘ Command+V) to paste The IBM Common User Access (CUA) standard also uses combinations of the Insert, Del, Shift and Control keys. Early versions of Windows used the IBM standard. Microsoft later also adopted the Apple key combinations with the introduction of Windows, using the control key as modifier key. Similar patterns of key combinations, later borrowed by others, are widely available in most GUI applications. The original cut, copy, and paste workflow, as implemented at PARC, utilizes a unique workflow: With two windows on the same screen, the user could use the mouse to pick a point at which to make an insertion in one window (or a segment of text to replace). Then, by holding shift and selecting the copy source elsewhere on the same screen, the copy would be made as soon as the shift was released. Similarly, holding shift and control would copy and cut (delete) the source. This workflow requires many fewer keystrokes/mouse clicks than the current multi-step workflows, and did not require an explicit copy buffer. It was dropped, one presumes, because the original Apple and IBM GUIs were not high enough density to permit multiple windows, as were the PARC machines, and so multiple simultaneous windows were rarely used. == Cut and paste == Computer-based editing can involve very frequent use of cut-and-paste operations. Most software-suppliers provide several methods for performing such tasks, and this can involve (for example) key combinations, pulldown menus, pop-up menus, or toolbar buttons. The user selects or "highlights" the text or file for moving by some method, typically by dragging over the text or file name with the pointing-device or holding down the Shift key while using the arrow keys to move the text cursor. The user performs a "cut" operation via key combination Ctrl+x (⌘+x for Macintosh users), menu, or other means. Visibly, "cut" text immediately disappears from its location. "Cut" files typically change color to indicate that they will be moved. Conceptually, the text has now moved to a location often called the clipboard. The clipboard typically remains invisible. On most systems only one clipboard location exists, hence another cut or copy operation overwrites the previously stored information. Many UNIX text-editors provide multiple clipboard entries, as do some Macintosh programs such as Clipboard Master, and Windows clipboard-manager programs such as the one in Microsoft Office. The user selects a location for insertion by some method, typically by clicking at the desired insertion point. A paste operation takes place which visibly inserts the clipboard text at the insertion point. (The paste operation does not typically destroy the clipboard text: it remains available in the clipboard and the user can insert additional copies at other points). Whereas cut-and-paste often takes place with a mouse-equivalent in Windows-like GUI environments, it may also occur entirely from the keyboard, especially in UNIX text editors, such as Pico or vi. Cutting and pasting without a mouse can involve a selection (for which Ctrl+x is pressed in most graphical systems) or the entire current line, but it may also involve text after the cursor until the end of the line and other more sophisticated operations. The clipboard usually stays invisible, because the operations of cutting and pasting, while actually independent, usually take place in quick succession, and the user (usually) needs no assistance in understanding the operation or maintaining mental context. Some application programs provide a means of viewing, or sometimes even editing, the data on the clipboard. == Copy and paste == The term "copy-and-paste" refers to the popular, simple method of reproducing text or other data from a source to a destination. It differs from cut and paste in that the original source text or data does not get deleted or removed. The popularity of this method stems from its simplicity and the ease with which users can move data between various applications visually – without resorting to permanent storage. Use in healthcare do
Feature (machine learning)
In machine learning and pattern recognition, a feature is an individual measurable property or characteristic of a data set. Choosing informative, discriminating, and independent features is crucial to producing effective algorithms for pattern recognition, classification, and regression tasks. Features are usually numeric, but other types such as strings and graphs are used in syntactic pattern recognition, after some pre-processing step such as one-hot encoding. The concept of "features" is related to that of explanatory variables used in statistical techniques such as linear regression. == Feature types == In feature engineering, two types of features are commonly used: numerical and categorical. Numerical features are continuous values that can be measured on a scale. Examples of numerical features include age, height, weight, and income. Numerical features can be used in machine learning algorithms directly. Categorical features are discrete values that can be grouped into categories. Examples of categorical features include gender, color, and zip code. Categorical features typically need to be converted to numerical features before they can be used in machine learning algorithms. This can be done using a variety of techniques, such as one-hot encoding, label encoding, and ordinal encoding. The type of feature that is used in feature engineering depends on the specific machine learning algorithm that is being used. Some machine learning algorithms, such as decision trees, can handle both numerical and categorical features. Other machine learning algorithms, such as linear regression, can only handle numerical features. == Classification == A numeric feature can be conveniently described by a feature vector. One way to achieve binary classification is using a linear predictor function (related to the perceptron) with a feature vector as input. The method consists of calculating the scalar product between the feature vector and a vector of weights, qualifying those observations whose result exceeds a threshold. Algorithms for classification from a feature vector include nearest neighbor classification, neural networks, and statistical techniques such as Bayesian approaches. == Examples == In character recognition, features may include histograms counting the number of black pixels along horizontal and vertical directions, number of internal holes, stroke detection and many others. In speech recognition, features for recognizing phonemes can include noise ratios, length of sounds, relative power, filter matches, logarithmic Mel-scale spectral vectors and Mel-frequency cepstral coefficients, which represent the frequency characteristics of audio signals. In spam detection algorithms, features may include the presence or absence of certain email headers, the email structure, the language, the frequency of specific terms, the grammatical correctness of the text. In computer vision, there are a large number of possible features, such as edges and objects. == Feature vectors == In pattern recognition and machine learning, a feature vector is an n-dimensional vector of numerical features that represent some object. Many algorithms in machine learning require a numerical representation of objects, since such representations facilitate processing and statistical analysis. When representing images, the feature values might correspond to the pixels of an image, while when representing texts the features might be the frequencies of occurrence of textual terms. Feature vectors are equivalent to the vectors of explanatory variables used in statistical procedures such as linear regression. Feature vectors are often combined with weights using a dot product in order to construct a linear predictor function that is used to determine a score for making a prediction. The vector space associated with these vectors is often called the feature space. In order to reduce the dimensionality of the feature space, a number of dimensionality reduction techniques can be employed. Higher-level features can be obtained from already available features and added to the feature vector; for example, for the study of diseases the feature 'Age' is useful and is defined as Age = 'Year of death' minus 'Year of birth' . This process is referred to as feature construction. Feature construction is the application of a set of constructive operators to a set of existing features resulting in construction of new features. Examples of such constructive operators include checking for the equality conditions {=, ≠}, the arithmetic operators {+,−,×, /}, the array operators {max(S), min(S), average(S)} as well as other more sophisticated operators, for example count(S, C) that counts the number of features in the feature vector S satisfying some condition C or, for example, distances to other recognition classes generalized by some accepting device. Feature construction has long been considered a powerful tool for increasing both accuracy and understanding of structure, particularly in high-dimensional problems. Applications include studies of disease and emotion recognition from speech. == Selection and extraction == The initial set of raw features can be redundant and large enough that estimation and optimization is made difficult or ineffective. Therefore, a preliminary step in many applications of machine learning and pattern recognition consists of selecting a subset of features, or constructing a new and reduced set of features to facilitate learning, and to improve generalization and interpretability. Extracting or selecting features is a combination of art and science; developing systems to do so is known as feature engineering. It requires the experimentation of multiple possibilities and the combination of automated techniques with the intuition and knowledge of the domain expert. Automating this process is feature learning, where a machine not only uses features for learning, but learns the features itself.
Forward anonymity
Forward anonymity is a property of a cryptographic system which prevents an attacker who has recorded past encrypted communications from discovering its contents and participants in the future. This property is analogous to forward secrecy. An example of a system which uses forward anonymity is a public key cryptography system, where the public key is well-known and used to encrypt a message, and an unknown private key is used to decrypt it. In this system, one of the keys is always said to be compromised, but messages and their participants are still unknown by anyone without the corresponding private key. In contrast, an example of a system which satisfies the perfect forward secrecy property is one in which a compromise of one key by an attacker (and consequent decryption of messages encrypted with that key) does not undermine the security of previously used keys. Forward secrecy does not refer to protecting the content of the message, but rather to the protection of keys used to decrypt messages. == History == Originally introduced by Whitfield Diffie, Paul van Oorschot, and Michael James Wiener to describe a property of STS (station-to-station protocol) involving a long term secret, either a private key or a shared password. == Public Key Cryptography == Public Key Cryptography is a common form of a forward anonymous system. It is used to pass encrypted messages, preventing any information about the message from being discovered if the message is intercepted by an attacker. It uses two keys, a public key and a private key. The public key is published, and is used by anyone to encrypt a plaintext message. The Private key is not well known, and is used to decrypt cyphertext. Public key cryptography is known as an asymmetric decryption algorithm because of different keys being used to perform opposing functions. Public key cryptography is popular because, while it is computationally easy to create a pair of keys, it is extremely difficult to determine the private key knowing only the public key. Therefore, the public key being well known does not allow messages which are intercepted to be decrypted. This is a forward anonymous system because one compromised key (the public key) does not compromise the anonymity of the system. == Web of Trust == A variation of the public key cryptography system is a Web of trust, where each user has both a public and private key. Messages sent are encrypted using the intended recipient's public key, and only this recipient's private key will decrypt the message. They are also signed with the senders private key. This creates added security where it becomes more difficult for an attacker to pretend to be a user, as the lack of a private key signature indicates a non-trusted user. == Limitations == A forward anonymous system does not necessarily mean a wholly secure system. A successful cryptanalysis of a message or sequence of messages can still decode the information without the use of a private key or long term secret. == News == Forward anonymity, along with other privacy-protecting measures, received a burst of media attention after the leak of classified information by Edward Snowden, beginning in June, 2013, which indicated that the NSA and FBI, through specially crafted backdoors in software and computer systems, were conducting mass surveillance over large parts of the population of both the United States (see Mass surveillance in the United States), Europe, Asia, and other parts of the world. They justified this practice as an aid to catch predatory pedophiles. Opponents to this practice argue that leaving in a back door to law enforcement increases the risk of attackers being able to decrypt information, as well as questioning its legality under the US Constitution, specifically being a form of illegal Search and Seizure.