AI Generator Question Paper

AI Generator Question Paper — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

Two-phase locking

In databases and transaction processing, two-phase locking (2PL) is a pessimistic concurrency control method that guarantees conflict-serializability. It is also the name of the resulting set of database transaction schedules (histories). The protocol uses locks, applied by a transaction to data, which may block (interpreted as signals to stop) other transactions from accessing the same data during the transaction's life. By the 2PL protocol, locks are applied and removed in two phases: Expanding phase: locks are acquired and no locks are released. Shrinking phase: locks are released and no locks are acquired. Two types of locks are used by the basic protocol: Shared and Exclusive locks. Refinements of the basic protocol may use more lock types. Using locks that block processes, 2PL, S2PL, and SS2PL may be subject to deadlocks that result from the mutual blocking of two or more transactions. == Read and write locks == Locks are used to guarantee serializability. A transaction is holding a lock on an object if that transaction has acquired a lock on that object which has not yet been released. For 2PL, the only used data-access locks are read-locks (shared locks) and write-locks (exclusive locks). Below are the rules for read-locks and write-locks: A transaction is allowed to read an object if and only if it is holding a read-lock or write-lock on that object. A transaction is allowed to write an object if and only if it is holding a write-lock on that object. A schedule (i.e., a set of transactions) is allowed to hold multiple locks on the same object simultaneously if and only if none of those locks are write-locks. If a disallowed lock attempts on being held simultaneously, it will be blocked. == Variants == Note that all conflict serializable schedules are also view serializable (but not vice-versa). === Two-phase locking === According to the two-phase locking protocol, each transaction handles its locks in two distinct, consecutive phases during the transaction's execution: Expanding phase (aka Growing phase): locks are acquired and no locks are released (the number of locks can only increase). Shrinking phase (aka Contracting phase): locks are released and no locks are acquired. The two phase locking rules can be summarized as: each transaction must never acquire a lock after it has released a lock. The serializability property is guaranteed for a schedule with transactions that obey this rule. Typically, without explicit knowledge in a transaction on end of phase 1, the rule is safely determined only when a transaction has completed processing and requested commit. In this case, all the locks can be released at once (phase 2). === Conservative two-phase locking === Conservative two-phase locking (C2PL) differs from 2PL in that transactions obtain all the locks they need before the actual execution begins. This is to ensure that a transaction that already holds some locks will not block waiting for other locks. C2PL prevents deadlocks. In cases of heavy lock contention, C2PL reduces the time locks are held on average, relative to 2PL and Strict 2PL, because transactions that hold locks are never blocked. In light lock contention, C2PL holds more locks than is necessary, because it is difficult to predict which locks will be needed in the future, thus leading to higher overhead. A C2PL transaction will not obtain any locks if it cannot obtain all the locks it needs in its initial request. Furthermore, each transaction needs to declare its read and write set (the data items that will be read/written), which is not always possible. Because of these limitations, C2PL is not used very frequently. === Strict two-phase locking === To comply with the strict two-phase locking (S2PL) protocol, a transaction needs to comply with 2PL, and release its write (exclusive) locks only after the transaction has ended (i.e., either committed or aborted). On the other hand, read (shared) locks are released regularly during the shrinking phase. Unlike 2PL, S2PL provides strictness (a special case of cascade-less recoverability). This protocol is not appropriate in B-trees because it causes Bottleneck (while B-trees always starts searching from the parent root). === Strong strict two-phase locking === or Rigorousness, or Rigorous scheduling, or Rigorous two-phase locking To comply with strong strict two-phase locking (SS2PL), a transaction's read and write locks are released only after that transaction has ended (i.e., either committed or aborted). A transaction obeying SS2PL has only a phase 1 and lacks a phase 2 until the transaction has completed. Every SS2PL schedule is also an S2PL schedule, but not vice versa.
Read more →
Iteration

Iteration means repeating a process to generate a (possibly unbounded) sequence of outcomes. Each repetition of the process is a single iteration, and the outcome of each iteration is the starting point of the next iteration. In mathematics and computer science, iteration (along with the related technique of recursion) is a standard element of algorithms. == Mathematics == In mathematics, iteration may refer to the process of iterating a function, i.e. applying a function repeatedly, using the output from one iteration as the input to the next. Iteration of apparently simple functions can produce complex behaviors and difficult problems – for examples, see the Collatz conjecture and juggler sequences. Another use of iteration in mathematics is in iterative methods which are used to produce approximate numerical solutions to certain mathematical problems. Newton's method is an example of an iterative method. Manual calculation of a number's square root is a common use and a well-known example. == Computing == In computing, iteration is a technique that marks out of a block of statements within a computer program for a defined number of repetitions. That block of statements is said to be iterated. A computer programmer might also refer to that block of statements as an iteration. === Implementations === Loops constitute the most common language constructs for performing iterations. The following pseudocode "iterates" three times the line of code between begin & end through a for loop, and uses the values of i as increments. It is permissible, and often necessary, to use values from other parts of the program outside the bracketed block of statements, to perform the desired function. Iterators constitute alternative language constructs to loops, which ensure consistent iterations over specific data structures. They can eventually save time and effort in later coding attempts. In particular, an iterator allows one to repeat the same kind of operation at each node of such a data structure, often in some pre-defined order. Iteratees are purely functional language constructs, which accept or reject data during the iterations. === Relation with recursion === Recursions and iterations have different algorithmic definitions, even though they can generate identical results. The primary difference is that recursion can be a solution without prior knowledge as to how many times the action must repeat, while a successful iteration requires that foreknowledge. Some types of programming languages, known as functional programming languages, are designed such that they do not set up a block of statements for explicit repetition, as with the for loop. Instead, those programming languages exclusively use recursion. Rather than call out a block of code to repeate a pre-defined number of times, the executing code block instead "divides" the work into a number of separate pieces, after which the code block executes itself on each individual piece. Each piece of work is divided repeatedly until the "amount" of work is as small as possible, at which point the algorithm does that work very quickly. The algorithm then "reverses" and reassembles the pieces into a complete whole. The classic example of recursion is in list-sorting algorithms, such as merge sort. The merge sort recursive algorithm first repeatedly divides the list into consecutive pairs. Each pair is then ordered, then each consecutive pair of pairs, and so forth until the elements of the list are in the desired order. The code below is an example of a recursive algorithm in the Scheme programming language that outputs the same result as the pseudocode under the previous heading. == Education == In some schools of pedagogy, iterations are used to describe the process of teaching or guiding students to repeat experiments, assessments, or projects, until more accurate results are found, or the student has mastered the technical skill. This idea is found in the old adage, "Practice makes perfect." In particular, "iterative" is defined as the "process of learning and development that involves cyclical inquiry, enabling multiple opportunities for people to revisit ideas and critically reflect on their implication." Unlike computing and math, educational iterations are not predetermined; instead, the task is repeated until success according to some external criteria (often a test) is achieved.
Read more →
Ubiquitous robot

Ubiquitous robot is a term used in an analogous way to ubiquitous computing. Software useful for "integrating robotic technologies with technologies from the fields of ubiquitous and pervasive computing, sensor networks, and ambient intelligence". The emergence of mobile phone, wearable computers and ubiquitous computing makes it likely that human beings will live in a ubiquitous world in which all devices are fully networked. The existence of ubiquitous space resulting from developments in computer and network technology will provide motivations to offer desired services by any IT device at any place and time through user interactions and seamless applications. This shift has hastened the ubiquitous revolution, which has further manifested itself in the new multidisciplinary research area, ubiquitous robotics. It initiates the third generation of robotics following the first generation of the industrial robot and the second generation of the personal robot. Ubiquitous robot (Ubibot) is a robot incorporating three components including virtual software robot or avatar, real-world mobile robot and embedded sensor system in surroundings. Software robot within a virtual world can control a real-world robot as a brain and interact with human beings. Researchers of KAIST, Korea describe these three components as a Sobot (Software robot), Mobot (Mobile robot), and Embot (Embedded robot).
Read more →
System of record

A system of record (SOR) or source system of record (SSoR) is a data management term for an information storage system (commonly implemented on a computer system running a database management system) that is the authoritative data source for a given data element or piece of information, like for example a row (or record) in a table. In data vault it is referred to as the record source. == Background == The need to identify systems of record can become acute in organizations where management information systems have been built by taking output data from multiple source systems, re-processing this data, and then re-presenting the result for a new business use. In these cases, multiple information systems may disagree about the same piece of information. These disagreements may stem from semantic differences, differences in opinion, use of different sources, differences in the timing of the extract, transform, load processes that create the data they report against, or may simply be the result of bugs. == Use == The integrity and validity of any data set is open to question when there is no traceable connection to a good source, and listing a source system of record is a solution to this. Where the integrity of the data is vital, if there is an agreed system of record, the data element must either be linked to, or extracted directly from it. In other cases, the provenance and estimated data quality should be documented. The "system of record" approach is a good fit for environments where both: there is a single authority over all data consumers, and all consumers have similar needs == Trade-offs == In diverse environments, one instead needs to support the presence of multiple opinions. Consumers may accept different authorities or may differ on what constitutes an authoritative source—researchers may prefer carefully vetted data, while tactical military systems may require the most recent credible report.
Read more →
JSGF

JSGF stands for Java Speech Grammar Format or the JSpeech Grammar Format (in a W3C Note). Developed by Sun Microsystems, it is a textual representation of grammars for use in speech recognition for technologies like XHTML+Voice. JSGF adopts the style and conventions of the Java programming language in addition to use of traditional grammar notations. The Speech Recognition Grammar Specification was derived from this specification. == Example == The following JSGF grammar will recognize the words coffee, tea, and milk.
Read more →
Information school

Information school (sometimes abbreviated I-school or iSchool) is a university-level institution committed to understanding the role of information in nature and human endeavors. Synonyms include school of information, department of information studies, or information department. Information schools faculty conduct research into the fundamental aspects of information and related technologies. In addition to granting academic degrees, information schools educate information professionals, researchers, and scholars for an increasingly information-driven world. Information school can also refer, in a more restricted sense, to the members of the iSchools organization (formerly the "iSchools Project"), as governed by the iCaucus. Members of this group share a fundamental interest in the relationships between people, information, technology, and science. These schools, colleges, and departments have been either newly established or have evolved from programs focused on information systems, library science, informatics, computer science, library and information science and information science. Information schools promote an interdisciplinary approach to understanding the opportunities and challenges of information management, with a core commitment to concepts like universal access and user-centered organization of information. The field is concerned broadly with questions of design and preservation across information spaces, from digital and virtual spaces like online communities, the World Wide Web, and databases to physical spaces such as libraries, museums, archives, and other repositories. Information school degree programs include course offerings in areas such as data science, information architecture, design, economics, policy, retrieval, security, and telecommunications; knowledge management, user experience design, and usability; conservation and preservation, including digital preservation; librarianship and library administration; the sociology of information; and human–computer interaction.
Read more →
Energy informatics

Energy informatics is a research field covering the use of information and communication technology to address energy utilization and management challenges. Methods used for "smart" implementations often combine IoT sensors with artificial intelligence and machine learning. Energy Informatics is founded on flow networks that are the major suppliers and consumers of energy. Their efficiency can be improved by collecting and analyzing information. == Application areas == The field among other consider application areas within: Smart Buildings by developing ICT-centred solutions for improving the energy-efficiency of buildings. Smart Cities by investigating the synergies between demand patterns and supply availability of energy flows in cities and communities to improve energy efficiency, increase integration of renewable sources, and provide resilience towards system faults caused by extreme situations, like hurricanes and flooding. Smart Industries including the development of ICT-centred solutions for improving the energy efficiency and predictability of energy intensive industrial processes, without compromising process and product quality. Smart Energy Networks by developing ICT-centred solutions for coordinating the supply and demand in environmentally sustainable energy networks.
Read more →
Information scientist

The term information scientist developed in the latter part of the twentieth century by Wm. Hovey Smith to describe an individual, usually with a relevant subject degree (such as one in Information and Computer Science - CIS) or high level of subject knowledge, providing focused information to scientific and technical research staff in industry. It is a role quite distinct from and complementary to that of a librarian. Developments in end-user searching, together with some convergence between the roles of librarian and information scientist, have led to a diminution in its use in this context, and the term information officer or information professional (information specialist) are also now used. The term was, and is, also used for an individual carrying out research in information science. Brian C. Vickery mentions that the Institute of Information Scientists (IIS) was established in London during 1958 and lists the criteria put forward by this institute "Criteria for Information Science" (appendix 1) as well as his own "Areas of study in information science" (appendix 2). The IIS merged with the Library Association in 2002 to form the Chartered Institute of Library and Information Professionals (CILIP). == Notable Information Scientists == See also Award of Merit - Association for Information Science and Technology Marcia Bates David Blair (information technologist) Samuel C. Bradford Michael Buckland John M. Carroll Blaise Cronin Emilia Currás Brenda Dervin Eugene Garfield Paul B. Kantor Frederick Wilfrid Lancaster Calvin Mooers Tefko Saracevic Linda C. Smith Robert Saxton Taylor Brian Campbell Vickery Thomas D. Wilson == Additional reading == Ellis, David and Merete Haugan. (1997) "Modelling the information seeking patterns of engineers and research scientists in an industrial environment" (Journal of Documentation, Volume 53(4): pp. 384–403) Poole, Alex H. (2024). "'There's a big difference between going through life with the wind at your back, and going through life leaning into the wind': Feminism in Post-World War II Information Science". Proceedings of the Association for Information Science and Technology. 61: 300–313. doi:10.1002/pra2.1029. Vickery, Brian Campbell (1988) "Essays presented to B. C. Vickery" (Journal of Documentation, Volume 44, pp. 199–283). Vickery, B. & Vickery, A. (1987) Information Science in theory and practice (London: Bowker-Saur, pp. 361–369)
Read more →
Mojito (framework)

Mojito is an environment agnostic, Model-View-Controller (MVC) web application framework. It was designed by Yahoo. == Features == Mojito supports agile development of web applications. Mojito has built-in support for unit testing, Internationalization, syntax and coding convention checks. Both server and client components are written in JavaScript. Mojito allows developers designing web applications to leverage the utilities of both configuration and MVC framework. Mojito is capable of running on both JavaScript-enabled web browsers and servers using Node.js because they both utilize JavaScript. Mojito applications mainly consist of two components: JSON Configuration files: these define relationships between code components, assets, routing paths, and framework defaults and are available at the application and mojit level. Directories: these reflect MVC architecture and are used to separate resources such as assets, libraries, middleware, etc. == Architecture == In Mojito, both server and "client" side scripting is done in JavaScript, allowing it to run on both client and server thereby breaking the "front-end back-end barrier." It has both client and server runtimes. === Server runtime === This block houses operations needed by server side components. Services include: Routing rules, HTTP Server, config loader and disk-based loader. === Client runtime === This block houses operations called upon while running client sides components. Services include local storage/cache access and JSON based /URL based loader === Core === Core function can be accessed on client or server. Services include Registry, Dispatcher, Front controller, Resource store. === Container === mojit object comes into the picture. This container also include the services used by mojits. API and Mojito services are the blocks which caters to services needed for execution of mojits. === API (Action Context) === Mojito services are a customizable service block. It offers mojits a range of services which might be needed by mojit to carry out certain actions. These services can be availed at both client and server side. Reusable services can be created and aggregated to the core here. == Mojits == Mojits are the modules of a Mojito application. An application consists of one or more mojits. A mojit encompasses a Model, Views and a Controller defined by JSON configuration files. It includes a View factory where views are created according to the model and a View cache that holds frequently requested views to aid performance. === Application Architecture === A Mojito application is a set of mojits facilitated by configurable JSON files which define the code for model, view and controller. This MVC structure works with API block and Mojito services, and can be deployed at both client and server side. While the application is deployed at client side, it can call server-side modules using binders. Binders are mojit codes that let mojits request services from each other. Mojit Proxy acts as an intermediary between binders and mojit's API (application context) block and other mojits. Controllers are command-issuing units of mojits. Models mirror the core logic and hold data. Applications can have multiple models. They can be centrally accessed from controllers. View files are created in accordance with controllers and models, and are marked-up before they are sent to users as output. === Application Directory Structure === Directory structure of a Mojito application with one mojit: [mojito_app]/ |-- application.json |-- assets/ | `-- favicon.icon |-- yui_modules/ | `-- .{affinity}.js |-- index.js |-- mojits/ | `-- [mojit_name | |-- assets/ | |-- yui_modules/ | | `-- .{affinity}.js | |-- binders/ | | `-- {view_name}.js | |-- controller.{affinity}.js | |-- defaults.json | |-- definition.json | |-- lang/ | | `-- {mojit_name}_{lang}.js | |-- models/ | | `-- {model_name}.{affinity}.js | |-- tests/ | | |-- yui_modules/ | | | `-- {module_name}.{affinity}-tests.js | | |-- controller.{affinity}-tests.js | | `-- models/ | | `-- {model_name}.{affinity}-tests.js | `-- views/ | |-- {view_name}.{view_engine}.html | `-- {view_name}.{device}.{view_engine}.html |-- package.json |-- routes.json (deprecated) |-- server.js == Model, View and Controller == The Model hosts data, which is accessed by the Controller and presented to the View. Controller also handles any client requests for data, in which case controller fetches data from the model and passes the data to the client. All three components are clustered in the mojit. Mojits are physically illustrated by directory structures and an application can have multiple mojits. Every mojit can have one controller, one or more views and zero or more models. === Model === The model it represents the application data and is independent of view or controller. Model contains code to manipulate the data. They are found in the models directory of each mojit. Functions include: Storing information for access by controller. Validation and error handling. Metadata required by the view === Controller === The controller acts like a connecting agent between model and view. It supplies input to Model and after fetching data from model, passes it to View. Functions include Redirection Monitors authentication Web safety Encoding === View === The view acts as a presentation filter by highlighting some model attributes and suppressing others. A view can be understood as a visual permutation of the model. The view renders data received from controller and displays it to the end user.
Read more →
Two-phase commit protocol

In transaction processing, databases, and computer networking, the two-phase commit protocol (2PC, tupac) is a type of atomic commitment protocol (ACP). It is a distributed algorithm that coordinates all the processes that participate in a distributed atomic transaction on whether to commit or abort (roll back) the transaction. This protocol (a specialised type of consensus protocol) achieves its goal even in many cases of temporary system failure (involving either process, network node, communication, etc. failures), and is thus widely used. However, it is not resilient to all possible failure configurations, and in rare cases, manual intervention is needed to remedy an outcome. To accommodate recovery from failure (automatic in most cases) the protocol's participants use logging of the protocol's states. Log records, which are typically slow to generate but survive failures, are used by the protocol's recovery procedures. Many protocol variants exist that primarily differ in logging strategies and recovery mechanisms. Though usually intended to be used infrequently, recovery procedures compose a substantial portion of the protocol, due to many possible failure scenarios to be considered and supported by the protocol. In a "normal execution" of any single distributed transaction (i.e., when no failure occurs, which is typically the most frequent situation), the protocol consists of two phases: The commit-request phase (or voting phase), in which a coordinator process attempts to prepare all the transaction's participating processes (named participants, cohorts, or workers) to take the necessary steps for either committing or aborting the transaction and to vote, either "Yes": commit (if the transaction participant's local portion execution has ended properly), or "No": abort (if a problem has been detected with the local portion), and The commit phase, in which, based on voting of the participants, the coordinator decides whether to commit (only if all have voted "Yes") or abort the transaction (otherwise), and notifies the result to all the participants. The participants then follow with the needed actions (commit or abort) with their local transactional resources (also called recoverable resources; e.g., database data) and their respective portions in the transaction's other output (if applicable). The two-phase commit (2PC) protocol should not be confused with the two-phase locking (2PL) protocol, a concurrency control protocol. == Assumptions == The protocol works in the following manner: one node is a designated coordinator, which is the master site, and the rest of the nodes in the network are designated the participants. The protocol assumes that: there is stable storage at each node with a write-ahead log, no node crashes forever, the data in the write-ahead log is never lost or corrupted in a crash, and any two nodes can communicate with each other. The last assumption is not too restrictive, as network communication can typically be rerouted. The first two assumptions are much stronger; if a node is totally destroyed then data can be lost. The protocol is initiated by the coordinator after the last step of the transaction has been reached. The participants then respond with an agreement message or an abort message depending on whether the transaction has been processed successfully at the participant. == Basic algorithm == === Commit request (or voting) phase === The coordinator sends a query to commit message to all participants and waits until it has received a reply from all participants. The participants execute the transaction up to the point where they will be asked to commit. They each write an entry to their undo log and an entry to their redo log. Each participant replies with: either an agreement message (participant votes Yes to commit), if the participant's actions succeeded; or an abort message (participant votes No to commit), if the participant experiences a failure that will make it impossible to commit. === Commit (or completion) phase === ==== Success ==== If the coordinator received an agreement message from all participants during the commit-request phase: The coordinator sends a commit message to all the participants. Each participant completes the operation, and releases all the locks and resources held during the transaction. Each participant sends an acknowledgement to the coordinator. The coordinator completes the transaction when all acknowledgements have been received. ==== Failure ==== If any participant votes No during the commit-request phase (or the coordinator's timeout expires): The coordinator sends a rollback message to all the participants. Each participant undoes the transaction using the undo log, and releases the resources and locks held during the transaction. Each participant sends an acknowledgement to the coordinator. The coordinator undoes the transaction when all acknowledgements have been received. ==== Message flow ==== Coordinator Participant QUERY TO COMMIT --------------------------------> VOTE YES/NO prepare/abort <------------------------------- commit/abort COMMIT/ROLLBACK --------------------------------> ACKNOWLEDGEMENT commit/abort <-------------------------------- end An next to the record type means that the record is forced to stable storage. == Disadvantages == The greatest disadvantage of the two-phase commit protocol is that it is a blocking protocol. If the coordinator fails permanently, some participants will never resolve their transactions: After a participant has sent an agreement message as a response to the commit-request message from the coordinator, it will block until a commit or rollback is received. A two-phase commit protocol cannot dependably recover from a failure of both the coordinator and a cohort member during the commit phase. If only the coordinator had failed, and no cohort members had received a commit message, it could safely be inferred that no commit had happened. If, however, both the coordinator and a cohort member failed, it is possible that the failed cohort member was the first to be notified, and had actually done the commit. Even if a new coordinator is selected, it cannot confidently proceed with the operation until it has received an agreement from all cohort members, and hence must block until all cohort members respond. == Implementing the two-phase commit protocol == === Common architecture === In many cases the 2PC protocol is distributed in a computer network. It is easily distributed by implementing multiple dedicated 2PC components similar to each other, typically named transaction managers (TMs; also referred to as 2PC agents or Transaction Processing Monitors), that carry out the protocol's execution for each transaction (e.g., The Open Group's X/Open XA). The databases involved with a distributed transaction, the participants, both the coordinator and participants, register to close TMs (typically residing on respective same network nodes as the participants) for terminating that transaction using 2PC. Each distributed transaction has an ad hoc set of TMs, the TMs to which the transaction participants register. A leader, the coordinator TM, exists for each transaction to coordinate 2PC for it, typically the TM of the coordinator database. However, the coordinator role can be transferred to another TM for performance or reliability reasons. Rather than exchanging 2PC messages among themselves, the participants exchange the messages with their respective TMs. The relevant TMs communicate among themselves to execute the 2PC protocol schema above, "representing" the respective participants, for terminating that transaction. With this architecture the protocol is fully distributed (does not need any central processing component or data structure), and scales up with number of network nodes (network size) effectively. This common architecture is also effective for the distribution of other atomic commitment protocols besides 2PC, since all such protocols use the same voting mechanism and outcome propagation to protocol participants. === Protocol optimizations === Database research has been done on ways to get most of the benefits of the two-phase commit protocol while reducing costs by protocol optimizations and protocol operations saving under certain system's behavior assumptions. ==== Presumed abort and presumed commit ==== Presumed abort or Presumed commit are common such optimizations. An assumption about the outcome of transactions, either commit, or abort, can save both messages and logging operations by the participants during the 2PC protocol's execution. For example, when presumed abort, if during system recovery from failure no logged evidence for commit of some transaction is found by the recovery procedure, then it assumes that the transaction has been aborted, and acts accordingly. This means that it does not matter if aborts are logged at all, and such logging can be saved under this assumption. Typical
Read more →
Block swap algorithms

In computer algorithms, block swap algorithms swap two regions of elements of an array. It is simple to swap two non-overlapping regions of an array of equal size. However, it is not as simple to swap two contiguous regions of an array of unequal sizes (algorithms that perform such swapping are called rotation algorithms). A few well-known algorithms can accomplish this: Bentley's juggling (also known as the dolphin algorithm), Gries-Mills rotation, triple reversal algorithm, conjoined triple reversal algorithm (also known as the trinity rotation) and Successive rotation. == Triple reversal algorithm == The triple reversal algorithm is the simplest to explain, using rotations. A rotation is an in-place reversal of array elements. This method swaps two elements of an array from outside in within a range. The rotation works for an even or odd number of array elements. The reversal algorithm uses three in-place rotations to accomplish an in-place block swap: Rotate region A Rotate region B Rotate region AB Where A and B are adjacent regions of an array that together form the region AB. Gries-Mills and reversal algorithms perform better than Bentley's juggling, because of their cache-friendly memory access pattern behavior. The triple reversal algorithm parallelizes well, because rotations can be split into sub-regions, which can be rotated independently of others.
Read more →
Australian Geoscience Data Cube

The Australian Geoscience Data Cube (AGDC) is an approach to storing, processing and analyzing large collections of Earth observation data. The technology is designed to meet challenges of national interest by being agile and flexible with vast amounts of layered grid data. The AGDC reduces processing time of traditional image analysis by calibrating, pre-computing known extents, pixel alignment and storing metadata in a cell lattice structure. The temporal-pixel aligned data can often be analysed faster across space and time dimensions than previous scene based techniques. This allows the AGDC to be flexible in tackling future challenges and improve analysis times on every-increasing data repositories of earth observation. The AGDC has also been used internationally to allow countries to maintain ecologically sustainable programs and reduce the difficulty curve of utilizing Remote Sensing data. == Background == The AGDC was originally conceived by Geoscience Australia but is now maintained in a partnership between Geoscience Australia, Commonwealth Scientific and Industrial Research Organisation (CSIRO) and National Computational Infrastructure National Facility (Australia) (NCI). This is made possible by the funding from the partnership and a number of organisations such as National Collaborative Research Infrastructure Strategy (NCRIS). == Analysis ready data, ingestion and indexing == The data processed in the cube is made analysis ready before being ingested and indexed into the AGDC. Analysis ready data is pre-processed data that has applied corrections for instrument calibration (gains and offsets), geolocation (spatial alignment) and radiometry (solar illumination, incidence angle, topography, atmospheric interference). The ingestion process manages the translation of datasets into the storage units while maintaining a database index. The data within the storage and index can be accessed via API calls often compiled within code such as Python (programming language). Example: s2a_l1c = dc.load(product='s2a_level1c_granule',x=(147.36, 147.41), y=(-35.1, -35.15), measurements=['04','03','02'], output_crs='EPSG:4326', resolution=(-0.00025,0.00025)) === Datasets currently stored === Geoscience Australia Landsat Surface Reflectance (1987 to present) Landsat Pixel Quality Landsat Fractional Cover Landsat NDVI === Datasets that have been piloted === USGS Landsat Surface Reflectance SRTM DEM Himawari 8 MODIS Sentinel-2 L1C / S2A Australian Gridded Climate Data == Open source == The AGDC code base is situated in GitHub as an open repository. The core code base moved to the Open Data Cube in early 2017 as part of an international collaboration. Whilst the code base is the Open Data Cube, individual cubes exist as their own right such as the AGDC on the National Computational Infrastructure National Facility (Australia) (NCI) using the High-Performance Computing Cluster HPCC. The core code can be installed on personal computers or public computers (using git) and has many unit tests. Documentation for the code base exists on Read the Docs. == Challenges of the AGDC == The AGDC is designed to meet nationally significant challenges such as the following. Sustainability Environment Water resource management Disaster assist Policy development Community planning Forest preservation Carbon measurement == International awards == The AGDC won the 2016 Content Platform of the Year award from Geospatial World Forum.
Read more →
Aphelion (software)

The Aphelion Imaging Software Suite is a software suite that includes three base products - Aphelion Lab, Aphelion Dev, and Aphelion SDK for addressing image processing and image analysis applications. The suite also includes a set of extension programs to implement specific vertical applications that benefit from imaging techniques. The Aphelion software products can be used to prototype and deploy applications, or can be integrated, in whole or in part, into a user's system as processing and visualization libraries whose components are available as both DLLs or .Net components. == History and evolution == The development of Aphelion started in 1995 as a joint project of a French company, ADCIS S.A., and an American company, Amerinex Applied Imaging, Inc. (AAI) Aphelion's image processing and analysis functions were made from operators available from the KBVision software developed and sold by Amerinex's predecessor, Amerinex Artificial Intelligence Inc. In the 1990s, the XLim software library was developed at the Center of Mathematical Morphology of Mines ParisTech, and both companies carried out its development tasks. The first version of Aphelion was completed and released in April 1996. Successive versions were released before the first official stable release in December 1996 at the Photonics East conference in Boston and the Solutions Vision show in Paris in January 1997, where at the latter it competed with Stemmer Imaging's CVB imaging toolbox. In 1998, version 2.3 of Aphelion for Windows 98 was released, and its user base was growing in both France and the United States. Version 3.0, totally rewritten to take advantage of Microsoft's then-recent ActiveX technology, was officially released in 2000. It also became available as a « Developer » version, for rapid prototyping of applications using its intuitive GUI and the macro recording capability, and a « Core » version, including the full library as a set of ActiveX components to be used by software developers, integrators and original equipment manufacturers (OEM). As AAI turned its focus to security, in 2001, ADCIS took the lead on developing Aphelion. AAI focused on millimeter wave scanners for concealed weapon detection at airports, and eventually merged with Millimetrics to become Millivision. In 2004, ADCIS specified version 4.0 of Aphelion. The set of image processing/analysis functions was rewritten one more time to be compatible with the .NET technology and the emergence of 64 bit architecture PCs. In addition, the GUI was redesigned to address two usage types: a semi-automatic use where the user is guided through the different steps of functions, and a fully automatic use where the expert user can quickly invoke imaging functions. Its first release was presented at the IPOT exhibition in Birmingham, UK the same year. During the Vision Show in Paris in October 2008, the new Aphelion Lab product was launched for users that are not specialists in image processing. It is easier to use, and only includes fewer image processing functions. It was then included in the Aphelion Image Processing Suite, consisting of Aphelion Dev (replacing Aphelion Developer), Aphelion Lab, Aphelion SDK (replacing Aphelion Core), and a set of extensions. Nowadays, ADCIS is still working on the suite, and updated versions with new extensions and functionalities continually become available from the websites of both companies. In 2015, support was added for very large images and scan microscope images (virtual slides compound into a very large JPEG 2000 image) for high throughput imaging, and new specific extensions were also added. In late 2015, ADCIS announced Aphelion's port for tablets and smartphones, for vertical applications. The name "Aphelion" comes from the astronomical term of the same name, meaning the point on a planet rotating around the Sun where it lies farthest from it, applying the term in a metaphorical sense. Unix was the operating system used on scientific workstations in the 1990s, such as on the workstations manufactured by market leader Sun Microsystems, which Windows suite Aphelion was quite removed from. == Description == Aphelion is a software suite to be used for image processing and image analysis. It supports 2D and 3D, monochrome, color, and multi-band images. It is developed by ADCIS, a French software house located in Saint-Contest, Calvados, Normandy. Aphelion is widely used in the scientific/industry community to solve basic and complex imaging applications. First, the imaging application is quickly developed from the Graphical User Interface, involving a set of functions that can be automatically recorded into a macro command. The macro languages available in Aphelion (i.e. BasicScript, Python, and C#) help to process batch of images, and prompt the user if needed for specific parameters that are applied to the imaging functions. All Aphelion image processing functions are written in C++, and the Aphelion user interface is written in C#. C++ functions can be called from the C# language thanks the use of dedicated wrappers. The main principle of image processing is to automatically process pixels of a digital image, then extract one or more objects of interest (i.e. cells in the field of biology, inclusions in the field of material science) and compute one or more measurements on those objects to quantify the image and generate a verdict (good image, image with defects, cancerous cells). In other words, starting from an image, pixels are processed by a set of successive functions or operators until only measurements are computed and used as the input of a 3rd party system or a classification software that will classify objects of interest that have been extracted during the imaging process. An acquisition system such as a digital camera, a video camera, an optical or electron microscope, a medical scanner, or a smartphone can be used to capture images. The set of values or pixels can be processed as a 1D image (1D signal), a 2D image (array of pixel values corresponding to a monochrome or color image), or a 3D image displayed using volume rendering (array of voxels in the 3D space) or displaying surfaces by using 3D rendering. A 2D color image is made of 3 value pixels (typically Red, Green, and Blue information or another color space), and a 3D image is made of monochrome, color (indexed color are often used), multispectral, or hyperspectral data. When dealing with videos, an additional band is added corresponding to temporal information. The Aphelion Software Suite includes three base products, and a set of optional extensions for specific applications: Aphelion Lab: Entry-level package for non-experts in image processing. It helps to quickly segment an image in a semi-automatic or manual ways, and compute a set of measurements computed on objects of interest that have been extracted during the segmentation process. A set of wizards guides the user from image acquisition to report generation. Aphelion Dev: Full imaging environment including over 450 functions to develop and deploy an application that involves image processing and analysis. It also includes a set of macro-command languages to automate any application to be invoked from the user interface. It also helps to run the imaging algorithm on more than one image that are stored on disk, available on the network, or captured by an acquisition device. Aphelion libraries for image processing and visualization are provided in Aphelion Dev as DLLs and .Net components. Aphelion SDK: A set of libraries to develop a stand-alone application with a custom interface based on the Aphelion libraries. This software development kit including display, processing and analysis functions that can be used by software developers and OEMs. It is provided as DLLs and .Net components. The stand-alone application is typically developed in C# on one computer, and then deployed on multiple PCs and systems. A set of optional extensions can be added to the « Aphelion Dev » product, depending on the application. An evaluation version of Aphelion can be run on a PC for 30 days. A permanent version of Aphelion is available based on a perpetual license. Upgrades are available through a maintenance agreement based on a yearly fee. Technical support is provided by the engineers who are developing the product. The goal of image processing is usually to extract object(s) of interest in an image, and then to classify them based on some characteristics such as shape, density, position, etc. Using Aphelion, this goal is achieved by performing the following tasks: Load an image from disk or acquire an image using an acquisition device. Enhance the image removing noise or modifying its contrast. Segment the image extracting objects of interest to be measured and analyzed. Typically, for simple applications, a threshold is performed to generate a binary image. Then, morphological operators are applied to clean the image and only keep obj
Read more →
Enterprise bus matrix

The enterprise bus matrix is a data warehouse planning tool and model created by Ralph Kimball, and is part of the data warehouse bus architecture. The matrix is the logical definition of one of the core concepts of Kimball's approach to dimensional modeling conformed dimension. The bus matrix defines part of the data warehouse bus architecture and is an output of the business requirements phase in the Kimball lifecycle. It is applied in the following phases of dimensional modeling and development of the data warehouse. The matrix can be categorized as a hybrid model, being part technical design tool, part project management tool and part communication tool == Background == The need for an enterprise bus matrix stems from the way one goes about creating the overall data warehouse environment. Historically there have been two approaches: a structured, centralized and planned approach and a more loosely defined, department specific approach, in which solutions are developed in a more independent matter. Autonomous projects can result in a range of isolated stove pipe data marts. Naturally each approach has its issues; the visionary approach often struggles with long delivery cycles and lack of reaction time as needs emerge and scope issues arise. On the other hand, the development of isolated data marts leads to stovepipe systems that lack synergy in development. Over time this approach will lead to a so-called data-mart-in-a-box architecture where interoperability and lack of cohesion is apparent, and can hinder the realization of an overall enterprise data warehouse. As an attempt to handle this issue, Ralph Kimball introduced the enterprise bus. == Description == The bus matrix purpose is one of high abstraction and visionary planning on the data warehouse architectural level. By dictating coherency in the development and implementation of an overall data warehouse the bus architecture approach enables an overall vision of the broader enterprise integration and consistency while at the same time dividing the problem into more manageable parts – all in a technology and software independent manner. The bus matrix and architecture builds upon the concept of conformed dimensions, creating a structure of common dimensions that ideally can be used across the enterprise by all business processes related to the data warehouse and the corresponding fact tables from which they derive their context. According to Kimball and Margy Ross's article “Differences of Opinion” "The Enterprise Data warehouse built on the bus architecture ”identifies and enforces the relationship between business process metrics (facts) and descriptive attributes (dimensions)”. The concept of a bus is well known in the language of information technology, and is what reflects the conformed dimension concept in the data warehouse, creating the skeletal structure where all parts of a system connect, ensuring interoperability and consistency of data, and at the same time considers future expansion. This makes the conformed dimensions act as the integration ‘glue’, creating a robust backbone of the enterprise Data Warehouse.
Read more →
Personal, Inc.

Personal (also referred to as Personal.com or Personal, Inc.) was a consumer personal data service and identity management system for individuals to aggregate, manage and reuse their own data. It merged with digi.me in August 2017, a business in Europe that has the same business model. The combined company is called digi.me. One of its product lines, a collaborative data management and information security solution for the workplace called TeamData, was spun off as a new company as a result of the merger. == History == Personal was founded in 2009 in Washington, DC by the management team that built The Map Network, a location data and mapping platform that was acquired by Nokia/NAVTEQ in 2006. Personal was the first online consumer-facing company to be named an Ambassador for Privacy by Design for its technical, business and legal commitments to providing users with control over the data they store in Personal's service. Called a “life management platform” by The Economist and a “personal encrypted cloud service” by TIME for its user-centric approach to data, the company has been associated with both the Infomediary model originated in 1999 by John Hagel III and Mark Singer, as well as the vendor relationship management (VRM) model developed by Doc Searls. Personal raised $30m in funding to develop its platform and products from such leading investors as Steve Case's Revolution Ventures, Grotech Ventures, Allen & Company, Ted Leonsis, Neil Ashe, Jonathan Miller, Bill Miller of Legg Mason, Esther Dyson of EDventures, and Eric C. Anderson. The company received recognition for its user agreement, called the Owner Data Agreement, which acted like a reverse license agreement when data was shared between registered parties and emphasized that data ownership resides with the user. Doc Searls wrote in The Intention Economy: When Customers Take Charge that the Owner Data Agreement “had no precedent and modeled a new legal position, both for vendors and for intermediaries.” Personal was early to embrace “small data,” which it defines as “big data for the benefit of individuals.” The term “small data” may have been originally coined by Jeremie Miller of Sing.ly, who mentioned it in a talk at the Web 2.0 Summit in November 2011 and is cited in The Intention Economy. In 2011, Personal was a part of the first group of companies to join the Personal Data Ecosystem Consortium's Startup Circle. A Small Data Meetup group has also formed in New York City, bringing together technology, legal and business experts to exchange ideas about user-centric and user-driven models for internet products and services. Personal has been included in case studies by Ctrl-Shift and Forrester regarding Personal Data Stores and Personal Identity Management. In 2011, Personal received the Innovator Spotlight Award at Privacy Identity Innovation Conference (pii2011) and participated in the Technology Showcase at pii2012. In 2012, TechHive named Personal as one of the top five apps or web services of SXSW. Personal won the 2013 Campus Technology Innovators Award with Lone Star College in July 2013. Personal was included in a list of Executive Travel Magazine's favorite travel apps for 2013 in its May/June issue. In 2013, Personal was also included as part of NYU GovLab's Open Data 500 and was named by J. Walter Thompson as one of 100 things to watch for in 2014. In 2015, the National Law Journal named Company Chief Policy Officer and General Counsel, Joshua P. Galper, as one of their 50 "Cybersecurity & Privacy Trailblazers." == Products and services == === Overview === The Personal Platform was a privacy- and security-by-design platform for individuals to manage and reuse their own data and information. The Fill It app was a 1-click form-filling solution for web and mobile logins, checkouts and forms, and the Data Vault app served as the main cloud-based repository for a user's data. Personal helped individuals take control and benefit from their information while knowing that the information in their Data Vault remained legally theirs and could not be used without their permission. === Data Vault with Cloud Sync === Personal spent two years building the Personal Platform before launching its Data Vault product in beta in November 2011. Following Privacy by Design principles, Personal only enabled users to see or share the sensitive data and all the files they stored in their Data Vault. Such information was encrypted, and could only be decrypted with a user's password. Only users could choose and know their passwords to their vault because Personal did not store user passwords – and therefore could not reset them without deleting a user's sensitive data and all files stored in their vault. All Personal apps and services were linked to a user's private Data Vault. The Data Vault featured automatic synchronization of data and files added on any device logged into Personal. It also featured a “Secure Share” function that created a live, private network, allowing registered users to share access to data and files through an exchange of encrypted keys without the risk of transmitting the data or files through non-secure, direct means. It also allowed users to immediately update data across their own network and revoke access to it when they choose. Fast Company called the Data Vault “a tool that will simplify our lives.” Personal launched its Android app on November 30, 2011. The iOS Data Vault app was released on May 7, 2012. Personal officially launched its application programming interface (APIs) on October 2, 2012 at the Mashery Business of APIs Conference. A review by CNET highlighted the challenges of getting people to trust such a new service with their sensitive data and spending the time required entering enough data to make it useful. === Fill It App and Form Index === When the Data Vault was launched in November 2011, Mashable posed the question: “Never Fill Out a Form Again?” The World Economic Forum in its February 2013 report highlighted the possibility of saving 10 billion hours globally “and improv[ing] the delivery of public and private sector services” through automated form-filling tools, specifically citing Personal's Fill It app. In January 2013, Personal launched Fill It in beta as a web bookmarklet for automatic form-filling. On June 11, 2014, Personal released Fill It as a web extension and announced that it was publishing an index of over 140,000 1-click online forms at www.fillit.com. The company also announced that a mobile version of the product will launch later in the year. According to a story in Tech Cocktail about the launch, Personal's “web extension and mobile app are able to support over 1,200 different types of reusable data, even enabling them to unlock more confidential information so they can complete longer forms, including patient registrations, job applications, event registrations, school admissions, insurance and bank applications, and government forms.” In November 2014, a mobile version of Fill It was launched that could autofill mobile forms using APIs. Personal's form portal ultimately indexed more than 500,000 forms with three components, which, together, allowed data to be captured and reused across any of the forms: (1) a form graph, which mapped individual form fields to the Personal ontology; (2) a semantic layer, which determined how data was required on a form (e.g. one field vs. three fields for a U.S. telephone number); and (3) a correlations graph, which helped individuals match their specific data to a form without looking at the data value (e.g. knowing which phone number is a mobile phone number, which address is a billing address, or that a person uses their middle name as a first name on most forms). === Monetizing personal data === With the initial public offering of Facebook in May 2012, there was media interest in the question of the monetary value of personal data and whether tools and services might emerge to help consumers monetize their own data. Personal was frequently cited as a company that could potentially offer such a service. Articles and pieces focusing on this subject have appeared in The New York Times, AdWeek, the MIT Technology Review, and on CNN and National Public Radio. Company Co-founder and CEO Shane Green was quoted as saying that “the average American consumer would soon be able to realize over $1,000 per year” by granting limited, anonymous access to their data to marketers, but that figure was never supported by Green or the company. === Launch of TeamData === In May 2016, Personal shifted its product focus to TeamData, which focuses on the problem of securing and collaboratively managing data in the workplace. It is now a separate business.
Read more →