DataViva is an information visualization engine created by the Strategic Priorities Office of the government of Minas Gerais. DataViva makes official data about exports, industries, locations and occupations available for the entirety of Brazil through eight apps and more than 100 million possible visualizations. The first set of datum – also available at ALICEWEB – is provided by MDIC (Ministry of Development, Industry and Foreign Trade) / SECEX (Secretariat of Foreign Trade), an official institution of the Government of Brazil and shows foreign trade statistics for all exporting municipalities in the country. The other database, provided by Ministério do Trabalho e Emprego (MTE – Ministry of Labor and Employment), shows information about all the industries and occupations in Brazil (RAIS – Annual Social Information Report). The platform consists of eight core applications, each of which allows different ways of visualizing the data available. Some applications are descriptive, that is, showing data aggregated at various levels in a simple and comparative way, such as Treemapping. Others are prescriptive, using calculations that allow an analytic visualization of the data, based on theories such as the Product Space. All the applications are generated using D3plus, an open source JavaScript library built on top of D3.js by Alexander Simoes and Dave Landry. Inspired by The Observatory of Economic Complexity, DataViva is an open data, open-source, and free to use tool. It was developed in a partnership with Datawheel, co-founded by MIT Media Lab Professor César Hidalgo, and is maintained by the Government of Minas Gerais.
Inductive probability
Inductive probability attempts to give the probability of future events based on past events. It is the basis for inductive reasoning, and gives the mathematical basis for learning and the perception of patterns. It is a source of knowledge about the world. There are three sources of knowledge: inference, communication, and deduction. Communication relays information found using other methods. Deduction establishes new facts based on existing facts. Inference establishes new facts from data. Its basis is Bayes' theorem. Information describing the world is written in a language. For example, a simple mathematical language of propositions may be chosen. Sentences may be written down in this language as strings of characters. But in the computer it is possible to encode these sentences as strings of bits (1s and 0s). Then the language may be encoded so that the most commonly used sentences are the shortest. This internal language implicitly represents probabilities of statements. Occam's razor says the "simplest theory, consistent with the data is most likely to be correct". The "simplest theory" is interpreted as the representation of the theory written in this internal language. The theory with the shortest encoding in this internal language is most likely to be correct. == History == Probability and statistics was focused on probability distributions and tests of significance. Probability was formal, well defined, but limited in scope. In particular its application was limited to situations that could be defined as an experiment or trial, with a well defined population. Bayes's theorem is named after Rev. Thomas Bayes 1701–1761. Bayesian inference broadened the application of probability to many situations where a population was not well defined. But Bayes' theorem always depended on prior probabilities, to generate new probabilities. It was unclear where these prior probabilities should come from. Ray Solomonoff developed algorithmic probability which gave an explanation for what randomness is and how patterns in the data may be represented by computer programs, that give shorter representations of the data circa 1964. Chris Wallace and D. M. Boulton developed minimum message length circa 1968. Later Jorma Rissanen developed the minimum description length circa 1978. These methods allow information theory to be related to probability, in a way that can be compared to the application of Bayes' theorem, but which give a source and explanation for the role of prior probabilities. Marcus Hutter combined decision theory with the work of Ray Solomonoff and Andrey Kolmogorov to give a theory for the Pareto optimal behavior for an Intelligent agent, circa 1998. === Minimum description/message length === The program with the shortest length that matches the data is the most likely to predict future data. This is the thesis behind the minimum message length and minimum description length methods. At first sight Bayes' theorem appears different from the minimimum message/description length principle. At closer inspection it turns out to be the same. Bayes' theorem is about conditional probabilities, and states the probability that event B happens if firstly event A happens: P ( A ∧ B ) = P ( B ) ⋅ P ( A | B ) = P ( A ) ⋅ P ( B | A ) {\displaystyle P(A\land B)=P(B)\cdot P(A|B)=P(A)\cdot P(B|A)} becomes in terms of message length L, L ( A ∧ B ) = L ( B ) + L ( A | B ) = L ( A ) + L ( B | A ) . {\displaystyle L(A\land B)=L(B)+L(A|B)=L(A)+L(B|A).} This means that if all the information is given describing an event then the length of the information may be used to give the raw probability of the event. So if the information describing the occurrence of A is given, along with the information describing B given A, then all the information describing A and B has been given. ==== Overfitting ==== Overfitting occurs when the model matches the random noise and not the pattern in the data. For example, take the situation where a curve is fitted to a set of points. If a polynomial with many terms is fitted then it can more closely represent the data. Then the fit will be better, and the information needed to describe the deviations from the fitted curve will be smaller. Smaller information length means higher probability. However, the information needed to describe the curve must also be considered. The total information for a curve with many terms may be greater than for a curve with fewer terms, that has not as good a fit, but needs less information to describe the polynomial. === Inference based on program complexity === Solomonoff's theory of inductive inference is also inductive inference. A bit string x is observed. Then consider all programs that generate strings starting with x. Cast in the form of inductive inference, the programs are theories that imply the observation of the bit string x. The method used here to give probabilities for inductive inference is based on Solomonoff's theory of inductive inference. ==== Detecting patterns in the data ==== If all the bits are 1, then people infer that there is a bias in the coin and that it is more likely also that the next bit is 1 also. This is described as learning from, or detecting a pattern in the data. Such a pattern may be represented by a computer program. A short computer program may be written that produces a series of bits which are all 1. If the length of the program K is L ( K ) {\displaystyle L(K)} bits then its prior probability is, P ( K ) = 2 − L ( K ) {\displaystyle P(K)=2^{-L(K)}} The length of the shortest program that represents the string of bits is called the Kolmogorov complexity. Kolmogorov complexity is not computable. This is related to the halting problem. When searching for the shortest program some programs may go into an infinite loop. ==== Considering all theories ==== The Greek philosopher Epicurus is quoted as saying "If more than one theory is consistent with the observations, keep all theories". As in a crime novel all theories must be considered in determining the likely murderer, so with inductive probability all programs must be considered in determining the likely future bits arising from the stream of bits. Programs that are already longer than n have no predictive power. The raw (or prior) probability that the pattern of bits is random (has no pattern) is 2 − n {\displaystyle 2^{-n}} . Each program that produces the sequence of bits, but is shorter than the n is a theory/pattern about the bits with a probability of 2 − k {\displaystyle 2^{-k}} where k is the length of the program. The probability of receiving a sequence of bits y after receiving a series of bits x is then the conditional probability of receiving y given x, which is the probability of x with y appended, divided by the probability of x. ==== Universal priors ==== The programming language affects the predictions of the next bit in the string. The language acts as a prior probability. This is particularly a problem where the programming language codes for numbers and other data types. Intuitively we think that 0 and 1 are simple numbers, and that prime numbers are somehow more complex than numbers that may be composite. Using the Kolmogorov complexity gives an unbiased estimate (a universal prior) of the prior probability of a number. As a thought experiment an intelligent agent may be fitted with a data input device giving a series of numbers, after applying some transformation function to the raw numbers. Another agent might have the same input device with a different transformation function. The agents do not see or know about these transformation functions. Then there appears no rational basis for preferring one function over another. A universal prior insures that although two agents may have different initial probability distributions for the data input, the difference will be bounded by a constant. So universal priors do not eliminate an initial bias, but they reduce and limit it. Whenever we describe an event in a language, either using a natural language or other, the language has encoded in it our prior expectations. So some reliance on prior probabilities are inevitable. A problem arises where an intelligent agent's prior expectations interact with the environment to form a self reinforcing feed back loop. This is the problem of bias or prejudice. Universal priors reduce but do not eliminate this problem. === Universal artificial intelligence === The theory of universal artificial intelligence applies decision theory to inductive probabilities. The theory shows how the best actions to optimize a reward function may be chosen. The result is a theoretical model of intelligence. It is a fundamental theory of intelligence, which optimizes the agents behavior in, Exploring the environment; performing actions to get responses that broaden the agents knowledge. Competing or co-operating with another agent; games. Balancing short and long term rewards. In general no agent will always provi
Open Threat Exchange
Open Threat Exchange (OTX) is a crowd-sourced computer-security platform. It has more than 180,000 participants in 140 countries who share more than 19 million potential threats daily. It is free to use. Founded in 2012, OTX was created and is run by AlienVault (now AT&T Cybersecurity), a developer of commercial and open source solutions to manage cyber attacks. The collaborative threat exchange was created partly as a counterweight to criminal hackers successfully working together and sharing information about viruses, malware and other cyber attacks. == Components == OTX is cloud-hosted. Information sharing covers a wide range of security-related issues, including viruses, malware, intrusion detection and firewalls. Its automated tools cleanse, aggregate, validate and publish data shared by participants. The OTX platform validates the data, then strips the information identifying the participating contributor. In 2015, OTX 2.0 added a social network, enabling members to share, discuss and research security threats, including via a real-time threat feed. Users can share the IP addresses or websites from where attacks originated or look up specific threats to see if anyone has already left such information. Users can subscribe to a “Pulse,” an analysis of a specific threat, including data on IoC, impact, and the targeted software. Pulses can be exported as STIX, JSON, OpenloC, MAEC and CSV, and can be used to update local security products automatically. Users can up-vote and comment on specific pulses to assist others in identifying the most important threats. OTX combines social contributions with automated machine-to-machine tools that integrate with major security products such as firewalls and perimeter security hardware. The platform can read security reports in .pdf, .csv, .json and other open formats. Relevant information is extracted automatically, assisting IT professionals in analyzing data more readily. Specific OTX components include a dashboard with details about the top malicious IPs around the world and to check the status of specific IPs; notifications should an organization's IP or domain be found in a hacker forum, blacklist or be listed by OTX; and a feature to review log files to determine if there has been communication with known malicious IPs. In 2016, AlienVault released a new version of OTX, allowing participants to create private communities and discussion groups to share information on threats only within the group. The feature is intended to facilitate more in-depth discussions on specific threats, particular industries, and different regions worldwide. Threat data from groups can also be distributed to subscribers of managed service providers using OTX." == Technology == OTX is a large data platform that integrates natural language processing and machine learning. It uses these features to facilitate the collection and correlation of data from many sources, including third-party threat feeds, websites, external APIs and local agents. == Partners == In 2015, AlienVault partnered with Intel to coordinate real-time threat information on OTX. A similar deal with Hewlett Packard was announced the same year. == Competitors == Both Facebook and IBM have threat exchange platforms. The Facebook ThreatExchange is in beta and requires an application or invitation to join. IBM launched IBM X-Force Exchange in April 2015.
Showcase Workshop
Showcase Workshop, also referred to as Showcase, is a SaaS company that develops a presentation-building application for business use. Users upload files and images to a web platform which generates presentations viewable on a suite of mobile apps. Showcase was founded in 2011. The company’s headquarters are in Wellington, New Zealand. == History == Showcase Workshop was originally developed in response to dynamically changing content being presented on iPads at the 2012 Olympics. After market-testing a beta version of the core application, Showcase Workshop launched commercially in 2012. In 2014 Showcase partnered with Vodafone Global Enterprise. == Product == Users upload pre-existing PDFs, videos, images and Microsoft Office documents to a secure server, building presentations or ‘showcases’ which can then be downloaded via the mobile apps. The presentations are used for mobile sales enablement, training, or operational/health and safety purposes. == Reception == Reviewers have praised the ease of use of Showcase, calling it a “better alternative to developing a native app” and “intuitive”. Criticisms include the lack of differing templates and a lack of complex customisation controls. Showcase was nominated for a Tabby Award in 2014 and won a Tabby Award in 2015 for its Windows app.
PowerBuilder
PowerBuilder is an integrated development environment owned by SAP since the acquisition of Sybase in 2010. On July 5, 2016, SAP and Appeon entered into an agreement whereby Appeon, an independent company, would be responsible for developing, selling, and supporting PowerBuilder. Over the years, PowerBuilder has been updated with new standards. In 2010, a major upgrade of PowerBuilder was released to provide support for the Microsoft .NET Framework. In 2014, support was added for OData, dockable windows, and 64-bit native applications. In 2019 support was added for rapidly creating RESTful Web APIs and non-visual .NET assemblies using the C# language and the .NET Core framework. And PowerScript client app development was revamped with new UI technologies and cloud architecture. In 2025 the IDE was revamped with new code editor and ultra-fast compiler. Appeon has been releasing new features every 6-12 month cycles, which per the product roadmap focus on four key focus areas: sustaining core features, modernizing application UI, improving developer productivity, and incorporating more Cloud technology. == Features == PowerBuilder has a native data-handling component called a DataWindow, which can be used to create, edit, and display data from a database. This object gives the programmer a number of tools for specifying and controlling user interface appearance and behavior, and also provides simplified access to database content and JSON or XML from Web services. To some extent, the DataWindow frees the programmer from considering the differences between Database Management Systems from different vendors. DataWindow can display data using multiple presentation styles and can connect to various data sources. == Usage == PowerBuilder is used primarily for building business-oriented CRUD applications. Although new software products are rarely built with PowerBuilder, many client-server ERP products and line-of-business applications built in the late 1980s to early 2000s with PowerBuilder still provide core database functions for large enterprises in government, higher education, manufacturing, insurance, banking, energy, and telecommunications. == History == === Early history === PowerBuilder originated from Computer Solutions Inc. (CSI), a software consulting firm founded in 1974 by Mitchell Kertzman in Massachusetts. CSI developed GrowthPower, an MRP II software package with integrated financial modules released in 1981, which ran exclusively on the HP 3000 platform and achieved over 1,000 customer installations at its peak. In the late 1980s, as demand increased for graphical user interfaces amid the rise of Microsoft Windows, Kertzman partnered with Dave Litwack, former executive vice president of product development at Cullinet Software (acquired by Computer Associates in 1989). Litwack joined the company in 1988 as head of research and development to develop a client/server GUI tool, leading to its rebranding as Powersoft Corporation in 1990. PowerBuilder 1.0 was released in July 1991 as a rapid application development tool featuring the DataWindow and PowerScript language. Powersoft went public on February 3, 1993, with shares closing at $38 from an initial $20 price. Sybase announced its acquisition of Powersoft on November 15, 1994, in a stock swap valued at approximately $940 million; the merger closed on February 14, 1995, at a revised value of about $904 million due to Sybase's stock fluctuations. === Recent history === In December 2013 SAP announced the new version going directly to number 15 and released a beta version. Key features included support for the .NET Framework v4.5, SQL Server 2012, Oracle 12, Windows 8, OData and Dockable Windows. SAP later released this as version 12.6. On May 31, 2019, PowerBuilder 2019 was released by Appeon. This release supports C# development. It provides a new C# IDE, .NET data access objects, C# migration solution, Web API client, and UI themes. On April 3, 2020, PowerBuilder 2019 R2 was launched by Appeon. This release includes a first-ever PowerScript-to-C# code converter, which can automatically migrate 80-95% of PowerBuilder business logic and DataWindows to C#. Interoperability between PowerScript and .NET programming languages is also now supported. Many existing features have also been enhanced. On January 22, 2021, PowerBuilder 2019 R3 was launched by Appeon. This release provides a groundbreaking new app deployment technology called PowerClient, which securely automates the installation and update of client apps over HTTPS. C# Web API development has been greatly enhanced with asynchronous programming and support for Amazon Aurora and Azure cloud databases. Aside from many other new features, PowerBuilder 2019 R3 is a long-term support (LTS) version that replaces previous LTS versions On August 6, 2021, PowerBuilder 2021 was launched by Appeon. The Cloud deployment capability of the PowerBuilder 2021 IDE, in conjunction with the matching PowerServer 2021 runtime, was revamped, bringing PowerBuilder up-to-date with the latest .NET technologies. The presentation layer now executes PowerScript natively on Windows devices. The middle-tier has been rebuilt around REST API standard with a pure .NET Core implementation. A new CI/CD utility that integrates with Git/SVN and Jenkins, witch compiles all PowerBuilder projects using the command-line interface, was added alongside other features. On September 4, 2022, PowerBuilder 2022 was launched by Appeon. This release brings enhancements to the productivity of developing both client/server & installable cloud apps and more security measures to safeguard your apps. It includes many new features, including Windows 11 support, introducing time-saving functionalities to the IDE, such as Tabbed Code Editor, Jump to Objects, and Quick Code Search, and supports the latest HTTP/2 and TLS 1.3 protocols and two-way TLS authentication. On August 4, 2023, PowerBuilder 2022 R2 was launched by Appeon. This release introduces a range of new features aimed at helping developers build powerful, feature-rich, and secure client/server and installable cloud apps more efficiently, including tabbed windows, fillable PDFs, and SMTP client. On January 8, 2024, PowerBuilder 2022 R3 was launched by Appeon. This release is a long-term support version. Features previously released in earlier releases have been enhanced and/or corrected. On May 7, 2025, PowerBuilder 2025 was launched by Appeon. This release delivers a revamped IDE that boosts developer productivity throughout the SLDC—from writing and extending code to debugging, automating builds, and deploying applications. It features a new-generation code editor, ultra-fast compiler, automatic REST API creation, faster GIT operations, and codeless UI modernization features. == Features == PowerBuilder is an object-oriented programming language. Nearly all of the visual and non-visual objects support inheritance, polymorphism, and encapsulation. The programmer may utilize a common code framework such as PowerBuilder Foundation Classes, also known as PFC, to inherit objects from and leverage pre-existing code. The DataWindow is the key component (and selling point) of PowerBuilder. The DataWindow offers a visual SQL painter which supports outer joins, unions and subquery operations. It can convert SQL to visual representation and back, so the developer can use native SQL if desired. DataWindow updates are automatic — it produces the proper SQL at runtime based on the DBMS to which the user is currently connected. This feature makes it easier for developers who are not experienced with SQL. The DataWindow also has the built-in ability to both retrieve data and update data via stored procedures or REST Web APIs as well as import/export JSON data. The RESTClient object introduced in PowerBuilder 2017 facilitates bridging the DataWindow with REST Web APIs and requiring minimal coding. === RDBMS interfaces === PowerBuilder offers native interfaces to all major databases, as well as ODBC and OLE-DB, in the Enterprise version. There are many connectivity options that allow performance monitoring and tuning, such as: Integrated security Tracing of all SQL Isolation level Password expiration dialog Blocking factor Number of SQL statements to cache Use connection pool Thread safety Trace ODBC API calls Due to the information about the database schema (such as primary key information) that are stored in PowerBuilder's data dictionary, the code required to implement data display and browsing is greatly simplified, because the dictionary information allows generation of the appropriate SQL behind the scenes. PowerBuilder supports the following ways of interacting with a database: DataWindow this is the simplest approach, relying on automatically generated SQL. Embedded SQL Embedded SQL supports SELECT, INSERT, UPDATE, DELETE and cursors. This option is used when the developer desires more control than is available with the
Vanish (computer science)
Vanish was a project to "give users control over the lifetime of personal data stored on the web." It was led by Roxana Geambasu at the University of Washington. The project proposed to allow a user to enter information to send across the internet, thereby relinquishing control of it. However, the user can include an "expiration date," after which the information is no longer usable by anyone who may have a copy of it, even the creator. The Vanish approach was found to be vulnerable to a Sybil attack and thus insecure by a team called Unvanish from the University of Texas, University of Michigan, and Princeton. == Theory == Vanish acts by automating the encryption of information entered by the user with an encryption key that is unknown to the user. Along with the information the user enters, the user also enters metadata concerning how long the information should remain available. The system then encrypts the information but does not store either the encryption key or the original information. Instead, it breaks up the decryption key into smaller components that are disseminated across distributed hash tables, or DHTs, via the Internet. The DHTs refresh information within their nodes on a set schedule unless configured to make the information persistent. The time delay entered by the user in the metadata controls how long the DHTs should allow the information to persist, but once that time period is over, the DHTs will reuse those nodes, making the information about the decryption stored irretrievable. As long as the decryption key may be reassembled from the DHTs, the information is retrievable. However, once the period entered by the user has lapsed, the information is no longer recoverable, as the user never possessed the decryption key. == Implementation == Vanish currently exists as a Firefox plug-in which allows a user to enter text into either a standard Gmail email or Facebook message and choose to send the message via Vanish. The message is then encrypted and sent via the normal networking pathways through the cloud to the recipient. The recipient must have the same Firefox plug-in to decrypt the message. The plugin accesses BitTorrent DHTs, which have 8-hour lifespans. This means the user may select an expiration date for the message in increments of 8 hours. After the expiration of the user-defined time span, the information in the DHT is overwritten, thereby eliminating the key. While both the user and recipient may have copies of the original encrypted message, the key used to turn it back into plain text is now gone. Although this particular instance of the data has become inaccessible, it's important to note that the information can always be saved by other means before expiration (copied or even via screen shots) and published again.
Human visual system model
A human visual system model (HVS model) is used by image processing, video processing and computer vision experts to deal with biological and psychological processes that are not yet fully understood. Such a model is used to simplify the behaviors of what is a very complex system. As our knowledge of the true visual system improves, the model is updated. Psychovisual study is the study of the psychology of vision. The human visual system model can produce desired effects in perception and vision. Examples of using an HVS model include color television, lossy compression, and Cathode-ray tube (CRT) television. Originally, it was thought that color television required too high a bandwidth for the then available technology. Then it was noticed that the color resolution of the HVS was much lower than the brightness resolution; this allowed color to be squeezed into the signal by chroma subsampling. Another example is lossy image compression, like JPEG. Our HVS model says we cannot see high frequency detail, so in JPEG we can quantize these components without a perceptible loss of quality. Similar concepts are applied in audio compression, where sound frequencies inaudible to humans are band-stop filtered. Several HVS features are derived from evolution when we needed to defend ourselves or hunt for food. We often see demonstrations of HVS features when we are looking at optical illusions. == Block diagram of HVS == == Assumptions about the HVS == Low-pass filter characteristic (limited number of rods in human eye): see Mach bands Lack of color resolution (fewer cones in human eye than rods) Motion sensitivity More sensitive in peripheral vision Stronger than texture sensitivity, e.g. viewing a camouflaged animal Texture stronger than disparity – 3D depth resolution does not need to be so accurate Integral Face recognition (babies smile at faces) Depth inverted face looks normal (facial features overrule depth information) Upside down face with inverted mouth and eyes looks normal == Examples of taking advantage of an HVS model == Flicker frequency of film and television using persistence of vision to fool viewer into seeing a continuous image Interlaced television painting half images to give the impression of a higher flicker frequency Color television (chrominance at half resolution of luminance corresponding to proportions of rods and cones in eye) Image compression (difficult to see higher frequencies more harshly quantized) Motion estimation (use luminance and ignore color) Watermarking and Steganography