ChatScript is a combination Natural Language engine and dialog management system designed initially for creating chatbots, but is currently also used for various forms of NL processing. It is written in C++. The engine is an open source project at SourceForge. and GitHub. ChatScript was written by Bruce Wilcox and originally released in 2011, after Suzette (written in ChatScript) won the 2010 Loebner Prize, fooling one of four human judges. == Features == In general ChatScript aims to author extremely concisely, since the limiting scalability of hand-authored chatbots is how much/fast one can write the script. Because ChatScript is designed for interactive conversation, it automatically maintains user state across volleys. A volley is any number of sentences the user inputs at once and the chatbots response. The basic element of scripting is the rule. A rule consists of a type, a label (optional), a pattern, and an output. There are three types of rules. Gambits are something a chatbot might say when it has control of the conversation. Rejoinders are rules that respond to a user remark tied to what the chatbot just said. Responders are rules that respond to arbitrary user input which is not necessarily tied to what the chatbot just said. Patterns describe conditions under which a rule may fire. Patterns range from extremely simplistic to deeply complex (analogous to Regex but aimed for NL). Heavy use is typically made of concept sets, which are lists of words sharing a meaning. ChatScript contains some 2000 predefined concepts and scripters can easily write their own. Output of a rule intermixes literal words to be sent to the user along with common C-style programming code. Rules are bundled into collections called topics. Topics can have keywords, which allows the engine to automatically search the topic for relevant rules based on user input. == Example code == Words starting with ~ are concept sets. For example, ~fruit is the list of all known fruits. The simple pattern (~fruit) reacts if any fruit is mentioned immediately after the chatbot asks for favorite food. The slightly more complex pattern for the rule labelled WHATMUSIC requires all the words what, music, you and any word or phrase meaning to like, but they may occur in any order. Responders come in three types. ?: rules react to user questions. s: rules react to user statements. u: rules react to either. ChatScript code supports standard if-else, loops, user-defined functions and calls, and variable assignment and access. == Data == Some data in ChatScript is transient, meaning it will disappear at the end of the current volley. Other data is permanent, lasting forever until explicitly killed off. Data can be local to a single user or shared across all users at the bot level. Internally all data is represented as text and is automatically converted to a numeric form as needed. === Variables === User variables come in several kinds. Variables purely local to a topic or function are transient. Global variables can be declared as transient or permanent. A variable is generally declared merely by using it, and its type depends on its prefix ($, $$, $_). === Facts === In addition to variables, ChatScript supports facts – triples of data, which can also be transient or permanent. Functions can query for facts having particular values of some of the fields, making them act like an in-memory database. Fact retrieval is very quick and efficient the number of available in-memory facts is largely constrained to the available memory of the machine running the ChatScript engine. Facts can represent record structures and are how ChatScript represents JSON internally. Tables of information can be defined to generate appropriate facts. The above table links people to what they invented (1 per line) with Einstein getting a list of things he did. == External communication == ChatScript embeds the Curl library and can directly read and write facts in JSON to a website. == Server == A ChatScript engine can run in local or server mode. == Pos-tagging, parsing, and ontology == ChatScript comes with a copy of English WordNet embedded within, including its ontology, and creates and extends its own ontology via concept declarations. It has an English language pos-tagger and parser and supports integration with TreeTagger for pos-tagging a number of other languages (TreeTagger commercial license required). == Databases == In addition to an internal fact database, ChatScript supports PostgreSQL, MySQL, MSSQL and MongoDB both for access by scripts, but also as a central filesystem if desired so ChatScript can be scaled horizontally. A common use case is to use a centralized database to host the user files and multiple servers to scale the ChatScript engine. == JavaScript == ChatScript also embeds DukTape, ECMAScript E5/E5.1 compatibility, with some semantics updated from ES2015+. == Spelling Correction == ChatScript has built-in automatic spell checking, which can be augmented in script as both simple word replacements or context sensitive changes. With appropriate simple rules you can change perfect legal words into other words or delete them. E.g., if you have a concept of ~electronic_goods and don't want an input of Radio Shack (a store name) to be detected as an electronic good, you can get the input to change to Radio_Shack (a single word), or allow the words to remain but block the detection of the concept. This is particularly useful when combined with speech-to-text code that is imperfect, but you are familiar with common failings of it and can compensate for them in script. == Control flow == A chatbot's control flow is managed by the control script. This is merely another ordinary topic of rules, that invokes API functions of the engine. Thus control is fully configurable by the scripter (and functions exist to allow introspection into the engine). There are pre-processing control flow and post-processing control flow options available, for special processing.
Artipic
Artipic is a graphics editor developed for Microsoft Windows. An older version for macOS is still available but unsupported. Artipic features drawing, editing, retouching, transforming and composing images including color corrections, effects and layer-based operations. It converts all common image formats and imports camera raw formats. In the global image editing ecosystem Artipic can be positioned somewhere in the middle. It differs from simple free photo editors by more advanced capabilities, however it does not cover the complete professional-level functionality pack provided by industry leaders like Adobe Photoshop. == History == Artipic developed by Swedish company Artipic AB. Artipic 1.0 was released in March 2014 as a free version. The first commercial version on Microsoft Windows was released in November 2014, on macOS – in October 2015. == Features == Supports Microsoft Windows and macOS Standard tools: select, crop, move, rotate, transform, stamp, color picking, text Advanced tools: custom brushes, gradients, shapes, paths, layers and masks Special tools: healing brush, red-eye effect reduction, dodge and burn brushes Adjustments: Brightness & Contrast, Hue & Saturation, Curves, Levels, Color Balance, Gamma Correction, Exposure, Color Temperature, Tint, Color Enhancer, Photo Filter Simulation, Posterization, Thresholding Filters: Smoothen, Sharpen, Vignetting, High-pass, Diffuse Glow, Shadow, Gaussian Blur Reversible (non-destructive) stylization presets Batch processing White balance RAW-converter including Gray Card Adobe Photoshop images supported == Version history ==
Liveness test
A liveness test, liveness check or liveness detection is an automated method for determining whether a subject is a real person or part of a spoofing attack. The technique is used as part of know your customer checks in financial services and during facial age estimation. Liveness detection is a cornerstone of digital safety. == Test process == The threat in face spoofing attacks is that "the attacker only needs to find a good face swap library on Github and understand how to inject the model into the camera feed during the KYC process". Fraudsters usually buy stolen IDs on the dark web to start a deepfake attack. An AI-powered generative adversarial network (GAN) can then generate the face swapping model that many online verification services fail to detect. Low level hackers may use face swapping apps such as SwapFace, DeepFaceLive, and Swapstream (increasing interest for those apps in 2023 according to Google Trends). In a video liveness test, users are typically asked to look into a camera and to move, smile or blink, and features of their moving face may then be compared to that of a still image. Artificial intelligence is used to counter presentation attacks such as deepfakes or users wearing hyperrealistic masks, or video injection attacks. Other forms of liveness test include checking for a pulse when using a fingerprint scanner or checking that a person's voice is not a recording or artificially generated during speaker recognition. == Adoption and certification == In a 2022 report published by the security firm Sensity, it was demonstrated that the liveness test of most US banks was easily cheated with new and publicly-available AI-powered techniques. Many of these banks disregarded the results of the report. In the first half of 2023, the security firm iProov detected a 704% increase in face-swap attacks. In 2023, in the UK, many customers of Ryanair were upset to have to go through many ID verification checks, including liveness tests, before boarding, as the airline was using it as a mean to deter customers to buy tickets through third-party websites. In the first half of 2024 iBeta Quality Assurance issued 18 new ISO/IEC 30107-3 Presentation Attack Detection certificates, raising the cumulative total to 85 since 2018. In January 2024, the Department of Homeland Security (DHS) opened applications from vendors to test their Liveness test. Identity frauds peaked during the COVID-19 lockdown, leading government agencies to take reinforced measures to secure their digital applications.
Protégé (software)
Protégé is a free, open source ontology editor and a knowledge management system. The Protégé meta-tool was first built by Mark Musen in 1987 and has since been developed by a team at Stanford University. The software is the most popular and widely used ontology editor in the world. == Overview == Protégé provides a graphical user interface to define ontologies. It also includes deductive classifiers to validate that models are consistent and to infer new information based on the analysis of an ontology. Like Eclipse, Protégé is a framework for which various other projects suggest plugins. This application is written in Java and makes heavy use of Swing to create the user interface. According to their website, there are over 300,000 registered users. A 2009 book calls it "the leading ontological engineering tool". Protégé is developed at Stanford University and is made available under the BSD 2-clause license. Earlier versions of the tool were developed in collaboration with the University of Manchester.
Drools
Drools is a business rule management system (BRMS) with a forward and backward chaining inference-based rules engine, more correctly known as a production rule system, using an enhanced implementation of the Rete algorithm. Drools supports the Java Rules Engine API (Java Specification Request 94) standard for its business rule engine and enterprise framework for the construction, maintenance, and enforcement of business policies in an organization, application, or service. == Drools in Apache Kie == Drools, as part of the Kie Community has entered Apache Incubator in January, 2023. == Red Hat Decision Manager == Red Hat Decision Manager (formerly Red Hat JBoss BRMS) is a business rule management system and reasoning engine for business policy and rules development, access, and change management. JBoss Enterprise BRMS is a productized version of Drools with enterprise-level support available. JBoss Rules is also a productized version of Drools, but JBoss Enterprise BRMS is the flagship product. Components of the enterprise version: JBoss Enterprise Web Platform – the software infrastructure, supported to run the BRMS components only JBoss Enterprise Application Platform or JBoss Enterprise SOA Platform – the software infrastructure, supported to run the BRMS components only Business Rules Engine – Drools Expert using the Rete algorithm and the Drools Rule Language (DRL) Business Rules Manager – Drools Guvnor - Guvnor is a centralized repository for Drools Knowledge Bases, with rich web-based GUIs, editors, and tools to aid in the management of large numbers of rules. Business Rules Repository – Drools Guvnor Drools and Guvnor are JBoss Community open source projects. As they are mature, they are brought into the enterprise-ready product JBoss Enterprise BRMS. Components of the JBoss Community version: Drools Guvnor (Business Rules Manager) – a centralized repository for Drools Knowledge Bases Drools Expert (rule engine) – uses the rules to perform reasoning Drools Flow (process/workflow), or jBPM 5 – provides for workflow and business processes Drools Fusion (event processing/temporal reasoning) – provides for complex event processing Drools Planner/OptaPlanner (automated planning) – optimizes automated planning, including NP-hard planning problems == Example == This example illustrates a simple rule to print out information about a holiday in July. It checks a condition on an instance of the Holiday class, and executes Java code if that condition is true. The purpose of dialect "mvel" is to point the getter and setters of the variables of your Plain Old Java Object (POJO) classes. Consider the above example, in which a Holiday class is used and inside the circular brackets (parentheses) "month" is used. So with the help of dialect "mvel" the getter and setters of the variable "month" can be accessed. Dialect "java" is used to help us write our Java code in our rules. There is one restriction or characteristic on this. We cannot use Java code inside the "when" part of the rule but we can use Java code in the "then" part. We can also declare a Reference variable $h1 without the $ symbol. There is no restriction on this. The main purpose of putting the $ symbol before the variable is to mark the difference between variables of POJO classes and Rules.
Concept drift
In predictive analytics, data science, machine learning and related fields, concept drift or drift is an evolution of data that invalidates the data model. It happens when the statistical properties of the target variable, which the model is trying to predict, change over time in unforeseen ways. This causes problems because the predictions become less accurate as time passes. Drift detection and drift adaptation are of paramount importance in the fields that involve dynamically changing data and data models. == Predictive model decay == In machine learning and predictive analytics this drift phenomenon is called concept drift. In machine learning, a common element of a data model are the statistical properties, such as probability distribution of the actual data. If they deviate from the statistical properties of the training data set, then the learned predictions may become invalid, if the drift is not addressed. == Data configuration decay == Another important area is software engineering, where three types of data drift affecting data fidelity may be recognized. Changes in the software environment ("infrastructure drift") may invalidate software infrastructure configuration. "Structural drift" happens when the data schema changes, which may invalidate databases. "Semantic drift" is changes in the meaning of data while the structure does not change. In many cases this may happen in complicated applications when many independent developers introduce changes without proper awareness of the effects of their changes in other areas of the software system. For many application systems, the nature of data on which they operate are subject to changes for various reasons, e.g., due to changes in business model, system updates, or switching the platform on which the system operates. In the case of cloud computing, infrastructure drift that may affect the applications running on cloud may be caused by the updates of cloud software. There are several types of detrimental effects of data drift on data fidelity. Data corrosion is passing the drifted data into the system undetected. Data loss happens when valid data are ignored due to non-conformance with the applied schema. Squandering is the phenomenon when new data fields are introduced upstream in the data processing pipeline, but somewhere downstream these data fields are absent. == Inconsistent data == "Data drift" may refer to the phenomenon when database records fail to match the real-world data due to the changes in the latter over time. This is a common problem with databases involving people, such as customers, employees, citizens, residents, etc. Human data drift may be caused by unrecorded changes in personal data, such as place of residence or name, as well as due to errors during data input. "Data drift" may also refer to inconsistency of data elements between several replicas of a database. The reasons can be difficult to identify. A simple drift detection is to run checksum regularly. However the remedy may be not so easy. == Examples == The behavior of the customers in an online shop may change over time. For example, if weekly merchandise sales are to be predicted, and a predictive model has been developed that works satisfactorily. The model may use inputs such as the amount of money spent on advertising, promotions being run, and other metrics that may affect sales. The model is likely to become less and less accurate over time – this is concept drift. In the merchandise sales application, one reason for concept drift may be seasonality, which means that shopping behavior changes seasonally. Perhaps there will be higher sales in the winter holiday season than during the summer, for example. Concept drift generally occurs when the covariates that comprise the data set begin to explain the variation of your target set less accurately — there may be some confounding variables that have emerged, and that one simply cannot account for, which renders the model accuracy to progressively decrease with time. Generally, it is advised to perform health checks as part of the post-production analysis and to re-train the model with new assumptions upon signs of concept drift. == Possible remedies == To prevent deterioration in prediction accuracy because of concept drift, reactive and tracking solutions can be adopted. Reactive solutions retrain the model in reaction to a triggering mechanism, such as a change-detection test or control charts from statistical process control, to explicitly detect concept drift as a change in the statistics of the data-generating process. When concept drift is detected, the current model is no longer up-to-date and must be replaced by a new one to restore prediction accuracy. A shortcoming of reactive approaches is that performance may decay until the change is detected. Tracking solutions seek to track the changes in the concept by continually updating the model. Methods for achieving this include online machine learning, frequent retraining on the most recently observed samples, and maintaining an ensemble of classifiers where one new classifier is trained on the most recent batch of examples and replaces the oldest classifier in the ensemble. Contextual information, when available, can be used to better explain the causes of the concept drift: for instance, in the sales prediction application, concept drift might be compensated by adding information about the season to the model. By providing information about the time of the year, the rate of deterioration of your model is likely to decrease, but concept drift is unlikely to be eliminated altogether. This is because actual shopping behavior does not follow any static, finite model. New factors may arise at any time that influence shopping behavior, the influence of the known factors or their interactions may change. Concept drift cannot be avoided for complex phenomena that are not governed by fixed laws of nature. All processes that arise from human activity, such as socioeconomic processes, and biological processes are likely to experience concept drift. Therefore, periodic retraining, also known as refreshing, of any model is necessary. === Remedy methods === DDM (Drift Detection Method): detects drift by monitoring the model's error rate over time. When the error rate passes a set threshold, it enters a warning phase, and if it passes another threshold, it enters a drift phase. EDDM (Early Drift Detection Method): improves DDM's detection rate by tracking the average distance between two errors instead of only the error rate. ADWIN (Adaptive Windowing): dynamically stores a window of recent data and warns the user if it detects a significant change between the statistics of the window's earlier data compared to more recent data. KSWIN (Kolmogorov–Smirnov Windowing): detects drift based on the Kolmogorov-Smirnov statistical test. DDM and EDDM: Concept Drift Detection online supervised methods that rely on sequential error monitoring to estimate the evolving error rate. ADWIN and KSWIN: Windowing maintain a "window", a subset of the most recent data, of the data stream, which it checks for statistical differences across the window. == Applications in security == Concept drift is a recurring issue in security analytics, especially in malware and intrusion detection. In these systems, models are often trained on past logs, binaries or network traces, but the behaviour of attackers changes over time as new malware families, obfuscation techniques and campaigns appear. When the data no longer resemble the training set, the decision boundaries learned by classifiers or anomaly detectors can become misaligned with the current threat landscape and detection performance can drop unless the models are updated or replaced. Several studies on Windows malware model detection as an evolving data stream and track how performance changes as time passes. They show that classifiers trained on a fixed time window can perform well on nearby data but deteriorate quickly when evaluated on samples collected months or years later, even when large amounts of training data are available. In order to keep up with this, security systems often use sliding or adaptive windows, which restrict training to the most recent portion of the data so that older, less relevant examples are gradually discarded. They also employ drift detectors such as ADWIN and KSWIN that monitor error rates or changes in the distribution of recent observations and signal when the statistics of the incoming stream differ significantly from the past, prompting retraining or model replacement. Related problems appear in spam filtering, fraud detection and intrusion detection, where adversaries change content, patterns of activity or network behavior to evade models trained on historical data. In these settings drift can be gradual, as new types of spam or fraud emerge, or abrupt, after a sudden shift in attack techniques. Common strategies to remain eff
Automatic1111
AUTOMATIC1111 Stable Diffusion Web UI (SD WebUI, A1111, or Automatic1111) is an open source generative artificial intelligence program that allows users to generate images from a text prompt. It uses Stable Diffusion as the base model for its image capabilities together with a large set of extensions and features to customize its output. == History == SD WebUI was released on GitHub on August 22, 2022, by AUTOMATIC1111, 1 month after the initial release of Stable Diffusion. At the time, Stable Diffusion could only be run via the command line. SD WebUI quickly rose in popularity and has been described as "the most popular tool for running diffusion models locally." SD WebUI is one of the most popular user interfaces for Stable Diffusion, together with ComfyUI. In February 2024, a book was published by ja:Gijutsu Hyoronsha on using Stable Diffusion with SD WebUI in Japanese. As of July 2024, the project had 136,000 stars on GitHub. == Features == SD WebUI uses Gradio for its user interface. Each parameter in the Stable Diffusion program is exposed via a UI interface within SD WebUI. SD WebUI contains additional parameters not included in Stable Diffusion itself, such as support for Low-rank adaptations, ControlNet and custom variational autoencoders. SD WebUI supports prompt weighting, image-to-image based generation, inpainting, outpainting and image scaling. It supports over 20 samplers including DDIM, Euler, Euler a, DPM++ 2M Karras, and UniPC. It is also used for its various optimizations over the base Stable Diffusion. == Stable Diffusion WebUI Forge == Stable Diffusion WebUI Forge (Forge) is a notable fork of SD WebUI started by Lvmin Zhang, who is also the creator of ControlNet and Fooocus. The initial goal of Forge was to improve the performance and features of SD WebUI with the intention to upstream changes back to SD WebUI. One of Forge's optimizations allowed users with low VRAM to generate images faster on some versions of Stable Diffusion. It improved generation speed for users with 8GB and 6GB VRAM by 30-45% and 60-75%, respectively. Forge also includes extra features such as support for more samplers than standard SD WebUI. Some of Forge's optimizations were borrowed from ComfyUI, and others were developed by the Forge team. In August 2024, Forge added support for the Flux diffusion model developed by Black Forest Labs, which is not yet supported by SD WebUI.