The Linguistic Data Consortium is an open consortium of universities, companies and government research laboratories. It creates, collects and distributes speech and text databases, lexicons, and other resources for linguistics research and development purposes. The University of Pennsylvania is the LDC's host institution. The LDC was founded in 1992 with a grant from the US Defense Advanced Research Projects Agency (DARPA), and is partly supported by grant IRI-9528587 from the Information and Intelligent Systems division of the National Science Foundation. The director of LDC is Mark Liberman. It subsumed the previous ACL Data Collection Initiative. Part of the motivation was to support the benchmark-oriented methodology of DARPA's Human Language Technology program. Previously, John R. Pierce directed the committee that produced the ALPAC report (1966), which caused a severe decrease in funding for linguistic AI for about 10 years. Later, Charles Wayne restarted funding in speech and language in the mid-1980s. In order to avoid the criticisms from the ALPAC report, they needed a way to demonstrate objective progress, which led to the benchmark-oriented methodology. DARPA would propose specific quantifiable and testable score targets on benchmarks, and teams being funded would attempt to reach the score targets. It was noted that by 1993, the data needed for training and benchmarking the models was big enough that "Not even the largest companies can easily afford enough of [the needed] data... Researchers at smaller companies and in universities risk being frozen out of the process almost entirely." The LDC provided a central location for creating and dispensing such data. There is a membership fee that has been increased once since its founding.
Transfer function matrix
In control system theory, and various branches of engineering, a transfer function matrix, or just transfer matrix is a generalisation of the transfer functions of single-input single-output (SISO) systems to multiple-input and multiple-output (MIMO) systems. The matrix relates the outputs of the system to its inputs. It is a particularly useful construction for linear time-invariant (LTI) systems because it can be expressed in terms of the s-plane. In some systems, especially ones consisting entirely of passive components, it can be ambiguous which variables are inputs and which are outputs. In electrical engineering, a common scheme is to gather all the voltage variables on one side and all the current variables on the other regardless of which are inputs or outputs. This results in all the elements of the transfer matrix being in units of impedance. The concept of impedance (and hence impedance matrices) has been borrowed into other energy domains by analogy, especially mechanics and acoustics. Many control systems span several different energy domains. This requires transfer matrices with elements in mixed units. This is needed both to describe transducers that make connections between domains and to describe the system as a whole. If the matrix is to properly model energy flows in the system, compatible variables must be chosen to allow this. == General == A MIMO system with m outputs and n inputs is represented by a m × n matrix. Each entry in the matrix is in the form of a transfer function relating an output to an input. For example, for a three-input, two-output system, one might write, [ y 1 y 2 ] = [ g 11 g 12 g 13 g 21 g 22 g 23 ] [ u 1 u 2 u 3 ] {\displaystyle {\begin{bmatrix}y_{1}\\y_{2}\end{bmatrix}}={\begin{bmatrix}g_{11}&g_{12}&g_{13}\\g_{21}&g_{22}&g_{23}\end{bmatrix}}{\begin{bmatrix}u_{1}\\u_{2}\\u_{3}\end{bmatrix}}} where the un are the inputs, the ym are the outputs, and the gmn are the transfer functions. This may be written more succinctly in matrix operator notation as, Y = G U {\displaystyle \mathbf {Y} =\mathbf {G} \mathbf {U} } where Y is a column vector of the outputs, G is a matrix of the transfer functions, and U is a column vector of the inputs. In many cases, the system under consideration is a linear time-invariant (LTI) system. In such cases, it is convenient to express the transfer matrix in terms of the Laplace transform (in the case of continuous time variables) or the z-transform (in the case of discrete time variables) of the variables. This may be indicated by writing, for instance, Y ( s ) = G ( s ) U ( s ) {\displaystyle \mathbf {Y} (s)=\mathbf {G} (s)\mathbf {U} (s)} which indicates that the variables and matrix are in terms of s, the complex frequency variable of the s-plane arising from Laplace transforms, rather than time. The examples in this article are all assumed to be in this form, although that is not explicitly indicated for brevity. For discrete time systems s is replaced by z from the z-transform, but this makes no difference to subsequent analysis. The matrix is particularly useful when it is a proper rational matrix, that is, all its elements are proper rational functions. In this case, the state-space representation can be applied. In systems engineering, the overall system transfer matrix G (s) is decomposed into two parts: H (s) representing the system being controlled, and C(s) representing the control system. C (s) takes as its inputs the inputs of G (s) and the outputs of H (s). The outputs of C (s) form the inputs for H (s). == Electrical systems == In electrical systems, it is often the case that the distinction between input and output variables is ambiguous. They can be either, depending on circumstance and point of view. In such cases, the concept of port (a place where energy is transferred from one system to another) can be more useful than input and output. It is customary to define two variables for each port (p): the voltage across it (Vp) and the current entering it (Ip). For instance, the transfer matrix of a two-port network can be defined as follows, [ V 1 V 2 ] = [ z 11 z 12 z 21 z 22 ] [ I 1 I 2 ] {\displaystyle {\begin{bmatrix}V_{1}\\V_{2}\end{bmatrix}}={\begin{bmatrix}z_{11}&z_{12}\\z_{21}&z_{22}\\\end{bmatrix}}{\begin{bmatrix}I_{1}\\I_{2}\end{bmatrix}}} where the zmn are called the impedance parameters, or z-parameters. They are so-called because they are in units of impedance and relate port currents to a port voltage. The z-parameters are not the only way that transfer matrices are defined for two-port networks. Six basic matrices relate voltages and currents, each with advantages for particular system network topologies. However, only two of these can be extended beyond two ports to an arbitrary number of ports. These two are the z-parameters and their inverse, the admittance parameters or y-parameters. To understand the relationship between port voltages and currents and inputs and outputs, consider the simple voltage divider circuit. If we only wish to consider the output voltage (V2) resulting from applying the input voltage (V1) then the transfer function can be expressed as, [ V 2 ] = [ R 2 R 1 + R 2 ] [ V 1 ] {\displaystyle {\begin{bmatrix}V_{2}\end{bmatrix}}={\begin{bmatrix}{\dfrac {R_{2}}{R_{1}+R_{2}}}\end{bmatrix}}{\begin{bmatrix}V_{1}\end{bmatrix}}} which can be considered the trivial case of a 1×1 transfer matrix. The expression correctly predicts the output voltage if there is no current leaving port 2, but is increasingly inaccurate as the load increases. If, however, we attempt to use the circuit in reverse, driving it with a voltage at port 2 and calculate the resulting voltage at port 1 the expression gives completely the wrong result even with no load on port 1. It predicts a greater voltage at port 1 than was applied at port 2, an impossibility with a purely resistive circuit like this one. To correctly predict the behaviour of the circuit, the currents entering or leaving the ports must also be taken into account, which is what the transfer matrix does. The impedance matrix for the voltage divider circuit is, [ V 1 V 2 ] = [ R 1 + R 2 R 2 R 2 R 2 ] [ I 1 I 2 ] {\displaystyle {\begin{bmatrix}V_{1}\\V_{2}\end{bmatrix}}={\begin{bmatrix}R_{1}+R_{2}&R_{2}\\R_{2}&R_{2}\end{bmatrix}}{\begin{bmatrix}I_{1}\\I_{2}\end{bmatrix}}} which fully describes its behaviour under all input and output conditions. At microwave frequencies, none of the transfer matrices based on port voltages and currents are convenient to use in practice. Voltage is difficult to measure directly, current next to impossible, and the open circuits and short circuits required by the measurement technique cannot be achieved with any accuracy. For waveguide implementations, circuit voltage and current are entirely meaningless. Transfer matrices using different sorts of variables are used instead. These are the powers transmitted into, and reflected from a port, which are readily measured in the transmission line technology used in distributed-element circuits in the microwave band. The most well-known and widely used of these sorts of parameters is the scattering parameters, or s-parameters. == Mechanical and other systems == The concept of impedance can be extended into the mechanical and other domains through a mechanical-electrical analogy, hence the impedance parameters and other forms of 2-port network parameters can also be extended to the mechanical domain. To do this, an effort variable and a flow variable are made analogues of voltage and current, respectively. For mechanical systems under translation these variables are force and velocity respectively. Expressing the behaviour of a mechanical component as a two-port or multi-port with a transfer matrix is a useful thing to do because, like electrical circuits, the component can often be operated in reverse and its behaviour is dependent on the loads at the inputs and outputs. For instance, a gear train is often characterised simply by its gear ratio, a SISO transfer function. However, the gearbox output shaft can be driven around to turn the input shaft, requiring a MIMO analysis. In this example, the effort and flow variables are torque T and angular velocity ω, respectively. The transfer matrix in terms of z-parameters will look like, [ T 1 T 2 ] = [ z 11 z 12 z 21 z 22 ] [ ω 1 ω 2 ] {\displaystyle {\begin{bmatrix}T_{1}\\T_{2}\end{bmatrix}}={\begin{bmatrix}z_{11}&z_{12}\\z_{21}&z_{22}\end{bmatrix}}{\begin{bmatrix}\omega _{1}\\\omega _{2}\end{bmatrix}}} However, the z-parameters are not necessarily the most convenient for characterising gear trains. A gear train is the analogue of an electrical transformer and the h-parameters (hybrid parameters) better describe transformers because they directly include the turns ratios (the analogue of gear ratios). The gearbox transfer matrix in h-parameter format is, [ T 1 ω 2 ] = [ h 11 h 12 h 21 h 22 ] [ ω 1 T 2 ] {\displaystyle {\begin{bmatrix}T_{1}\\\omega _{2}\end{bm
Independent component analysis
In signal processing, independent component analysis (ICA) is a computational method for separating a multivariate signal into additive subcomponents. This is done by assuming that at most one subcomponent is Gaussian and that the subcomponents are statistically independent from each other. ICA was invented by Jeanny Hérault and Christian Jutten in 1985. ICA is a special case of blind source separation. A common example application of ICA is the "cocktail party problem" of listening in on one person's speech in a noisy room. == Introduction == Independent component analysis attempts to decompose a multivariate signal into independent non-Gaussian signals. As an example, sound is usually a signal that is composed of the numerical addition, at each time t, of signals from several sources. The question then is whether it is possible to separate these contributing sources from the observed total signal. When the statistical independence assumption is correct, blind ICA separation of a mixed signal gives very good results. It is also used for signals that are not supposed to be generated by mixing for analysis purposes. A simple application of ICA is the "cocktail party problem", where the underlying speech signals are separated from a sample data consisting of people talking simultaneously in a room. Usually the problem is simplified by assuming no time delays or echoes. Note that a filtered and delayed signal is a copy of a dependent component, and thus the statistical independence assumption is not violated. Mixing weights for constructing the M {\textstyle M} observed signals from the N {\textstyle N} components can be placed in an M × N {\textstyle M\times N} matrix. An important thing to consider is that if N {\textstyle N} sources are present, at least N {\textstyle N} observations (e.g. microphones if the observed signal is audio) are needed to recover the original signals. When there are an equal number of observations and source signals, the mixing matrix is square ( M = N {\textstyle M=N} ). Other cases of underdetermined ( M < N {\textstyle M
Scale-invariant feature operator
In the fields of computer vision and image analysis, the scale-invariant feature operator (or SFOP) is an algorithm to detect local features in images. The algorithm was published by Förstner et al. in 2009. == Algorithm == The scale-invariant feature operator (SFOP) is based on two theoretical concepts: spiral model feature operator Desired properties of keypoint detectors: Invariance and repeatability for object recognition Accuracy to support camera calibration Interpretability: Especially corners and circles, should be part of the detected keypoints (see figure). As few control parameters as possible with clear semantics Complementarity to known detectors scale-invariant corner/circle detector. == Theory == === Maximize the weight === Maximize the weight w {\displaystyle w} = 1/variance of a point p {\displaystyle p} w ( p , α , τ , σ ) = ( N ( σ ) − 2 ) λ m i n ( M ( p , α , τ , σ ) ) Ω ( p , α , τ , σ ) {\displaystyle w(\mathbf {p} ,\alpha ,\tau ,\sigma )=\left(N(\sigma )-2\right){\frac {\lambda _{min}(M(\mathbf {p} ,\alpha ,\tau ,\sigma ))}{\Omega (\mathbf {p} ,\alpha ,\tau ,\sigma )}}} comprising: 1. the image model Ω ( p , α , τ , σ ) = ∑ n = 1 N ( σ ) [ ( q n − p ) T R α ∇ T g ( q n ) ] 2 G σ ( q n − p ) = N ( σ ) t r { R α ∇ τ ∇ τ T R α T ∗ p p T G σ ( p ) } {\displaystyle {\begin{aligned}\Omega (\mathbf {p} ,\alpha ,\tau ,\sigma )&=\sum _{n=1}^{N(\sigma )}[(\mathbf {q} _{n}-\mathbf {p} )^{T}\mathbf {R} _{\alpha }\mathbf {\nabla } _{T}g(\mathbf {q} _{n})]^{2}G_{\sigma }(\mathbf {q} _{n}-\mathbf {p} )\\&=N(\sigma )\mathbf {tr} \left\{R_{\alpha }\mathbf {\nabla } _{\tau }\mathbf {\nabla } _{\tau }^{T}R_{\alpha }^{T}\mathbf {p} \mathbf {p} ^{T}G_{\sigma }(\mathbf {p} )\right\}\end{aligned}}} 2. the smaller eigenvalue of the structure tensor M ( p , α , τ , σ ) ⏟ structure tensor = G σ ( p ) ⏟ weighted summation ∗ ( R σ ∇ τ ∇ τ T R σ T ) ⏟ squared rotated gradients {\displaystyle \underbrace {M(\mathbf {p} ,\alpha ,\tau ,\sigma )} _{\text{structure tensor}}=\underbrace {G_{\sigma }(\mathbf {p} )} _{\text{weighted summation}}\underbrace {(R_{\sigma }\nabla _{\tau }\nabla _{\tau }^{T}R_{\sigma }^{T})} _{\text{squared rotated gradients}}} === Reduce the search space === Reduce the 5-dimensional search space by linking the differentiation scale τ {\displaystyle \tau } to the integration scale τ = σ / 3 {\displaystyle \tau =\sigma /3} solving for the optimal α ^ {\displaystyle {\hat {\alpha }}} using the model Ω ( α ) = a − b cos ( 2 α − 2 α 0 ) {\displaystyle \Omega (\alpha )=a-b\cos(2\alpha -2\alpha _{0})} and determining the parameters from three angles, e. g. Ω ( 0 ∘ ) , Ω ( 60 ∘ ) , Ω ( 120 ∘ ) → a , b , α 0 → α ^ {\displaystyle \Omega (0^{\circ }),\Omega (60^{\circ }),\Omega (120^{\circ })\quad \rightarrow \quad a,b,\alpha _{0}\quad \rightarrow \quad {\hat {\alpha }}} pre-selection possible: α = 0 ∘ → junctions , α = 90 ∘ → circular features {\displaystyle \alpha =0^{\circ }\,\rightarrow \,{\mbox{junctions}},\quad \alpha =90^{\circ }\,\rightarrow \,{\mbox{circular features}}} === Filter potential keypoints === non-maxima suppression over scale, space and angle thresholding the isotropy λ 2 ( M ) {\displaystyle \lambda _{2(M)}} :eigenvalues characterize the shape of the keypoint, smallest eigenvalue has to be larger than threshold T λ {\displaystyle T_{\lambda }} derived from noise variance V ( n ) {\displaystyle V(n)} and significance level S {\displaystyle S} : T λ ( V ( n ) , τ , σ , S ) = N ( σ ) 16 π τ 4 V ( n ) χ 2 , S 2 {\displaystyle T_{\lambda }(V(n),\tau ,\sigma ,S)={\frac {N(\sigma )}{16\pi \tau ^{4}}}V(n)\chi _{2,S}^{2}} == Algorithm == == Results == === Interpretability of SFOP keypoints ===
PVLV
The primary value learned value (PVLV) model is a possible explanation for the reward-predictive firing properties of dopamine (DA) neurons. It simulates behavioral and neural data on Pavlovian conditioning and the midbrain dopaminergic neurons that fire in proportion to unexpected rewards. It is an alternative to the temporal-differences (TD) algorithm. It is used as part of Leabra.
List of data science software
This is a list of data science software and platforms used in data science, which includes programming languages, programming environments, machine learning frameworks, data engineering tools, statistical software, data analysis, plotting, MLOps systems, and more. == Programming languages == == Development environments == These interactive notebooks, IDEs, and platforms provide specialised development environments. Apache Zeppelin Architect — Eclipse (software) CoCalc Dataiku Data Science Studio FreeMat GNU Octave Google Colab DataSpell Jupyter Notebook / JupyterLab Kaggle Notebooks MATLAB O-Matrix PyCharm RStudio SAS (software) and SAS Studio Spyder Visual Studio Code == Machine and deep learning software == The Machine learning / deep learning tools support development in those fields. == Data engineering == Examples of Data engineering tools. Apache Airflow Apache Flink Apache Hadoop Apache Kafka Apache NiFi Apache Spark Dask Data build tool (dbt) == Data mining == Examples of Data mining tools. === Free and open-source === === Proprietary === == Database management == === List of RDBMS === ==== Proprietary ==== == Data warehouses == Data warehouse environments include: Amazon Redshift Snowflake Google BigQuery Microsoft Azure Synapse Teradata Vertica == Data lakes == Data lake environments include: Apache Hadoop Cloudera Databricks Delta Lake Amazon S3 Google Cloud Storage Azure Data Lake == Algorithms == Apriori algorithm – frequent itemset mining and association rule learning in market basket analysis Backpropagation – algorithm for training artificial neural networks using gradient descent Decision Trees – tree-based algorithm for classification and regression Expectation–maximization algorithm – iterative procedure for maximum likelihood estimation with latent variables Gradient descent – iterative optimization algorithm for minimizing a loss function ID3 algorithm – used to generate a decision tree from a dataset K-Means – clustering algorithm based on minimizing within-cluster distances K-Nearest Neighbors (KNN) – instance-based learning and classification method Linear regression – estimation method for predicting a dependent variable based on independent variables Logistic regression – classification algorithm for predicting a binary outcome Naive Bayes – probabilistic classifier based on Bayes' theorem Ordinary least squares – estimation method for parameters in linear regression PageRank – graph-based algorithm for link analysis and search ranking Principal component analysis – technique to reduce high-dimensional data while preserving variance Q-learning – reinforcement learning algorithm for learning optimal actions Random forest – ensemble of decision trees for improved classification or regression Sequential minimal optimization – solver for training support vector machines Stochastic gradient descent – randomized variant of gradient descent for large-scale machine learning Support Vector Machines (SVM) – algorithm for finding a hyperplane to separate classes == Statistical software == === Open-source === === Public domain === CSPro Dataplot Epi Map X-13ARIMA-SEATS === Freeware === BV4.1 MINUIT WinBUGS Winpepi === Proprietary === == Data processing == Tools for Data processing and analysis: == Data and information visualization == Software for Data visualization: == Plotting software == Software for plotting data to support processing and visualise results. == Maps and geospatial visualization == ArcGIS Carto Epi Map GeoDA Google Earth Engine Leaflet Mapbox MountainsMap QGIS == Machine learning == MLOps and model deployment: BentoML Data Version Control (DVC) Kubeflow MLflow Seldon Core Streamlit TensorFlow Serving Weights & Biases == Data repositories == Kaggle – platform for data science competitions, datasets, and notebooks. OpenML – collaborative platform for sharing datasets, algorithms, and experiments. University of California, Irvine Machine Learning Repository Zenodo – open-access repository supported by CERN and the EU. == Educational data science software == Kaggle – online platform for data science education, competitions, datasets, and collaborative learning. KNIME – open-source data analytics platform used for teaching data science, machine learning, and workflow-based analysis. RapidMiner – used in academic research and education for data mining and machine learning. Statistics Online Computational Resource (SOCR) – online tools and instructional resources for statistics education. Tanagra (machine learning) – data mining software developed for research and teaching purposes. TinkerPlots – explore and analyze data through visual modeling.
Liquid state machine
A liquid state machine (LSM) is a type of reservoir computer that uses a spiking neural network. An LSM consists of a large collection of units (called nodes, or neurons). Each node receives time varying input from external sources (the inputs) as well as from other nodes. Nodes are randomly connected to each other. The recurrent nature of the connections turns the time varying input into a spatio-temporal pattern of activations in the network nodes. The spatio-temporal patterns of activation are read out by linear discriminant units. The soup of recurrently connected nodes will end up computing a large variety of nonlinear functions on the input. Given a large enough variety of such nonlinear functions, it is theoretically possible to obtain linear combinations (using the read out units) to perform whatever mathematical operation is needed to perform a certain task, such as speech recognition or computer vision. The word liquid in the name comes from the analogy drawn to dropping a stone into a still body of water or other liquid. The falling stone will generate ripples in the liquid. The input (motion of the falling stone) has been converted into a spatio-temporal pattern of liquid displacement (ripples). LSMs have been put forward as a way to explain the operation of brains. LSMs are argued to be an improvement over the theory of artificial neural networks because: Circuits are not hard coded to perform a specific task. Continuous time inputs are handled "naturally". Computations on various time scales can be done using the same network. The same network can perform multiple computations. Criticisms of LSMs as used in computational neuroscience are that LSMs don't actually explain how the brain functions. At best they can replicate some parts of brain functionality. There is no guaranteed way to dissect a working network and figure out how or what computations are being performed. There is very little control over the process. == Universal function approximation == If a reservoir has fading memory and input separability, with help of a readout, it can be proven the liquid state machine is a universal function approximator using Stone–Weierstrass theorem.