AI Code Base

AI Code Base — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  Double descent

  Double descent in statistics and machine learning is the phenomenon where a model's error rate on the test set initially decreases with the number of parameters, then peaks, then decreases again. This phenomenon has been considered surprising, as it contradicts assumptions about overfitting in classical machine learning. The increase usually occurs near the interpolation threshold, where the number of parameters is the same as the number of training data points (the model is just large enough to fit the training data). Or, more precisely, it is the maximum number of samples on which the model/training procedure achieves approximately on average 0 training error. == History == Early observations of what would later be called double descent in specific models date back to 1989. The term "double descent" was coined by Belkin et. al. in 2019, when the phenomenon gained popularity as a broader concept exhibited by many models. The latter development was prompted by a perceived contradiction between the conventional wisdom that too many parameters in the model result in a significant overfitting error (an extrapolation of the bias–variance tradeoff), and the empirical observations in the 2010s that some modern machine learning techniques tend to perform better with larger models. == Theoretical models == Double descent occurs in linear regression with isotropic Gaussian covariates and isotropic Gaussian noise. A model of double descent at the thermodynamic limit has been analyzed using the replica trick, and the result has been confirmed numerically. A number of works have suggested that double descent can be explained using the concept of effective dimension: While a network may have a large number of parameters, in practice only a subset of those parameters are relevant for generalization performance, as measured by the local Hessian curvature. This explanation is formalized through PAC-Bayes compression-based generalization bounds, which show that less complex models are expected to generalize better under a Solomonoff prior.

  Read more →
• 

  Holographic algorithm

  In computer science, a holographic algorithm is an algorithm that uses a holographic reduction. A holographic reduction is a constant-time reduction that maps solution fragments many-to-many such that the sum of the solution fragments remains unchanged. These concepts were introduced by Leslie Valiant, who called them holographic because "their effect can be viewed as that of producing interference patterns among the solution fragments". The algorithms are unrelated to laser holography, except metaphorically. Their power comes from the mutual cancellation of many contributions to a sum, analogous to the interference patterns in a hologram. Holographic algorithms have been used to find polynomial-time solutions to problems without such previously known solutions for special cases of satisfiability, vertex cover, and other graph problems. They have received notable coverage due to speculation that they are relevant to the P versus NP problem and their impact on computational complexity theory. Although some of the general problems are #P-hard problems, the special cases solved are not themselves #P-hard, and thus do not prove FP = #P. Holographic algorithms have some similarities with quantum computation, but are completely classical. == Holant problems == Holographic algorithms exist in the context of Holant problems, which generalize counting constraint satisfaction problems (#CSP). A #CSP instance is a hypergraph G=(V,E) called the constraint graph. Each hyperedge represents a variable and each vertex v {\displaystyle v} is assigned a constraint f v . {\displaystyle f_{v}.} A vertex is connected to an hyperedge if the constraint on the vertex involves the variable on the hyperedge. The counting problem is to compute ∑ σ : E → { 0 , 1 } ∏ v ∈ V f v ( σ | E ( v ) ) , ( 1 ) {\displaystyle \sum _{\sigma :E\to \{0,1\}}\prod _{v\in V}f_{v}(\sigma |_{E(v)}),~~~~~~~~~~(1)} which is a sum over all variable assignments, the product of every constraint, where the inputs to the constraint f v {\displaystyle f_{v}} are the variables on the incident hyperedges of v {\displaystyle v} . A Holant problem is like a #CSP except the input must be a graph, not a hypergraph. Restricting the class of input graphs in this way is indeed a generalization. Given a #CSP instance, replace each hyperedge e of size s with a vertex v of degree s with edges incident to the vertices contained in e. The constraint on v is the equality function of arity s. This identifies all of the variables on the edges incident to v, which is the same effect as the single variable on the hyperedge e. In the context of Holant problems, the expression in (1) is called the Holant after a related exponential sum introduced by Valiant. == Holographic reduction == A standard technique in complexity theory is a many-one reduction, where an instance of one problem is reduced to an instance of another (hopefully simpler) problem. However, holographic reductions between two computational problems preserve the sum of solutions without necessarily preserving correspondences between solutions. For instance, the total number of solutions in both sets can be preserved, even though individual problems do not have matching solutions. The sum can also be weighted, rather than simply counting the number of solutions, using linear basis vectors. === General example === It is convenient to consider holographic reductions on bipartite graphs. A general graph can always be transformed it into a bipartite graph while preserving the Holant value. This is done by replacing each edge in the graph by a path of length 2, which is also known as the 2-stretch of the graph. To keep the same Holant value, each new vertex is assigned the binary equality constraint. Consider a bipartite graph G=(U,V,E) where the constraint assigned to every vertex u ∈ U {\displaystyle u\in U} is f u {\displaystyle f_{u}} and the constraint assigned to every vertex v ∈ V {\displaystyle v\in V} is f v {\displaystyle f_{v}} . Denote this counting problem by Holant ( G , f u , f v ) . {\displaystyle {\text{Holant}}(G,f_{u},f_{v}).} If the vertices in U are viewed as one large vertex of degree |E|, then the constraint of this vertex is the tensor product of f u {\displaystyle f_{u}} with itself |U| times, which is denoted by f u ⊗ | U | . {\displaystyle f_{u}^{\otimes |U|}.} Likewise, if the vertices in V are viewed as one large vertex of degree |E|, then the constraint of this vertex is f v ⊗ | V | . {\displaystyle f_{v}^{\otimes |V|}.} Let the constraint f u {\displaystyle f_{u}} be represented by its weighted truth table as a row vector and the constraint f v {\displaystyle f_{v}} be represented by its weighted truth table as a column vector. Then the Holant of this constraint graph is simply f u ⊗ | U | f v ⊗ | V | . {\displaystyle f_{u}^{\otimes |U|}f_{v}^{\otimes |V|}.} Now for any complex 2-by-2 invertible matrix T (the columns of which are the linear basis vectors mentioned above), there is a holographic reduction between Holant ( G , f u , f v ) {\displaystyle {\text{Holant}}(G,f_{u},f_{v})} and Holant ( G , f u T ⊗ ( deg ⁡ u ) , ( T − 1 ) ⊗ ( deg ⁡ v ) f v ) . {\displaystyle {\text{Holant}}(G,f_{u}T^{\otimes (\deg u)},(T^{-1})^{\otimes (\deg v)}f_{v}).} To see this, insert the identity matrix T ⊗ | E | ( T − 1 ) ⊗ | E | {\displaystyle T^{\otimes |E|}(T^{-1})^{\otimes |E|}} in between f u ⊗ | U | f v ⊗ | V | {\displaystyle f_{u}^{\otimes |U|}f_{v}^{\otimes |V|}} to get f u ⊗ | U | f v ⊗ | V | {\displaystyle f_{u}^{\otimes |U|}f_{v}^{\otimes |V|}} = f u ⊗ | U | T ⊗ | E | ( T − 1 ) ⊗ | E | f v ⊗ | V | {\displaystyle =f_{u}^{\otimes |U|}T^{\otimes |E|}(T^{-1})^{\otimes |E|}f_{v}^{\otimes |V|}} = ( f u T ⊗ ( deg ⁡ u ) ) ⊗ | U | ( f v ( T − 1 ) ⊗ ( deg ⁡ v ) ) ⊗ | V | . {\displaystyle =\left(f_{u}T^{\otimes (\deg u)}\right)^{\otimes |U|}\left(f_{v}(T^{-1})^{\otimes (\deg v)}\right)^{\otimes |V|}.} Thus, Holant ( G , f u , f v ) {\displaystyle {\text{Holant}}(G,f_{u},f_{v})} and Holant ( G , f u T ⊗ ( deg ⁡ u ) , ( T − 1 ) ⊗ ( deg ⁡ v ) f v ) {\displaystyle {\text{Holant}}(G,f_{u}T^{\otimes (\deg u)},(T^{-1})^{\otimes (\deg v)}f_{v})} have exactly the same Holant value for every constraint graph. They essentially define the same counting problem. === Specific examples === ==== Vertex covers and independent sets ==== Let G be a graph. There is a 1-to-1 correspondence between the vertex covers of G and the independent sets of G. For any set S of vertices of G, S is a vertex cover in G if and only if the complement of S is an independent set in G. Thus, the number of vertex covers in G is exactly the same as the number of independent sets in G. The equivalence of these two counting problems can also be proved using a holographic reduction. For simplicity, let G be a 3-regular graph. The 2-stretch of G gives a bipartite graph H=(U,V,E), where U corresponds to the edges in G and V corresponds to the vertices in G. The Holant problem that naturally corresponds to counting the number of vertex covers in G is Holant ( H , OR 2 , EQUAL 3 ) . {\displaystyle {\text{Holant}}(H,{\text{OR}}_{2},{\text{EQUAL}}_{3}).} The truth table of OR2 as a row vector is (0,1,1,1). The truth table of EQUAL3 as a column vector is ( 1 , 0 , 0 , 0 , 0 , 0 , 0 , 1 ) T = [ 1 0 ] ⊗ 3 + [ 0 1 ] ⊗ 3 {\displaystyle (1,0,0,0,0,0,0,1)^{T}={\begin{bmatrix}1\\0\end{bmatrix}}^{\otimes 3}+{\begin{bmatrix}0\\1\end{bmatrix}}^{\otimes 3}} . Then under a holographic transformation by [ 0 1 1 0 ] , {\displaystyle {\begin{bmatrix}0&1\\1&0\end{bmatrix}},} OR 2 ⊗ | U | EQUAL 3 ⊗ | V | {\displaystyle {\text{OR}}_{2}^{\otimes |U|}{\text{EQUAL}}_{3}^{\otimes |V|}} = ( 0 , 1 , 1 , 1 ) ⊗ | U | ( [ 1 0 ] ⊗ 3 + [ 0 1 ] ⊗ 3 ) ⊗ | V | {\displaystyle =(0,1,1,1)^{\otimes |U|}\left({\begin{bmatrix}1\\0\end{bmatrix}}^{\otimes 3}+{\begin{bmatrix}0\\1\end{bmatrix}}^{\otimes 3}\right)^{\otimes |V|}} = ( 0 , 1 , 1 , 1 ) ⊗ | U | [ 0 1 1 0 ] ⊗ | E | [ 0 1 1 0 ] ⊗ | E | ( [ 1 0 ] ⊗ 3 + [ 0 1 ] ⊗ 3 ) ⊗ | V | {\displaystyle =(0,1,1,1)^{\otimes |U|}{\begin{bmatrix}0&1\\1&0\end{bmatrix}}^{\otimes |E|}{\begin{bmatrix}0&1\\1&0\end{bmatrix}}^{\otimes |E|}\left({\begin{bmatrix}1\\0\end{bmatrix}}^{\otimes 3}+{\begin{bmatrix}0\\1\end{bmatrix}}^{\otimes 3}\right)^{\otimes |V|}} = ( ( 0 , 1 , 1 , 1 ) [ 0 1 1 0 ] ⊗ 2 ) ⊗ | U | ( ( [ 0 1 1 0 ] [ 1 0 ] ) ⊗ 3 + ( [ 0 1 1 0 ] [ 0 1 ] ) ⊗ 3 ) ⊗ | V | {\displaystyle =\left((0,1,1,1){\begin{bmatrix}0&1\\1&0\end{bmatrix}}^{\otimes 2}\right)^{\otimes |U|}\left(\left({\begin{bmatrix}0&1\\1&0\end{bmatrix}}{\begin{bmatrix}1\\0\end{bmatrix}}\right)^{\otimes 3}+\left({\begin{bmatrix}0&1\\1&0\end{bmatrix}}{\begin{bmatrix}0\\1\end{bmatrix}}\right)^{\otimes 3}\right)^{\otimes |V|}} = ( 1 , 1 , 1 , 0 ) ⊗ | U | ( [ 0 1 ] ⊗ 3 + [ 1 0 ] ⊗ 3 ) ⊗ | V | {\displaystyle =(1,1,1,0)^{\otimes |U|}\left({\begin{bmatrix}0\\1\end{bmatrix}}^{\otimes 3}+{\begin{bmatrix}1\\0\end{bmatrix}}^{\otimes 3}\right)^{\otimes |V|}} = NAND 2 ⊗ | U | EQUAL 3 ⊗ | V | , {\displaystyle ={\text{NAND}}_{2}^{\otim

  Read more →
• 

  Reservoir sampling

  Reservoir sampling is a family of randomized algorithms for choosing a simple random sample, without replacement, of k items from a population of unknown size n in a single pass over the items. The size of the population n is not known to the algorithm and is typically too large for all n items to fit into main memory. The population is revealed to the algorithm over time, and the algorithm cannot look back at previous items. At any point, the current state of the algorithm must permit extraction of a simple random sample without replacement of size k over the part of the population seen so far. == Motivation == Suppose we see a sequence of items, one at a time. We want to keep 10 items in memory, and we want them to be selected at random from the sequence. If we know the total number of items n and can access the items arbitrarily, then the solution is easy: select 10 distinct indices i between 1 and n with equal probability, and keep the i-th elements. The problem is that we do not always know the exact n in advance. == Simple: Algorithm R == A simple and popular but slow algorithm, Algorithm R, was created by Jeffrey Vitter. Initialize an array R {\displaystyle R} indexed from 1 {\displaystyle 1} to k {\displaystyle k} , containing the first k items of the input x 1 , . . . , x k {\displaystyle x_{1},...,x_{k}} . This is the reservoir. For each new input x i {\displaystyle x_{i}} , generate a random number j uniformly in { 1 , . . . , i } {\displaystyle \{1,...,i\}} . If j ∈ { 1 , . . . , k } {\displaystyle j\in \{1,...,k\}} , then set R [ j ] := x i . {\displaystyle R[j]:=x_{i}.} Otherwise, discard x i {\displaystyle x_{i}} . Return R {\displaystyle R} after all inputs are processed. This algorithm works by induction on i ≥ k {\displaystyle i\geq k} . While conceptually simple and easy to understand, this algorithm needs to generate a random number for each item of the input, including the items that are discarded. The algorithm's asymptotic running time is thus O ( n ) {\displaystyle O(n)} . Generating this amount of randomness and the linear run time causes the algorithm to be unnecessarily slow if the input population is large. This is Algorithm R, implemented as follows: == Optimal: Algorithm L == If we generate n {\displaystyle n} random numbers u 1 , . . . , u n ∼ U [ 0 , 1 ] {\displaystyle u_{1},...,u_{n}\sim U[0,1]} independently, then the indices of the smallest k {\displaystyle k} of them is a uniform sample of the k {\displaystyle k} -subsets of { 1 , . . . , n } {\displaystyle \{1,...,n\}} . The process can be done without knowing n {\displaystyle n} : Keep the smallest k {\displaystyle k} of u 1 , . . . , u i {\displaystyle u_{1},...,u_{i}} that has been seen so far, as well as w i {\displaystyle w_{i}} , the index of the largest among them. For each new u i + 1 {\displaystyle u_{i+1}} , compare it with u w i {\displaystyle u_{w_{i}}} . If u i + 1 < u w i {\displaystyle u_{i+1} Read more →

• 

  Novell File Reporter

  Novell File Reporter (NFR) is software that allows network administrators to identify files stored on the network and generates reports regarding the size of individual files, file type, when files were last accessed, and where duplicates exist. Additionally, the File Reporter tracks storage volume capacity and usage. It is a component of the Novell File Management Suite. == How it works == Novell File Reporter examines and reports on terabytes of data via a central reporting engine (NFR Engine) and distributed agents (NFR Agents). The NFR Engine schedules the scans of file instances conducted by NFR Agents, processes and compiles the scans for reporting purposes, and provides report information to the user interface. In addition to the standard reports it can generate, the NFR Engine can also produce "trigger reports" in response to specific events (a server volume crossing a capacity threshold, for example). Accordingly, the NFR Engine monitors the data gathered by the NFR Agents in order to identify these "triggers." The NFR Engine when working in either eDirectory or Active Directory connects to the directory via a Directory Services Interface (DSI) and thus can monitor and check file permissions.

  Read more →
• 

  Pixel

  In digital imaging, a pixel (abbreviated px), pel, or picture element is the smallest addressable physical element of a raster image or the smallest controllable element of a display device or dot matrix printer. Pixels are arranged in a regular, two-dimensional grid, and each pixel serves as a sample of an original image, with a greater number of samples typically providing more accurate representations. Each pixel possesses a specific intensity or color, often composed of three or four component intensities, such as red, green, and blue (RGB), or cyan, magenta, yellow, and black (CMYK). The intensity of each pixel is variable, and in color imaging systems, these components are combined to produce a wide spectrum of colors. The concept of a picture element has existed since the early days of television, appearing as "Bildpunkt" in a 1888 German patent, and the term "pixel" has been used in various U.S. patents since 1911. In most digital display devices, pixels are the smallest element that can be manipulated through software. Each pixel is a sample of an original image; more samples typically provide more accurate representations of the original. The intensity of each pixel is variable. In color imaging systems, a color is typically represented by three or four component intensities such as red, green, and blue, or cyan, magenta, yellow, and black. In some contexts (such as descriptions of camera sensors), pixel refers to a single scalar element of a multi-component representation (called a photosite in the camera sensor context, although sensel 'sensor element' is sometimes used), while in yet other contexts (like MRI) it may refer to a set of component intensities for a spatial position. Software on early consumer computers was necessarily rendered at a low resolution, with large pixels visible to the naked eye; graphics made under these limitations may be called pixel art, especially in reference to video games. Modern computers and displays, however, can easily render orders of magnitude more pixels than was previously possible, necessitating the use of large measurements like the megapixel (one million pixels). == Etymology == The word pixel is a combination of pix (from "pictures", shortened to "pics") and el (for "element"); similar formations with 'el' include the words voxel 'volume pixel', and texel 'texture pixel'. The word pix appeared in Variety magazine headlines in 1932, as an abbreviation for the word pictures, in reference to movies. By 1938, "pix" was being used in reference to still pictures by photojournalists. The word "pixel" was first published in 1965 by Frederic C. Billingsley of JPL, to describe the picture elements of scanned images from space probes to the Moon and Mars. Billingsley had learned the word from Keith E. McFarland, at the Link Division of General Precision in Palo Alto, who in turn said he did not know where it originated. McFarland said simply it was "in use at the time" (c. 1963). The concept of a "picture element" dates to the earliest days of television, for example as "Bildpunkt" (the German word for pixel, literally 'picture point') in the 1888 German patent of Paul Nipkow. According to various etymologies, the earliest publication of the term picture element itself was in Wireless World magazine in 1927, though it had been used earlier in various U.S. patents filed as early as 1911. Some authors explain pixel as picture cell, as early as 1972. In graphics and in image and video processing, pel is often used instead of pixel. For example, IBM used it in their Technical Reference for the original PC. Pixilation, spelled with a second i, is an unrelated filmmaking technique that dates to the beginnings of cinema, in which live actors are posed frame by frame and photographed to create stop-motion animation. An archaic British word meaning "possession by spirits (pixies)", the term has been used to describe the animation process since the early 1950s; various animators, including Norman McLaren and Grant Munro, are credited with popularizing it. == Technical == A pixel is generally thought of as the smallest single component of a digital image. However, the definition is highly context-sensitive. For example, there can be "printed pixels" in a page, or pixels carried by electronic signals, or represented by digital values, or pixels on a display device, or pixels in a digital camera (photosensor elements). This list is not exhaustive and, depending on context, synonyms include pel, sample, byte, bit, dot, and spot. Pixels can be used as a unit of measure such as: 2400 pixels per inch, 640 pixels per line, or spaced 10 pixels apart. The measures "dots per inch" (dpi) and "pixels per inch" (ppi) are sometimes used interchangeably, but have distinct meanings, especially for printer devices, where dpi is a measure of the printer's density of dot (e.g. ink droplet) placement. For example, a high-quality photographic image may be printed with 600 ppi on a 1200 dpi inkjet printer. Even higher dpi numbers, such as the 4800 dpi quoted by printer manufacturers since 2002, do not mean much in terms of achievable resolution. The more pixels used to represent an image, the closer the result can resemble the original. The number of pixels in an image is sometimes called the resolution, though resolution has a more specific definition. Pixel counts can be expressed as a single number, as in a "three-megapixel" digital camera, which has a nominal three million pixels, or as a pair of numbers, as in a "640 by 480 display", which has 640 pixels from side to side and 480 from top to bottom (as in a VGA display) and therefore has a total number of 640 × 480 = 307,200 pixels, or 0.3 megapixels. The pixels, or color samples, that form a digitized image (such as a JPEG file used on a web page) may or may not be in one-to-one correspondence with screen pixels, depending on how a computer displays an image. In computing, an image composed of pixels is known as a bitmapped image or a raster image. The word raster originates from television scanning patterns, and has been widely used to describe similar halftone printing and storage techniques. === Sampling patterns === For convenience, pixels are normally arranged in a regular two-dimensional grid. By using this arrangement, many common operations can be implemented by uniformly applying the same operation to each pixel independently. Other arrangements of pixels are possible, with some sampling patterns even changing the shape (or kernel) of each pixel across the image. For this reason, care must be taken when acquiring an image on one device and displaying it on another, or when converting image data from one pixel format to another. For example: Liquid-crystal displays (LCDs) typically use a staggered grid, where the red, green, and blue components are sampled at slightly different locations. Subpixel rendering is a technology which takes advantage of these differences to improve the rendering of text on LCD screens. The vast majority of color digital cameras use a Bayer filter, resulting in a regular grid of pixels where the color of each pixel depends on its position on the grid. A clipmap uses a hierarchical sampling pattern, where the size of the support of each pixel depends on its location within the hierarchy. Warped grids are used when the underlying geometry is non-planar, such as images of the earth from space. The use of non-uniform grids is an active research area, attempting to bypass the traditional Nyquist limit. Pixels on computer monitors are normally "square" (that is, have equal horizontal and vertical sampling pitch); pixels in other systems are often "rectangular" (that is, have unequal horizontal and vertical sampling pitch – oblong in shape), as are digital video formats with diverse aspect ratios, such as the anamorphic widescreen formats of the Rec. 601 digital video standard. === Resolution of computer monitors === Computer monitors (and TV sets) generally have a fixed native resolution. What it is depends on the monitor, and size. See below for historical exceptions. Computers can use pixels to display an image, often an abstract image that represents a GUI. The resolution of this image is called the display resolution and is determined by the video card of the computer. Flat-panel monitors (and TV sets), e.g. OLED or LCD monitors, or E-ink, also use pixels to display an image, and have a native resolution, and it should (ideally) be matched to the video card resolution. Each pixel is made up of triads, with the number of these triads determining the native resolution. On older, historically available, CRT monitors the resolution was possibly adjustable (still lower than what modern monitor achieve), while on some such monitors (or TV sets) the beam sweep rate was fixed, resulting in a fixed native resolution. Most CRT monitors do not have a fixed beam sweep rate, meaning they do not have a native resolution at all – instead they

  Read more →
• 

  Bartels–Stewart algorithm

  In numerical linear algebra, the Bartels–Stewart algorithm is used to numerically solve the Sylvester matrix equation A X − X B = C {\displaystyle AX-XB=C} . Developed by R.H. Bartels and G.W. Stewart in 1971, it was the first numerically stable method that could be systematically applied to solve such equations. The algorithm works by using the real Schur decompositions of A {\displaystyle A} and B {\displaystyle B} to transform A X − X B = C {\displaystyle AX-XB=C} into a triangular system that can then be solved using forward or backward substitution. In 1979, G. Golub, C. Van Loan and S. Nash introduced an improved version of the algorithm, known as the Hessenberg–Schur algorithm. It remains a standard approach for solving Sylvester equations when X {\displaystyle X} is of small to moderate size. == The algorithm == Let X , C ∈ R m × n {\displaystyle X,C\in \mathbb {R} ^{m\times n}} , and assume that the eigenvalues of A {\displaystyle A} are distinct from the eigenvalues of B {\displaystyle B} . Then, the matrix equation A X − X B = C {\displaystyle AX-XB=C} has a unique solution. The Bartels–Stewart algorithm computes X {\displaystyle X} by applying the following steps: 1.Compute the real Schur decompositions R = U T A U , {\displaystyle R=U^{T}AU,} S = V T B T V . {\displaystyle S=V^{T}B^{T}V.} The matrices R {\displaystyle R} and S {\displaystyle S} are block-upper triangular matrices, with diagonal blocks of size 1 × 1 {\displaystyle 1\times 1} or 2 × 2 {\displaystyle 2\times 2} . 2. Set F = U T C V . {\displaystyle F=U^{T}CV.} 3. Solve the simplified system R Y − Y S T = F {\displaystyle RY-YS^{T}=F} , where Y = U T X V {\displaystyle Y=U^{T}XV} . This can be done using forward substitution on the blocks. Specifically, if s k − 1 , k = 0 {\displaystyle s_{k-1,k}=0} , then ( R − s k k I ) y k = f k + ∑ j = k + 1 n s k j y j , {\displaystyle (R-s_{kk}I)y_{k}=f_{k}+\sum _{j=k+1}^{n}s_{kj}y_{j},} where y k {\displaystyle y_{k}} is the k {\displaystyle k} th column of Y {\displaystyle Y} . When s k − 1 , k ≠ 0 {\displaystyle s_{k-1,k}\neq 0} , columns [ y k − 1 ∣ y k ] {\displaystyle [y_{k-1}\mid y_{k}]} should be concatenated and solved for simultaneously. 4. Set X = U Y V T . {\displaystyle X=UYV^{T}.} === Computational cost === Using the QR algorithm, the real Schur decompositions in step 1 require approximately 10 ( m 3 + n 3 ) {\displaystyle 10(m^{3}+n^{3})} flops, so that the overall computational cost is 10 ( m 3 + n 3 ) + 2.5 ( m n 2 + n m 2 ) {\displaystyle 10(m^{3}+n^{3})+2.5(mn^{2}+nm^{2})} . === Simplifications and special cases === In the special case where B = − A T {\displaystyle B=-A^{T}} and C {\displaystyle C} is symmetric, the solution X {\displaystyle X} will also be symmetric. This symmetry can be exploited so that Y {\displaystyle Y} is found more efficiently in step 3 of the algorithm. == The Hessenberg–Schur algorithm == The Hessenberg–Schur algorithm replaces the decomposition R = U T A U {\displaystyle R=U^{T}AU} in step 1 with the decomposition H = Q T A Q {\displaystyle H=Q^{T}AQ} , where H {\displaystyle H} is an upper-Hessenberg matrix. This leads to a system of the form H Y − Y S T = F {\displaystyle HY-YS^{T}=F} that can be solved using forward substitution. The advantage of this approach is that H = Q T A Q {\displaystyle H=Q^{T}AQ} can be found using Householder reflections at a cost of ( 5 / 3 ) m 3 {\displaystyle (5/3)m^{3}} flops, compared to the 10 m 3 {\displaystyle 10m^{3}} flops required to compute the real Schur decomposition of A {\displaystyle A} . == Software and implementation == The subroutines required for the Hessenberg-Schur variant of the Bartels–Stewart algorithm are implemented in the SLICOT library. These are used in the MATLAB control system toolbox. == Alternative approaches == For large systems, the O ( m 3 + n 3 ) {\displaystyle {\mathcal {O}}(m^{3}+n^{3})} cost of the Bartels–Stewart algorithm can be prohibitive. When A {\displaystyle A} and B {\displaystyle B} are sparse or structured, so that linear solves and matrix vector multiplies involving them are efficient, iterative algorithms can potentially perform better. These include projection-based methods, which use Krylov subspace iterations, methods based on the alternating direction implicit (ADI) iteration, and hybridizations that involve both projection and ADI. Iterative methods can also be used to directly construct low rank approximations to X {\displaystyle X} when solving A X − X B = C {\displaystyle AX-XB=C} .

  Read more →
• 

  Information flow

  In discourse-based grammatical theory, information flow is any tracking of referential information by speakers. Information may be new, i.e., just introduced into the conversation; given, i.e., already active in the speakers' consciousness; or old, i.e., no longer active. The various types of activation, and how these are defined, are model-dependent. Information flow affects grammatical structures such as: Word order (topic, focus, and afterthought constructions). Active, passive, or middle voice. Choice of deixis, such as articles; "medial" deictics such as Spanish ese and Japanese sore are generally determined by the familiarity of a referent rather than by physical distance. Overtness of information, such as whether an argument of a verb is indicated by a lexical noun phrase, a pronoun, or not mentioned at all. Clefting: Splitting a single clause into two clauses, each with its own verb, e.g. ‘The chicken turtles tasted like chicken.’ becomes ‘It was the chicken turtle | that tasted like chicken.’ In this case, clefting is used to shift the focus of the sentence to the subject, the chicken turtle. Front focus: Placing at the start (front) of a sentence information that would normally occur later in the sentence, to give it extra prominence. For example, in pop culture, Yoda's speech often utilizes such syntactic construction, such as when he says 'much to learn you still have' to Luke Skywalker. End focus (or end weight): Given or familiar information followed by new information. This gives prominence to the final part of the sentences and can enable suspense to build, e.g. ‘Through the door came a gigantic wolf’.(Umer Prince)

  Read more →
• 

  Algorithmic management

  Algorithmic management is a term used to describe certain labor management practices in the contemporary digital economy. In scholarly uses, the term was initially coined in 2015 by Min Kyung Lee, Daniel Kusbit, Evan Metsky, and Laura Dabbish to describe the managerial role played by algorithms on the Uber and Lyft platforms, but has since been taken up by other scholars to describe more generally the managerial and organisational characteristics of platform economies. However, digital direction of labor was present in manufacturing already since the 1970s and algorithmic management is becoming increasingly widespread across a wide range of industries. The concept of algorithmic management can be broadly defined as the delegation of managerial functions to algorithmic and automated systems. Algorithmic management has been enabled by "recent advances in digital technologies" which allow for the real-time and "large-scale collection of data" which is then used to "improve learning algorithms that carry out learning and control functions traditionally performed by managers". The term does not refer to a specific underlying technology, and encompasses the design choices, organisational policies, and governance that surround the managerial use of algorithms in workplaces. In the contemporary workplace, firms employ an ecology of accounting devices, such as "rankings, lists, classifications, stars and other symbols' in order to effectively manage their operations and create value without the need for traditional forms of hierarchical control." Many of these devices fall under the label of what is called algorithmic management, and were first developed by companies operating in the sharing economy or gig economy, functioning as effective labor and cost cutting measures. The Data&Society explainer of the term, for example, describes algorithmic management as 'a diverse set of technological tools and techniques that structure the conditions of work and remotely manage workforces. Data&Society also provides a list of five typical features of algorithmic management: Prolific data collection and surveillance of workers through technology; Real-time responsiveness to data that informs management decisions; Automated or semi-automated decision-making; Transfer of performance evaluations to rating systems or other metrics; and The use of "nudges" and penalties to indirectly incentivize worker behaviors. Proponents of algorithmic management claim that it "creates new employment opportunities, better and cheaper consumer services, transparency and fairness in parts of the labour market that are characterised by inefficiency, opacity and capricious human bosses." On the other hand, critics of algorithmic management claim that the practice leads to several issues, especially as it impacts the employment status of workers managed by its new array of tools and techniques. == History of the term == "Algorithmic management" was first described by Lee, Kusbit, Metsky, and Dabbish in 2015 in their study of the Uber and Lyft platforms. In their study, Lee et al. termed "software algorithms that assume managerial functions and surrounding institutional devices that support algorithms in practice" algorithmic management. Software algorithms, it was said, are increasingly used to "allocate, optimize, and evaluate work" by platforms in managing their vast workforces. In Lee et al.'s paper on Uber and Lyft this included the use of algorithms to assign work to drivers, as mechanisms to optimise pricing for services, and as systems for evaluating driver performance. In 2016, Alex Rosenblat and Luke Stark sought to extend on this understanding of algorithmic management "to elucidate on the automated implementation of company policies on the behaviours and practices of Uber drivers." Rosenblat and Stark found in their study that algorithmic management practices contributed to a system beset by power asymmetries, where drivers had little control over "critical aspects of their work", whereas Uber had far greater control over the labor of its drivers. Since this time, studies of algorithmic management have extended the use of the term to describe the management practices of various firms, where, for example, algorithms "are taking over scheduling work in fast food restaurants and grocery stores, using various forms of performance metrics ad even mood... to assign the fastest employees to work in peak times." Algorithmic management is seen to be especially prevalent in gig work on platforms, such as on Upwork and Deliveroo, and in the sharing economy, such as in the case of Airbnb. Furthermore, recent research has defined sub-constructs that fall under the umbrella term of algorithmic management, for example, "algorithmic nudging". A Harvard Business Review article published in 2021 explains: "Companies are increasingly using algorithms to manage and control individuals not by force, but rather by nudging them into desirable behavior — in other words, learning from their personalized data and altering their choices in some subtle way." While the concept builds on nudging theory popularized by University of Chicago economist Richard Thaler and Harvard Law School professor Cass Sunstein, "due to recent advances in AI and machine learning, algorithmic nudging is much more powerful than its non-algorithmic counterpart. With so much data about workers' behavioral patterns at their fingertips, companies can now develop personalized strategies for changing individuals' decisions and behaviors at large scale. These algorithms can be adjusted in real-time, making the approach even more effective." == Relationships with other labor management practices == Algorithmic management has been compared and contrasted with other forms of management, such as Scientific management approaches, as pioneered by Frederick Taylor in the early 1900s. Henri Schildt has called algorithmic management "Scientific management 2.0", where management "is no longer a human practice, but a process embedded in technology." Similarly, Kathleen Griesbach, Adam Reich, Luke Elliott-Negri, and Ruth Milkman suggest that, while "algorithmic control over labor may be relatively new, it replicates many features of older mechanisms of labor control." On the other hand, some commentators have argued that algorithmic management is not simply a new form of Scientific management or digital Taylorism, but represents a distinct approach to labor control in platform economies. David Stark and Ivana Pais, for example, state that, "In contrast to Scientific Management at the turn of the twentieth century, in the algorithmic management of the twenty-first century there are rules but these are not bureaucratic, there are rankings but not ranks, and there is monitoring but it is not disciplinary. Algorithmic management does not automate bureaucratic structures and practices to create some new form of algorithmic bureaucracy. Whereas the devices and practices of Taylorism were part of a system of hierarchical supervision, the devices and practices of algorithmic management take place within a different economy of attention and a new regime of visibility. Triangular rather than vertical, and not as a panopticon, the lines of vision in algorithmic management are not lines of supervision." Similarly, Data&Society's explainer for algorithmic management claims that the practice represents a marked departure from earlier management structures that more strongly rely on human supervisors to direct workers. In analyzing the difference and the similarities to previous management styles, David Stark and Pieter Vanden Broeck expand the applicability of algorithmic management beyond the workplace. They develop a theory of algorithmic management in terms of broader changes in the shape and structure of organization in the 21st century, attentive to the erosion of organization's boundaries whereby heterogeneous actors, assets, and activities, are coopted regardless of their place in organizational space. Stark and Vanden Broeck propose the following means of differentiating algorithmic management from other historical managerial paradigms: == Issues == Algorithmic management can provide an effective and efficient means of workforce control and value creation in the contemporary digital economy. However, commentators have highlighted several issues that algorithmic management poses, especially for the workers it manages. Criticisms of the practice often highlight several key issues pertaining to algorithmic management practices, such as the imperfection and scope of its surveillance and control measures, which also threaten to lock workers out of key decision-making processes; its lack of transparency for users and information asymmetries; its potential for bias and discrimination; its dehumanizing tendencies; and its potential to create conditions which sidestep traditional employer-employee accountability. This last point has been especi

  Read more →
• 

  Umoove

  Umoove is a high tech startup company that has developed and patented a software-only face and eye tracking technology. The idea was first conceived as an attempt to aid people with disabilities but has since evolved. The only compatibility qualification for tablet computers and smartphones to run Umoove software is a front-facing camera. Umoove headquarters are in Israel on Jerusalem’s Har Hotzvim. Umoove has 15 employees and received two million dollars in financing in 2012. The company's original founders invested around $800,000 to start the business in 2010. In 2013 Umoove was named one of the top three most promising Israeli start ups by Newsgeeks magazine. The company also participated in the 2013 LeWeb conference in Paris, France, where innovative technology startups are showcased. == Technology == The technology uses information extracted from previous frames, such as the angle of the user's head to predict where to look for facial targets in the next frame. This anticipation minimizes the amount of computation needed to scan each image. Umoove accounts for variances in environment, lighting conditions and user hand shake/movement. The technology is designed to provide a consistent experience, whether you're in a brightly lit area or a darkened basement, and to work fluidly between them by adapting its processing when it detects color and brightness shifts. It uses an active stabilization technique to filter out natural body movements from an unstable camera in order to minimize false-positive motion detection. Running the Umoove software on a Samsung Galaxy S3 is said to take up only 2% CPU. Umoove works exclusively with software and there is no hardware add-on necessary. It can be run on any smartphone or tablet computer that has a front-facing camera. Umoove claims that even a low-quality camera on an old device will run their software flawlessly. == Umoove Experience == In January 2014 Umoove released its first game onto the app store. The Umoove Experience game lets users control where they are 'flying' in the game through simple gestures and motions with their head. The avatar will basically go toward wherever the user looks. The game was created to showcase the technology for game developers but that did not stop some from criticizing its simplicity. Umoove also announced that they raised another one million dollars and that they are opening offices in Silicon Valley, California. In February 2014, Umoove announced that their face-tracking software development kit is available for Android developers as well as iOS. == Reviews == The Umoove Experience garnered mostly positive reviews from bloggers and mainstream media with some predicting that it could be the future of mobile gaming. Mashable wrote that Umoove's technology could be the emergence of gesture recognition technology in the mobile space, similar to Kinect with console gaming and what Leap Motion has done with desktop computers. Some, however, remain skeptical. CNET, for example, did not give the game a positive review and called the eye tracking technology 'freaky but cool'. They also noted that pioneering technologies have been known to fall short of expectations, citing Apple Inc’s Siri as an example. The technology blog GigaOM said that the Umoove Experience is ’awesome’ and technology evangelist Robert Scoble has called Umoove "brilliant". == uHealth == In January 2015, Umoove released uHealth, a mobile application that uses eye tracking game-like exercise to challenge the user's ability to be attentive, continuously focus, follow commands and avoid distractions. The app is designed in the form of two games, one to improve attention and another that hones focus. uHealth is a training tool, not a diagnostic. Umoove has stated that they want to use their technology for diagnosing neurological disorders but this will depend on clinical tests and FDA approval. The company cites the direct relationship between eye movements and brain activity as well as various vision-based therapies have been backed by many scientific studies conducted over the past decades. uHealth is the first time this type of therapy is delivered right to the end user through a simple download. == Collaboration rumors == In March 2013 there were rumors on the internet that Umoove would be the functioning software embedded into the Samsung Galaxy S4, which was due to launch that month. This rumor was perpetrated by, among others, New York Times, Techcrunch and Yahoo. Once Samsung launched without the Umoove technology rumors about a potential collaboration with Apple Inc hit the web. It has been said that due to the fact that Apple Inc is losing market share and stock value to Samsung they will be more aggressive and eye tracking is a logical place to make that move.

  Read more →
• 

  Knowledge organization system

  Knowledge organization system (KOS), concept system, or concept scheme is the generic term used in knowledge organization (KO) for the selection of concepts with an indication of selected semantic relations. Despite their differences in type, coverage, and application, all KOS aim to support the organization of knowledge and information to facilitate their management and retrieval. KOS vary in complexity from simple sorted lists to complex relational networks. They represent both structural and functional features, and serve to eliminate ambiguity, control synonyms, establish relationships, and present properties. From their origins in library and information science (LIS), KOS have been applied to other domains and disciplines within science and industry, although scholarly research and debate remain primarily within the KO field. Challenges of KOS include ambiguity of terminology, repercussions of biased systems, and potential obsolescence. KOS can be expressed in RDF and RDFS as per the Simple Knowledge Organization System (SKOS) recommendation by W3C, which aims to enable the sharing and linking of KOS via the Web. One of the largest collections of KOS is the BARTOC registry. == Types == While different schema of KOS have been proposed, most are generally arranged in terms of the complexity of their construction and maintenance. Some scholars argue that organizing KOS on a spectrum oversimplifies the shared characteristics among them, and may even result in a non-ideal structure being chosen. The following types are not exhaustive, and are often not mutually-exclusive in practice. === Term lists === Term lists are the least structured form of KOS. They include lists, glossaries, dictionaries, and synonym rings. Authority files and gazetteers may also be considered term lists, however other scholars categorize them and directories as "metadata-like models". Examples include the Union List of Artist Names name authority file and the GeoNames gazetteer. === Categorization and classification === KOS that emphasize specific (and often hierarchical) structures include subject headings, taxonomies, categorization schema, and classification schema & systems. Despite inconsistent use of the terms "categorization" and "classification" in some literature, categorization is generally loosely-assembled grouping schema and may include attributes that are not mutually exclusive (or having fuzzy boundaries), while classification is related to the arrangement of non-overlapping and mutually-exclusive classes. Classification schema may be universal (such as Dewey Decimal Classification and Information Coding Classification) or domain-specific (such as the National Library of Medicine Classification). === Relationship models === The types of KOS with greatest complexity and which utilize connections between concepts include thesauri, semantic networks, and ontologies. One of the most prominent examples of a semantic network is WordNet. === Others === Certain structures proposed to be considered types of KOS—but are not consistently included in schema—include folksonomies, topic maps, web directory structures, publication organization systems, and bibliometric maps. Some KOS organize other KOS themselves—for instance, PeriodO is a gazetteer of periodization categories. == Applications == Some early KOS were developed as a support system for abstracting and indexing services to be used by specially-trained searchers. With the growth of information digitization, usability became increasingly accessible, and more complex structures were developed. Prominent examples of KOS outside of LIS include organism taxonomy in biology, the periodic table of elements in chemistry, SIC and NAICS classification systems for industry & business, and AGROVOC agricultural controlled vocabulary. == Challenges == The study and design of KOS is an ongoing topic of discussion among KO scholars. === Terminology === [There is] a serious lack of vocabulary control in the literature on controlled vocabulary. Inconsistency of terminology within the study of KOS is a common issue. For instance, "ontology" is used for both a specific type of KOS as well as a generic term for any KOS. The terms "taxonomy", "classification", and "categorization" are also sometimes used interchangeably. === Bias === As knowledge can be historically and culturally biased, scholars have also discussed how KOS themselves can perpetuate harmful practices or stereotypes. For example, a number of concerns and criticisms about the classification of mental disorders in the Diagnostic and Statistical Manual of Mental Disorders have been raised, contributing to ongoing revisions. Ethical and intentional design approaches have been proposed for multi-perspective KOS in efforts to mitigate bias and other harmful practices. === Obsolescence === The possible obsolescence of the thesaurus and other simpler KOS has been the topic of debate, especially in the face of increasingly complex ontologies, the growing usage of "Google-like retrieval systems", and the move of KO theory and research away from LIS and toward computer science. Supporters of thesauri argue its continued usefulness for metadata enrichment, vocabulary mapping, and web services, as well as its usage in specific domains such as corporate intranets and digital image libraries.

  Read more →
• 

  Australian Geoscience Data Cube

  The Australian Geoscience Data Cube (AGDC) is an approach to storing, processing and analyzing large collections of Earth observation data. The technology is designed to meet challenges of national interest by being agile and flexible with vast amounts of layered grid data. The AGDC reduces processing time of traditional image analysis by calibrating, pre-computing known extents, pixel alignment and storing metadata in a cell lattice structure. The temporal-pixel aligned data can often be analysed faster across space and time dimensions than previous scene based techniques. This allows the AGDC to be flexible in tackling future challenges and improve analysis times on every-increasing data repositories of earth observation. The AGDC has also been used internationally to allow countries to maintain ecologically sustainable programs and reduce the difficulty curve of utilizing Remote Sensing data. == Background == The AGDC was originally conceived by Geoscience Australia but is now maintained in a partnership between Geoscience Australia, Commonwealth Scientific and Industrial Research Organisation (CSIRO) and National Computational Infrastructure National Facility (Australia) (NCI). This is made possible by the funding from the partnership and a number of organisations such as National Collaborative Research Infrastructure Strategy (NCRIS). == Analysis ready data, ingestion and indexing == The data processed in the cube is made analysis ready before being ingested and indexed into the AGDC. Analysis ready data is pre-processed data that has applied corrections for instrument calibration (gains and offsets), geolocation (spatial alignment) and radiometry (solar illumination, incidence angle, topography, atmospheric interference). The ingestion process manages the translation of datasets into the storage units while maintaining a database index. The data within the storage and index can be accessed via API calls often compiled within code such as Python (programming language). Example: s2a_l1c = dc.load(product='s2a_level1c_granule',x=(147.36, 147.41), y=(-35.1, -35.15), measurements=['04','03','02'], output_crs='EPSG:4326', resolution=(-0.00025,0.00025)) === Datasets currently stored === Geoscience Australia Landsat Surface Reflectance (1987 to present) Landsat Pixel Quality Landsat Fractional Cover Landsat NDVI === Datasets that have been piloted === USGS Landsat Surface Reflectance SRTM DEM Himawari 8 MODIS Sentinel-2 L1C / S2A Australian Gridded Climate Data == Open source == The AGDC code base is situated in GitHub as an open repository. The core code base moved to the Open Data Cube in early 2017 as part of an international collaboration. Whilst the code base is the Open Data Cube, individual cubes exist as their own right such as the AGDC on the National Computational Infrastructure National Facility (Australia) (NCI) using the High-Performance Computing Cluster HPCC. The core code can be installed on personal computers or public computers (using git) and has many unit tests. Documentation for the code base exists on Read the Docs. == Challenges of the AGDC == The AGDC is designed to meet nationally significant challenges such as the following. Sustainability Environment Water resource management Disaster assist Policy development Community planning Forest preservation Carbon measurement == International awards == The AGDC won the 2016 Content Platform of the Year award from Geospatial World Forum.

  Read more →
• 

  Parchive

  Parchive (a portmanteau of parity archive, and formally known as Parity Volume Set Specification) is an erasure code system that produces par files for checksum verification of data integrity, with the capability to perform data recovery operations that can repair or regenerate corrupted or missing data. Parchive was originally written to solve the problem of reliable file sharing on Usenet, but it can be used for protecting any kind of data from data corruption, disc rot, bit rot, and accidental or malicious damage. Despite the name, Parchive uses more advanced techniques (specifically error correction codes) than simplistic parity methods of error detection. As of 2015, PAR1 is obsolete, PAR2 is mature for widespread use, and PAR3 is a discontinued experimental version developed by MultiPar author Yutaka Sawada. The original SourceForge Parchive project has been inactive since April 30, 2015. A new PAR3 specification has been worked on since April 28, 2019 by PAR2 specification author Michael Nahas. An alpha version of the PAR3 specification has been published on January 29, 2022 while the program itself is being developed. == History == Parchive was intended to increase the reliability of transferring files via Usenet newsgroups. Usenet was originally designed for informal conversations, and the underlying protocol, NNTP was not designed to transmit arbitrary binary data. Another limitation, which was acceptable for conversations but not for files, was that messages were normally fairly short in length and limited to 7-bit ASCII text. Various techniques were devised to send files over Usenet, such as uuencoding and Base64. Later Usenet software allowed 8 bit Extended ASCII, which permitted new techniques like yEnc. Large files were broken up to reduce the effect of a corrupted download, but the unreliable nature of Usenet remained. With the introduction of Parchive, parity files could be created that were then uploaded along with the original data files. If any of the data files were damaged or lost while being propagated between Usenet servers, users could download parity files and use them to reconstruct the damaged or missing files. Parchive included the construction of small index files (.par in version 1 and .par2 in version 2) that do not contain any recovery data. These indexes contain file hashes that can be used to quickly identify the target files and verify their integrity. Because the index files were so small, they minimized the amount of extra data that had to be downloaded from Usenet to verify that the data files were all present and undamaged, or to determine how many parity volumes were required to repair any damage or reconstruct any missing files. They were most useful in version 1 where the parity volumes were much larger than the short index files. These larger parity volumes contain the actual recovery data along with a duplicate copy of the information in the index files (which allows them to be used on their own to verify the integrity of the data files if there is no small index file available). In July 2001, Tobias Rieper and Stefan Wehlus proposed the Parity Volume Set specification, and with the assistance of other project members, version 1.0 of the specification was published in October 2001. Par1 used Reed–Solomon error correction to create new recovery files. Any of the recovery files can be used to rebuild a missing file from an incomplete download. Version 1 became widely used on Usenet, but it did suffer some limitations: It was restricted to handle at most 255 files. The recovery files had to be the size of the largest input file, so it did not work well when the input files were of various sizes. (This limited its usefulness when not paired with the proprietary RAR compression tool.) The recovery algorithm had a bug, due to a flaw in the academic paper on which it was based. It was strongly tied to Usenet and it was felt that a more general tool might have a wider audience. In January 2002, Howard Fukada proposed that a new Par2 specification should be devised with the significant changes that data verification and repair should work on blocks of data rather than whole files, and that the algorithm should switch to using 16 bit numbers rather than the 8 bit numbers that PAR1 used. Michael Nahas and Peter Clements took up these ideas in July 2002, with additional input from Paul Nettle and Ryan Gallagher (who both wrote Par1 clients). Version 2.0 of the Parchive specification was published by Michael Nahas in September 2002. Peter Clements then went on to write the first two Par2 implementations, QuickPar and par2cmdline. Abandoned since 2004, Paul Houle created phpar2 to supersede par2cmdline. Yutaka Sawada created MultiPar to supersede QuickPar. MultiPar uses par2j.exe (which is partially based on par2cmdline's optimization techniques) to use as MultiPar's backend engine. == Versions == Versions 1 and 2 of the file format are incompatible. (However, many clients support both.) === Par1 === For Par1, the files f1, f2, ..., fn, the Parchive consists of an index file (f.par), which is CRC type file with no recovery blocks, and a number of "parity volumes" (f.p01, f.p02, etc.). Given all of the original files except for one (for example, f2), it is possible to create the missing f2 given all of the other original files and any one of the parity volumes. Alternatively, it is possible to recreate two missing files from any two of the parity volumes and so forth. Par1 supports up to a total of 256 source and recovery files. === Par2 === Par2 files generally use this naming/extension system: filename.vol000+01.PAR2, filename.vol001+02.PAR2, filename.vol003+04.PAR2, filename.vol007+06.PAR2, etc. The number after the "+" in the filename indicates how many blocks it contains, and the number after "vol" indicates the number of the first recovery block within the PAR2 file. If an index file of a download states that 4 blocks are missing, the easiest way to repair the files would be by downloading filename.vol003+04.PAR2. However, due to the redundancy, filename.vol007+06.PAR2 is also acceptable. There is also an index file filename.PAR2, it is identical in function to the small index file used in PAR1. Par2 specification supports up to 32,768 source blocks and up to 65,535 recovery blocks. Input files are split into multiple equal-sized blocks so that recovery files do not need to be the size of the largest input file. Although Unicode is mentioned in the PAR2 specification as an option, most PAR2 implementations do not support Unicode. Directory support is included in the PAR2 specification, but most or all implementations do not support it. === Par3 === The Par3 specification was originally planned to be published as an enhancement over the Par2 specification. However, to date, it has remained closed source by specification owner Yutaka Sawada. A discussion on a new format started in the GitHub issue section of the maintained fork par2cmdline on January 29, 2019. The discussion led to a new format which is also named as Par3. The new Par3 format's specification is published on GitHub, but remains being an alpha draft as of January 28, 2022. The specification is written by Michael Nahas, the author of Par2 specification, with the help from Yutaka Sawada, animetosho and malaire. The new format claims to have multiple advantages over the Par2 format, including support for: More than 216 files and more than 216 blocks. Packing small files into one block, as well as deduplication when a block appears in multiple files. UTF-8 file names. File permissions, hard links, symbolic/soft links, and empty directories. Embedding PAR data inside other formats, like ZIP archives or ISO disk images. "Incremental backups", where a user creates recovery files for some file or folder, change some data, and create new recovery files reusing some of the older files. More error correction code algorithms (such as LDPC and sparse random matrix). BLAKE3 hashes, dropping support for the MD5 hashes used in PAR2. == Software == === Multi-platform === par2+tbb (GPLv2) — a concurrent (multithreaded) version of par2cmdline 0.4 using TBB. Only compatible with x86 based CPUs. It is available in the FreeBSD Ports system as par2cmdline-tbb. Original par2cmdline — (obsolete). Available in the FreeBSD Ports system as par2cmdline. par2cmdline maintained fork by BlackIkeEagle. par2cmdline-mt is another multithreaded version of par2cmdline using OpenMP, GPLv2, or later. Currently merged into BlackIkeEagle's fork and maintained there. ParPar (CC0) is a high performance, multithreaded PAR2 client and Node.js library. Does not support verifying or repair, it can currently only create PAR2 archives. par2deep (LGPL-3.0) — Produce, verify and repair par2 files recursively, both on the command line as well as with the aid of a graphical user interface. It is available in the Python Package Index system as par2deep. par2cron (MIT License) is an o

  Read more →
• 

  Computational heuristic intelligence

  Computational heuristic intelligence (CHI) refers to specialized programming techniques in computational intelligence (also called artificial intelligence, or AI). These techniques have the express goal of avoiding complexity issues, also called NP-hard problems, by using human-like techniques. They are best summarized as the use of exemplar-based methods (heuristics), rather than rule-based methods (algorithms). Hence the term is distinct from the more conventional computational algorithmic intelligence, or symbolic AI. An example of a CHI technique is the encoding specificity principle of Tulving and Thompson. In general, CHI principles are problem solving techniques used by people, rather than programmed into machines. It is by drawing attention to this key distinction that the use of this term is justified in a field already replete with confusing neologisms. Note that the legal systems of all modern human societies employ both heuristics (generalisations of cases) from individual trial records as well as legislated statutes (rules) as regulatory guides. Another recent approach to the avoidance of complexity issues is to employ feedback control rather than feedforward modeling as a problem-solving paradigm. This approach has been called computational cybernetics, because (a) the term 'computational' is associated with conventional computer programming techniques which represent a strategic, compiled, or feedforward model of the problem, and (b) the term 'cybernetic' is associated with conventional system operation techniques which represent a tactical, interpreted, or feedback model of the problem. Of course, real programs and real problems both contain both feedforward and feedback components. A real example which illustrates this point is that of human cognition, which clearly involves both perceptual (bottom-up, feedback, sensor-oriented) and conceptual (top-down, feedforward, motor-oriented) information flows and hierarchies. The AI engineer must choose between mathematical and cybernetic problem solution and machine design paradigms. This is not a coding (program language) issue, but relates to understanding the relationship between the declarative and procedural programming paradigms. The vast majority of STEM professionals never get the opportunity to design or implement pure cybernetic solutions. When pushed, most responders will dismiss the importance of any difference by saying that all code can be reduced to a mathematical model anyway. Unfortunately, not only is this belief false, it fails most spectacularly in many AI scenarios. Mathematical models are not time agnostic, but by their very nature are pre-computed, i.e. feedforward. Dyer [2012] and Feldman [2004] have independently investigated the simplest of all somatic governance paradigms, namely control of a simple jointed limb by a single flexor muscle. They found that it is impossible to determine forces from limb positions- therefore, the problem cannot have a pre-computed (feedforward) mathematical solution. Instead, a top-down command bias signal changes the threshold feedback level in the sensorimotor loop, e.g. the loop formed by the afferent and efferent nerves, thus changing the so-called ‘equilibrium point’ of the flexor muscle/ elbow joint system. An overview of the arrangement reveals that global postures and limb position are commanded in feedforward terms, using global displacements (common coding), with the forces needed being computed locally by feedback loops. This method of sensorimotor unit governance, which is based upon what Anatol Feldman calls the ‘equilibrium Point’ theory, is formally equivalent to a servomechanism such as a car's ‘cruise control’.

  Read more →
• 

  Super column

  A super column is a tuple (a pair) with a binary super column name and a value that maps it to many columns. They consist of a key–value pairs, where the values are columns. Theoretically speaking, super columns are (sorted) associative array of columns. Similar to a regular column family where a row is a sorted map of column names and column values, a row in a super column family is a sorted map of super column names that maps to column names and column values. A super column is part of a keyspace together with other super columns and column families, and columns. == Code example == Written in the JSON-like syntax, a super column definition can be like this: Where: "databases" are keyspace; "Cassandra" and "HBase" are rowKeys; "name" and "address" are super column names; "firstName", "city", "age", etc. are column names.

  Read more →
• 

  Project workforce management

  Project workforce management is the practice of combining the coordination of all logistic elements of a project through a single software application (or workflow engine). This includes planning and tracking of schedules and mileposts, cost and revenue, resource allocation, as well as overall management of these project elements. Efficiency is improved by eliminating manual processes, like spreadsheet tracking to monitor project progress. It also allows for at-a-glance status updates and ideally integrates with existing legacy applications in order to unify ongoing projects, enterprise resource planning (ERP) and broader organizational goals. There are a lot of logistic elements in a project. Different team members are responsible for managing each element and often, the organisation may have a mechanism to manage some logistic areas as well. By coordinating these various components of project management, workforce management and financials through a single solution, the process of configuring and changing project and workforce details is simplified. == Introduction == A project workforce management system defines project tasks, project positions, and assigns personnel to the project positions. The project tasks and positions are correlated to assign a responsible project position or even multiple positions to complete each project task. Because each project position may be assigned to a specific person, the qualifications and availabilities of that person can be taken into account when determining the assignment. By associating project tasks and project positions, a manager can better control the assignment of the workforce and complete the project more efficiently. When it comes to project workforce management, it is all about managing all the logistic aspects of a project or an organisation through a software application. Usually, this software has a workflow engine defined. Therefore, all the logistic processes take place in the workflow engine. == About == === Technical field === This invention relates to project management systems and methods, more particularly to a software-based system and method for project and workforce management. === Software usage === Due to the software usage, all the project workflow management tasks can be fully automated without leaving many tasks for the project managers. This returns high efficiency to the project management when it comes to project tracking proposes. In addition to different tracking mechanisms, project workforce management software also offer a dashboard for the project team. Through the dashboard, the project team has a glance view of the overall progress of the project elements. Most of the times, project workforce management software can work with the existing legacy software systems such as ERP (enterprise resource planning) systems. This easy integration allows the organisation to use a combination of software systems for management purposes. === Background === Good project management is an important factor for the success of a project. A project may be thought of as a collection of activities and tasks designed to achieve a specific goal of the organisation, with specific performance or quality requirements while meeting any subject time and cost constraints. Project management refers to managing the activities that lead to the successful completion of a project. Furthermore, it focuses on finite deadlines and objectives. A number of tools may be used to assist with this as well as with assessment. Project management may be used when planning personnel resources and capabilities. The project may be linked to the objects in a professional services life cycle and may accompany the objects from the opportunity over quotation, contract, time and expense recording, billing, period-end-activities to the final reporting. Naturally the project gets even more detailed when moving through this cycle. For any given project, several project tasks should be defined. Project tasks describe the activities and phases that have to be performed in the project such as writing of layouts, customising, testing. What is needed is a system that allows project positions to be correlated with project tasks. Project positions describe project roles like project manager, consultant, tester, etc. Project-positions are typically arranged linearly within the project. By correlating project tasks with project positions, the qualifications and availability of personnel assigned to the project positions may be considered. == Benefits of project management == Good project management should: Reduce the chance of a project failing Ensure a minimum level of quality and that results meet requirements and expectations Free up other staff members to get on with their area of work and increase efficiency both on the project and within the business Make things simpler and easier for staff with a single point of contact running the overall project Encourage consistent communications amongst staff and suppliers Keep costs, timeframes and resources to budget == Workflow engine == When it comes to project workforce management, it is all about managing all the logistic aspects of a project or an organisation through a software application. Usually, this software has a workflow engine defined in them. So, all the logistic processes take place in the workflow engine. The regular and most common types of tasks handled by project workforce management software or a similar workflow engine are: === Planning and monitoring project schedules and milestones === Regularly monitoring your project's schedule performance can provide early indications of possible activity-coordination problems, resource conflicts, and possible cost overruns. To monitor schedule performance. Collecting information and evaluating it ensure a project accuracy. The project schedule outlines the intended result of the project and what's required to bring it to completion. In the schedule, we need to include all the resources involved and cost and time constraints through a work breakdown structure (WBS). The WBS outlines all the tasks and breaks them down into specific deliverables. === Tracking the cost and revenue aspects of projects === The importance of tracking actual costs and resource usage in projects depends upon the project situation. Tracking actual costs and resource usage is an essential aspect of the project control function. === Resource utilisation and monitoring === Organisational profitability is directly connected to project management efficiency and optimal resource utilisation. To sum up, organisations that struggle with either or both of these core competencies typically experience cost overruns, schedule delays and unhappy customers. The focus for project management is the analysis of project performance to determine whether a change is needed in the plan for the remaining project activities to achieve the project goals. == Other management aspects of project management == === Project risk management === Risk identification consists of determining which risks are likely to affect the project and documenting the characteristics of each. === Project communication management === Project communication management is about how communication is carried out during the course of the project === Project quality management === It is of no use completing a project within the set time and budget if the final product is of poor quality. The project manager has to ensure that the final product meets the quality expectations of the stakeholders. This is done by good: Quality planning: Identifying what quality standards are relevant to the project and determining how to meet them. Quality assurance: Evaluating overall project performance on a regular basis to provide confidence that the project will satisfy the relevant quality standards. Quality control: Monitoring specific project results to determine if they comply with relevant quality standards and identifying ways to remove causes of poor performance. == Project workforce management vs. traditional management == There are three main differences between Project Workforce Management and traditional project management and workforce management disciplines and solutions: === Workflow-driven === All project and workforce processes are designed, controlled and audited using a built-in graphical workflow engine. Users can design, control and audit the different processes involved in the project. The graphical workflow is quite attractive for the users of the system and allows the users to have a clear idea of the workflow engine. === Organisation and work breakdown structures === Project Workforce Management provides organization and work breakdown structures to create, manage and report on functional and approval hierarchies, and to track information at any level of detail. Users can create, manage, edit and report work breakdown structures. Work breakdown structures have different abstraction

  Read more →