AI Generator Letter

AI Generator Letter — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

  • YaDICs

    YaDICs

    YaDICs is a program written to perform digital image correlation on 2D and 3D tomographic images. The program was designed to be both modular, by its plugin strategy and efficient, by it multithreading strategy. It incorporates different transformations (Global, Elastic, Local), optimizing strategy (Gauss-Newton, Steepest descent), Global and/or local shape functions (Rigid-body motions, homogeneous dilatations, flexural and Brazilian test models)... == Theoretical background == === Context === In solid mechanics, digital image correlation is a tool that allows to identify the displacement field to register a reference image (called herein fixed image) to images during an experiment (mobile image). For example, it is possible to observe the face of a specimen with a painted speckle on it in order to determine its displacement fields during a tensile test. Before the appearance of such methods, researchers usually used strain gauges to measure the mechanical state of the material but strain gauges only measure the strain on a point and don't allow to understand material with an heterogeneous behavior. One can obtain a full in plane strain tensor by derivation of the displacement fields. Many methods are based upon the optical flow. In fluid mechanics a similar method is used, called Particle Image Velocimetry (PIV); the algorithms are similar to those of DIC but it is impossible to ensure that the optical flow is conserved so a vast majority of the software used the normalized cross correlation metric. In mechanics the displacement or velocity fields are the only concern, registering images is just a side effect. There is another process called image registration using the same algorithms (on monomodal images) but where the goal is to register images and thereby identifying the displacement field is just a side effect. YaDICs uses the general principle of image registration with a particular attention to the displacement fields basis. === Image registration principle === YaDICs can be explained using the classical image registration framework: === Image registration general scheme === The common idea of image registration and digital image correlation is to find the transformation between a fixed image and a moving one for a given metric using an optimization scheme. While there are many methods to achieve such a goal, Yadics focuses on registering images with the same modality. The idea behind the creation of this software is to be able to process data that comes from a μ-tomograph; i.e.: data cube over 10003 voxels. With such a size it is not possible to use naive approach usually used in a two-dimensional context. In order to get sufficient performances OpenMP parallelism is used and data are not globally stored in memory. As an extensive description of the different algorithms is given in. === Sampling === Contrary to image registration, Digital Image Correlation targets the transformation, one wants to extracted the most accurate transformation from the two images and not just match the images. Yadics uses the whole image as a sampling grid: it is thus a total sampling. === Interpolator === It is possible to choose between bilinear interpolation and bicubic interpolation for the grey level evaluation at non integer coordinates. The bi-cubic interpolation is the recommended one. === Metrics === ==== Sum of squared differences (SSD) ==== The SSD is also known as mean squared error. The equation below defines the SSD metric: S S D ( μ , I F , I M ) = 1 | Ω F | ∑ x i ∈ Ω F ( I F ( x i ) − I M ( T μ ( x i ) ) ) 2 , {\displaystyle SSD(\mu ,{\mathcal {I_{F}}},{\mathcal {I_{M}}})={\dfrac {1}{\left|\Omega _{F}\right|}}\sum _{x_{i}\in \Omega _{F}}\left({\mathcal {I_{F}}}(x_{i})-{\mathcal {I_{M}}}({T}_{\mu }(x_{i}))\right)^{2},} where I F {\displaystyle {\mathcal {I_{F}}}} is the fixed image, I M {\displaystyle {\mathcal {I_{M}}}} the moving one, Ω F {\displaystyle \Omega _{F}} the integration area | Ω F | {\displaystyle \left|\Omega _{F}\right|} the number of pi(vo)xels (cardinal) and T μ {\displaystyle {T}_{\mu }} the transformation parametrized by μ The transformation can be written as: T μ ( x ) = x + { Φ ( x ) } t { μ } . {\displaystyle T_{\mu }(x)=x+\left\{\Phi (x)\right\}^{t}\left\{\mu \right\}.} This metric is the main one used in the YaDICs as it works well with same modality images. One has to find the minimum of this metric ==== Normalized cross-correlation ==== The normalized cross-correlation (NCC) is used when one cannot assure the optical flow conservation; it happens in case of change of lighting or if particles disappear from the scene can occur in particle images velocimetry (PIV). The NCC is defined by: N C C ( μ , I F , I M ) = ∑ x i ∈ Ω F ( I F ( x i ) − I F ¯ ) ( I M ( T μ ( x i ) ) − I M ¯ ) ∑ x i ∈ Ω F ( I F ( x i ) − I F ¯ ) 2 ∑ x i ∈ Ω F ( I M ( T μ ( x i ) ) − I M ¯ ) 2 , {\displaystyle NCC(\mu ,{\mathcal {I_{F}}},{\mathcal {I_{M}}})={\dfrac {\sum _{x_{i}\in \Omega _{F}}\left({\mathcal {I_{F}}}(x_{i})-{\overline {\mathcal {I_{F}}}}\right)\left({\mathcal {I_{M}}}({T}_{\mu }(x_{i}))-{\overline {\mathcal {I_{M}}}}\right)}{\sqrt {\sum _{x_{i}\in \Omega _{F}}\left({\mathcal {I_{F}}}(x_{i})-{\overline {\mathcal {I_{F}}}}\right)^{2}\sum _{x_{i}\in \Omega _{F}}\left({\mathcal {I_{M}}}({T}_{\mu }(x_{i}))-{\overline {\mathcal {I_{M}}}}\right)^{2}}}},} where I F ¯ {\displaystyle {\overline {\mathcal {I_{F}}}}} and I M ¯ {\displaystyle {\overline {\mathcal {I_{M}}}}} are the mean values of the fixed and mobile images. This metric is only used to find local translation in Yadics. This metric with translation transform can be solved using cross-correlation methods, which are non iterative and can be accelerated using Fast Fourier Transform . === Classification of transformations === There are three categories of parametrization: elastic, global and local transformation. The elastic transformations respect the partition of unity, there are no holes created or surfaces counted several times. This is commonly used in Image Registration by the use of B-Spline functions and in solid mechanics with finite element basis. The global transformations are defined on the whole picture using rigid body or affine transformation (which is equivalent to homogeneous strain transformation). More complex transformations can be defined such as mechanically based one. These transformations have been used for stress intensity factor identification by and for rod strain by. The local transformation can be considered as the same global transformation defined on several Zone Of Interest (ZOI) of the fixed image. ==== Global ==== Several global transforms have been implemented: Rigid and homogeneous (Tx,Ty,Rz in 2D; Tx,Ty,Tz,Rx,Ry,Rz,Exx,Eyy,Ezz,Eyz,Exz,Exy in 3D) Brazilian (Only in 2D), Dynamic Flexion, ==== Elastic ==== First-order quadrangular finite elements Q4P1 are used in Yadics. ===== Local ===== Every global transform can be used on a local mesh. === Optimization === The YaDICs optimization process follows a gradient descent scheme. The first step is to compute the gradient of the metric regarding the transform parameters ∂ S S D ( μ , I F , I M ) ∂ μ = 2 | Ω F | ∑ x i ∈ Ω F ( I F ( x i ) − I M ( T μ ( x i ) ) ) ∂ I M ( T μ ( x i ) ∂ μ = 2 | Ω F | ∑ x i ∈ Ω F ( I F ( x i ) − I M ( T μ ( x i ) ) ) ( ∂ T μ ( x i ) ∂ μ ) t ∂ I M ( T μ ( x i ) ) ∂ x {\displaystyle {\begin{array}{lcl}{\dfrac {\partial SSD(\mu ,{\mathcal {I_{F}}},{\mathcal {I_{M}}})}{\partial \mu }}&=&{\dfrac {2}{\left|\Omega _{F}\right|}}\sum _{x_{i}\in \Omega _{F}}\left({\mathcal {I_{F}}}(x_{i})-{\mathcal {I_{M}}}({T}_{\mu }(x_{i}))\right){\dfrac {\partial {\mathcal {I_{M}}}({T}_{\mu }(x_{i})}{\partial \mu }}\\&=&{\dfrac {2}{\left|\Omega _{F}\right|}}\sum _{x_{i}\in \Omega _{F}}\left({\mathcal {I_{F}}}(x_{i})-{\mathcal {I_{M}}}({T}_{\mu }(x_{i}))\right)\left({\dfrac {\partial {T}_{\mu }(x_{i})}{\partial \mu }}\right)^{t}{\dfrac {\partial {\mathcal {I_{M}}}({T}_{\mu }(x_{i}))}{\partial x}}\\\end{array}}} ==== Gradient method ==== Once the metric gradient has been computed, one has to find an optimization strategy The gradient method principle is explained below: μ k + 1 = μ k + α k d k {\displaystyle \mu _{k+1}=\mu _{k}+\alpha _{k}d_{k}} The gradient step can be constant or updated at every iteration. d k = − γ k ∂ C ( μ , I F , I M ) ∂ μ {\displaystyle d_{k}=-\gamma _{k}{\dfrac {\partial {\mathcal {C}}(\mu ,{\mathcal {I_{F}}},{\mathcal {I_{M}}})}{\partial \mu }}} , γ k {\displaystyle \gamma _{k}} allows one to choose between the following methods : γ k {\displaystyle \gamma _{k}} ⟹ {\displaystyle \Longrightarrow } steepest descent, γ k = [ ∂ C ( μ , I F , I M ) ∂ μ ∂ C ( μ , I F , I M ) ∂ μ t ] − 1 {\displaystyle \gamma _{k}=\left[{\dfrac {\partial {\mathcal {C}}(\mu ,{\mathcal {I_{F}}},{\mathcal {I_{M}}})}{\partial \mu }}{\dfrac {\partial {\mathcal {C}}(\mu ,{\mathcal {I_{F}}},{\mathcal {I_{M}}})}{\partial \mu }}^{t}\right]^{-1}} ⟹ {\displaystyle \Longrightarrow } Gauss-Newto

    Read more →
  • Oracle Cloud

    Oracle Cloud

    Oracle Cloud is a cloud computing service offered by Oracle Corporation providing servers, storage, network, applications and services through a global network of Oracle Corporation managed data centers. The company allows these services to be provisioned on demand over the Internet. Oracle Cloud provides infrastructure as a service (IaaS), platform as a service (PaaS), software as a service (SaaS), and data as a service (DaaS). These services are used to build, deploy, integrate, and extend applications in the cloud. This platform supports numerous open standards (SQL, HTML5, REST, etc.), open-source applications (Kubernetes, Spark, Hadoop, Kafka, MySQL, Terraform, etc.), and a variety of programming languages, databases, tools, and frameworks including Oracle-specific, open source, and third-party software and systems. == Services == === Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) === Oracle's cloud infrastructure was made generally available (GA) on October 20, 2016 under the name "Oracle Bare Metal Cloud Services". Oracle Bare Metal Cloud Services was rebranded as Oracle Cloud Infrastructure in 2018 and dubbed Oracle's "Generation 2 Cloud" at Oracle OpenWorld 2018. Oracle Cloud Infrastructure offerings include the following services: Compute: The company provides Virtual Machine Instances to provide different shapes (VM sizes) catering to different types of workloads and performance characteristics. They also provide on-demand Bare metal servers and Bare metal GPU servers, without a hypervisor. In 2016, Oracle Cloud Infrastructure launched with bare metal instances with Intel processors. These first bare metal instances offered were powered by Intel servers. In 2018, Oracle Cloud added bare metal instances powered by AMD processors, followed by Ampere Cloud-native processors in 2021. In 2021, Oracle also released its first VM-based compute instances based on Arm processors. Storage: The platform provides block volumes, file storage, object storage, and archive storage for database, analytics, content, and other applications across common protocols and APIs. Networking: This cloud platform provides network with fully configurable IP addresses, subnets, routing, and firewalls to support new or existing private networks with end-to-end security. Governance: For auditing, identity and access management, the platform has data integrity checks, traceability, and access management features. Database Management / Data Management: Oracle offers a data management platform for database workloads as well as hyper-scale big data and streaming workloads including OLTP, data warehousing, Spark, machine learning, text search, image analytics, data catalog, and deep learning. The platform allows Oracle, MySQL, and NoSQL databases to be deployed on demand as managed cloud services. Oracle Databases uniquely offer the Oracle Autonomous Database (optimized for data warehouse, transaction processing, or JSON), the Exadata shape, as well as Real Application Clusters (RAC). Load Balancing: The cloud platform offers load balancing capability to automatically route traffic across fault domains and availability domains for high availability and fault-tolerance for hosted applications. Edge Services: These services can monitor the path between users and resources and adapt to changes and outages. They include Domain Name System (DNS) services from Oracle's acquisition of Dyn. FastConnect: The cloud platform provides private connectivity across on-premises and cloud networks through providers like Equinix, AT&T, and Colt. Application Development: For application development, the company's cloud offers an open, standards-based application development platform to build, deploy, and manage API-first, mobile-first cloud applications. This platform supports container-native, cloud-native, and low code development. This platform also provides a DevOps platform for CI/CD, diagnostics for Java applications, and integration with SaaS and on-prem applications. Services include Java, mobile, digital assistants (evolution from chatbots), messaging, application container cloud, developer cloud, visual builder, API catalog, AI platform, DataScience.com (Oracle acquired) and blockchain. Integration: This is a platform offering with adapters to integrate on-premise and cloud applications. Capabilities include data integration and replication, API management, integration analytics, along with data migration and integration. They offer services such as data integration platform cloud, data integrator cloud service, GoldenGate cloud service, integration cloud, process cloud service, API platform cloud service, apiary cloud service, and SOA cloud service. Business Analytics: The company provides this business analytics platform which can analyze and generate insights from data across various applications, data warehouses, and data lakes. The services offered include analytics cloud, business intelligence, big data discovery, big data preparation, data visualization, and essbase. Security: The Oracle Cloud Platform provides identity and security applications for providing secure access and monitoring of hybrid cloud environment and addressing IT governance and compliance requirements. This platform delivers an identity SOC (Security Operations Center) through a combined offering of SIEM, UEBA, CASB, and IDaaS. The services offered include Identity Cloud Service and CASB Cloud Service. Management: The platform provides an integrated monitoring, management, and analytics platform. This platform also uses machine learning and big data on the operational data set. The platform is used to improve IT stability, prevent application outages, improve DevOps, and harden security. Services offered include Application Performance Monitoring, Infrastructure Monitoring, Log Analytics, Orchestration, IT Analytics, Configuration and Compliance, Security Monitoring, and Analytics. Content and Experience: This is a platform for content, website, and workflow management. This service is used to provide content collaboration and web presence. This tool comes integrated with Oracle on-premise and SaaS services. The services offered are Content and Experience Cloud, WebCenter Portal Cloud, and DIVA Cloud. In 2016, Oracle acquired Dyn, an internet infrastructure company. On May 16, 2018 Oracle announced that it had acquired DataScience.com, a privately held cloud workspace platform for data science projects and workloads. In April 2020, Oracle became the cloud infrastructure provider for Zoom, an online and video meeting platform. The same month, Nissan announced its migration to Oracle Cloud for its high-performance computing (HPC) workloads used for simulating the structural impacts of a car design. Xerox announced a partnership with Oracle Cloud in 2021, where Xerox will use Oracle's cloud-computing capabilities within its business incubator. === Software as a Service (SaaS) === Oracle provides SaaS applications also known as Oracle Cloud Applications. These applications are offered across a variety of products, industrial sectors with various deployment options to adhere to compliance standards. The below list mentions Oracle Cloud Applications provided by Oracle Corporation. Customer Experience (CX) Human Capital Management (HCM) Enterprise Resource Planning (ERP) Supply Chain Management (SCM) Enterprise Performance Management (EPM) Internet of Things Applications (IoT) SaaS Analytics Data Industry Solutions (Communications, Financial Services, Consumer Goods, High Tech and Manufacturing, Higher Education, Hospitality, Utilities) Deployment (adhering to standards for sectors such as Financial Services, Retail Services, Public Sector, Defense) Block-Chain Cloud Service (in partnership with SAP, IBM and Microsoft) Blockchain Applications On July 28, 2016 Oracle bought NetSuite, the very first cloud company, for $9.3 billion. === Data as a Service (DaaS) === This platform is known as the Oracle Data Cloud. This platform aggregates and analyzes consumer data powered by Oracle ID Graph across channels and devices to create cross-channel consumer understanding. == Deployment models == Oracle Cloud is available in 44 regions as of July 2023, including North America, South America, UK, European Union, Middle East, Africa, India, Australia, Korea, and Japan. Oracle Cloud is available as a public cloud (Oracle-managed regions); to selected government agencies as an Oracle-managed government cloud in the United States (with FedRAMP High and DISA SRG IL5 compliance) and United Kingdom; and as a "private cloud" or "hybrid cloud" as an Oracle-managed database-only service or full-service dedicated region - what Oracle calls "Cloud at Customer". == Architecture == Oracle's public and government cloud is offered through a global network of Oracle-managed data centers, connected by an Oracle-managed backbone network. Oracle's Exadata Cloud at Customer leverages this network for contr

    Read more →
  • Convolution

    Convolution

    In mathematics (in particular, functional analysis), convolution is a mathematical operation on two functions f {\displaystyle f} and g {\displaystyle g} that produces a third function f ∗ g {\displaystyle fg} , as the integral of the product of the two functions after one is reflected about the y-axis and shifted. The term convolution refers to both the resulting function and to the process of computing it. The integral is evaluated for all values of shift, producing the convolution function. The choice of which function is reflected and shifted before the integral does not change the integral result (see commutativity). Graphically, it expresses how the 'shape' of one function is modified by the other. Some features of convolution are similar to cross-correlation: for real-valued functions, of a continuous or discrete variable, convolution f ∗ g {\displaystyle fg} differs from cross-correlation f ⋆ g {\displaystyle f\star g} only in that either f ( x ) {\displaystyle f(x)} or g ( x ) {\displaystyle g(x)} is reflected about the y-axis in convolution; thus it is a cross-correlation of g ( − x ) {\displaystyle g(-x)} and f ( x ) {\displaystyle f(x)} , or f ( − x ) {\displaystyle f(-x)} and g ( x ) {\displaystyle g(x)} . For complex-valued functions, the cross-correlation operator is the adjoint of the convolution operator. Convolution has applications that include probability, statistics, acoustics, spectroscopy, signal processing and image processing, computer vision and human vision, geophysics, engineering, physics, and differential equations. The convolution can be defined for functions on Euclidean space and other groups (as algebraic structures). For example, periodic functions, such as the discrete-time Fourier transform, can be defined on a circle and convolved by periodic convolution. (See row 18 at DTFT § Properties.) A discrete convolution can be defined for functions on the set of integers. Generalizations of convolution have applications in the field of numerical analysis and numerical linear algebra, and in the design and implementation of finite impulse response filters in signal processing. Computing the inverse of the convolution operation is known as deconvolution. == Definition == The convolution of f {\displaystyle f} and g {\displaystyle g} is written f ∗ g {\displaystyle fg} , denoting the operator with the symbol ∗ {\displaystyle } . It is defined as the integral of the product of the two functions after one is reflected about the y-axis and shifted. As such, it is a particular kind of integral transform: ( f ∗ g ) ( t ) := ∫ − ∞ ∞ f ( τ ) g ( t − τ ) d τ . {\displaystyle (fg)(t):=\int _{-\infty }^{\infty }f(\tau )g(t-\tau )\,d\tau .} An equivalent definition is (see commutativity): ( f ∗ g ) ( t ) := ∫ − ∞ ∞ f ( t − τ ) g ( τ ) d τ . {\displaystyle (fg)(t):=\int _{-\infty }^{\infty }f(t-\tau )g(\tau )\,d\tau .} While the symbol t {\displaystyle t} is used above, it need not represent the time domain. At each t {\displaystyle t} , the convolution formula can be described as the area under the function f ( τ ) {\displaystyle f(\tau )} weighted by the function g ( − τ ) {\displaystyle g(-\tau )} shifted by the amount t {\displaystyle t} . As t {\displaystyle t} changes, the weighting function g ( t − τ ) {\displaystyle g(t-\tau )} emphasizes different parts of the input function f ( τ ) {\displaystyle f(\tau )} ; If t {\displaystyle t} is a positive value, then g ( t − τ ) {\displaystyle g(t-\tau )} is equal to g ( − τ ) {\displaystyle g(-\tau )} that slides or is shifted along the τ {\displaystyle \tau } -axis toward the right (toward + ∞ {\displaystyle +\infty } ) by the amount of t {\displaystyle t} , while if t {\displaystyle t} is a negative value, then g ( t − τ ) {\displaystyle g(t-\tau )} is equal to g ( − τ ) {\displaystyle g(-\tau )} that slides or is shifted toward the left (toward − ∞ {\displaystyle -\infty } ) by the amount of | t | {\displaystyle |t|} . For functions f {\displaystyle f} , g {\displaystyle g} supported on only [ 0 , ∞ ) {\displaystyle [0,\infty )} (i.e., zero for negative arguments), the integration limits can be truncated, resulting in: ( f ∗ g ) ( t ) = ∫ 0 t f ( τ ) g ( t − τ ) d τ for f , g : [ 0 , ∞ ) → R . {\displaystyle (fg)(t)=\int _{0}^{t}f(\tau )g(t-\tau )\,d\tau \quad \ {\text{for }}f,g:[0,\infty )\to \mathbb {R} .} For the multi-dimensional formulation of convolution, see domain of definition (below). === Notation === A common engineering notational convention is: f ( t ) ∗ g ( t ) := ∫ − ∞ ∞ f ( τ ) g ( t − τ ) d τ ⏟ ( f ∗ g ) ( t ) , {\displaystyle f(t)g(t)\mathrel {:=} \underbrace {\int _{-\infty }^{\infty }f(\tau )g(t-\tau )\,d\tau } _{(fg)(t)},} which has to be interpreted carefully to avoid confusion. For instance, f ( t ) ∗ g ( t − t 0 ) {\displaystyle f(t)g(t-t_{0})} is equivalent to ( f ∗ g ) ( t − t 0 ) {\displaystyle (fg)(t-t_{0})} , but f ( t − t 0 ) ∗ g ( t − t 0 ) {\displaystyle f(t-t_{0})g(t-t_{0})} is in fact equivalent to ( f ∗ g ) ( t − 2 t 0 ) {\displaystyle (fg)(t-2t_{0})} . === Relations with other transforms === Given two functions f ( t ) {\displaystyle f(t)} and g ( t ) {\displaystyle g(t)} with bilateral Laplace transforms (two-sided Laplace transform) F ( s ) = ∫ − ∞ ∞ e − s u f ( u ) d u {\displaystyle F(s)=\int _{-\infty }^{\infty }e^{-su}\ f(u)\ {\text{d}}u} and G ( s ) = ∫ − ∞ ∞ e − s v g ( v ) d v {\displaystyle G(s)=\int _{-\infty }^{\infty }e^{-sv}\ g(v)\ {\text{d}}v} respectively, the convolution operation ( f ∗ g ) ( t ) {\displaystyle (fg)(t)} can be defined as the inverse Laplace transform of the product of F ( s ) {\displaystyle F(s)} and G ( s ) {\displaystyle G(s)} . More precisely, F ( s ) ⋅ G ( s ) = ∫ − ∞ ∞ e − s u f ( u ) d u ⋅ ∫ − ∞ ∞ e − s v g ( v ) d v = ∫ − ∞ ∞ ∫ − ∞ ∞ e − s ( u + v ) f ( u ) g ( v ) d u d v {\displaystyle {\begin{aligned}F(s)\cdot G(s)&=\int _{-\infty }^{\infty }e^{-su}\ f(u)\ {\text{d}}u\cdot \int _{-\infty }^{\infty }e^{-sv}\ g(v)\ {\text{d}}v\\&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }e^{-s(u+v)}\ f(u)\ g(v)\ {\text{d}}u\ {\text{d}}v\end{aligned}}} Let t = u + v {\displaystyle t=u+v} , then F ( s ) ⋅ G ( s ) = ∫ − ∞ ∞ ∫ − ∞ ∞ e − s t f ( u ) g ( t − u ) d u d t = ∫ − ∞ ∞ e − s t ∫ − ∞ ∞ f ( u ) g ( t − u ) d u ⏟ ( f ∗ g ) ( t ) d t = ∫ − ∞ ∞ e − s t ( f ∗ g ) ( t ) d t . {\displaystyle {\begin{aligned}F(s)\cdot G(s)&=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }e^{-st}\ f(u)\ g(t-u)\ {\text{d}}u\ {\text{d}}t\\&=\int _{-\infty }^{\infty }e^{-st}\underbrace {\int _{-\infty }^{\infty }f(u)\ g(t-u)\ {\text{d}}u} _{(fg)(t)}\ {\text{d}}t\\&=\int _{-\infty }^{\infty }e^{-st}(fg)(t)\ {\text{d}}t.\end{aligned}}} Note that F ( s ) ⋅ G ( s ) {\displaystyle F(s)\cdot G(s)} is the bilateral Laplace transform of ( f ∗ g ) ( t ) {\displaystyle (fg)(t)} . A similar derivation can be done using the unilateral Laplace transform (one-sided Laplace transform). The convolution operation also describes the output (in terms of the input) of an important class of operations known as linear time-invariant (LTI). See LTI system theory for a derivation of convolution as the result of LTI constraints. In terms of the Fourier transforms of the input and output of an LTI operation, no new frequency components are created. The existing ones are only modified (amplitude and/or phase). In other words, the output transform is the pointwise product of the input transform with a third transform (known as a transfer function). See Convolution theorem for a derivation of that property of convolution. Conversely, convolution can be derived as the inverse Fourier transform of the pointwise product of two Fourier transforms. == Visual explanation == == Historical developments == One of the earliest uses of the convolution integral appeared in D'Alembert's derivation of Taylor's theorem in Recherches sur différents points importants du système du monde, published in 1754. Also, an expression of the type: ∫ f ( u ) ⋅ g ( x − u ) d u {\displaystyle \int f(u)\cdot g(x-u)\,du} is used by Sylvestre François Lacroix on page 505 of his book entitled Treatise on differences and series, which is the last of 3 volumes of the encyclopedic series: Traité du calcul différentiel et du calcul intégral, Chez Courcier, Paris, 1797–1800. Soon thereafter, convolution operations appear in the works of Pierre Simon Laplace, Jean-Baptiste Joseph Fourier, Siméon Denis Poisson, and others. The term itself did not come into wide use until the 1950s or 1960s. Prior to that it was sometimes known as Faltung (which means folding in German), composition product, superposition integral, and Carson's integral. Yet it appears as early as 1903, though the definition is rather unfamiliar in older uses. The operation: ∫ 0 t φ ( s ) ψ ( t − s ) d s , 0 ≤ t < ∞ , {\displaystyle \int _{0}^{t}\varphi (s)\psi (t-s)\,ds,\quad 0\leq t<\infty ,} is a particular case of composition products considered by the Italian mathematician Vito Volterra in 1913. == Circular c

    Read more →
  • Color image pipeline

    Color image pipeline

    An image pipeline or video pipeline is the set of components commonly used between an image source (such as a camera, a scanner, or the rendering engine in a computer game), and an image renderer (such as a television set, a computer screen, a computer printer or cinema screen), or for performing any intermediate digital image processing consisting of two or more separate processing blocks. An image/video pipeline may be implemented as computer software, in a digital signal processor, on an FPGA, or as fixed-function ASIC. In addition, analog circuits can be used to do many of the same functions. Typical components include image sensor corrections (including debayering or applying a Bayer filter), noise reduction, image scaling, gamma correction, image enhancement, colorspace conversion (between formats such as RGB, YUV or YCbCr), chroma subsampling, framerate conversion, image compression/video compression (such as JPEG), and computer data storage/data transmission. Typical goals of an imaging pipeline may be perceptually pleasing end-results, colorimetric precision, a high degree of flexibility, low cost/low CPU utilization/long battery life, or reduction in bandwidth/file size. Some functions may be algorithmically linear. Mathematically, those elements can be connected in any order without changing the end-result. As digital computers use a finite approximation to numerical computing, this is in practice not true. Other elements may be non-linear or time-variant. For both cases, there is often one or a few sequences of components that makes sense for optimum precision and minimum hardware-cost/CPU-load.

    Read more →
  • Spike-and-slab regression

    Spike-and-slab regression

    Spike-and-slab regression is a type of Bayesian linear regression in which a particular hierarchical prior distribution for the regression coefficients is chosen such that only a subset of the possible regressors is retained. The technique is particularly useful when the number of possible predictors is larger than the number of observations. The idea of the spike-and-slab model was originally proposed by Mitchell & Beauchamp (1988). The approach was further significantly developed by Madigan & Raftery (1994) and George & McCulloch (1997). A recent and important contribution to this literature is Ishwaran & Rao (2005). == Model description == Suppose we have P possible predictors in some model. Vector γ has a length equal to P and consists of zeros and ones. This vector indicates whether a particular variable is included in the regression or not. If no specific prior information on initial inclusion probabilities of particular variables is available, a Bernoulli prior distribution is a common default choice. Conditional on a predictor being in the regression, we identify a prior distribution for the model coefficient, which corresponds to that variable (β). A common choice on that step is to use a normal prior with a mean equal to zero and a large variance calculated based on ( X T X ) − 1 {\displaystyle (X^{T}X)^{-1}} (where X {\displaystyle X} is a design matrix of explanatory variables of the model). A draw of γ from its prior distribution is a list of the variables included in the regression. Conditional on this set of selected variables, we take a draw from the prior distribution of the regression coefficients (if γi = 1 then βi ≠ 0 and if γi = 0 then βi = 0). βγ denotes the subset of β for which γi = 1. In the next step, we calculate a posterior probability for both inclusion and coefficients by applying a standard statistical procedure. All steps of the described algorithm are repeated thousands of times using the Markov chain Monte Carlo (MCMC) technique. As a result, we obtain a posterior distribution of γ (variable inclusion in the model), β (regression coefficient values) and the corresponding prediction of y. The model got its name (spike-and-slab) due to the shape of the two prior distributions. The "spike" is the probability of a particular coefficient in the model to be zero. The "slab" is the prior distribution for the regression coefficient values. An advantage of Bayesian variable selection techniques is that they are able to make use of prior knowledge about the model. In the absence of such knowledge, some reasonable default values can be used; to quote Scott and Varian (2013): "For the analyst who prefers simplicity at the cost of some reasonable assumptions, useful prior information can be reduced to an expected model size, an expected R2, and a sample size ν determining the weight given to the guess at R2." Some researchers suggest the following default values: R2 = 0.5, ν = 0.01, and π = 0.5 (parameter of a prior Bernoulli distribution).

    Read more →
  • Flat-field correction

    Flat-field correction

    Flat-field correction (FFC) is a digital imaging technique to mitigate pixel-to-pixel differences in the photodetector sensitivity and distortions in the optical path. It is a standard calibration procedure in everything from personal digital cameras to large telescopes. == Overview == Flat fielding refers to the process of compensating for different gains and dark currents in a detector. Once a detector has been appropriately flat-fielded, a uniform signal will create a uniform output (hence flat-field). This then means any further signal is due to the phenomenon being detected and not a systematic error. A flat-field image is acquired by imaging a uniformly-illuminated screen, thus producing an image of uniform color and brightness across the frame. For handheld cameras, the screen could be a piece of paper at arm's length, but a telescope will frequently image a clear patch of sky at twilight, when the illumination is uniform and there are few, if any, stars visible. Once the images are acquired, processing can begin. A flat-field consists of two numbers for each pixel, the pixel's gain and its dark current (or dark frame). The pixel's gain is how the amount of signal given by the detector varies as a function of the amount of light (or equivalent). The gain is almost always a linear variable, as such the gain is given simply as the ratio of the input and output signals. The dark-current is the amount of signal given out by the detector when there is no incident light (hence dark frame). In many detectors this can also be a function of time, for example in astronomical telescopes it is common to take a dark-frame of the same time as the planned light exposure. The gain and dark-frame for optical systems can also be established by using a series of neutral density filters to give input/output signal information and applying a least squares fit to obtain the values for the dark current and gain. C = ( R − D ) × m ( F − D ) = ( R − D ) × G {\displaystyle C={\frac {(R-D)\times m}{(F-D)}}=(R-D)\times G} where: C = corrected image R = raw image F = flat field image D = dark frame image m = image-averaged value of (F−D) G = Gain = m ( F − D ) {\displaystyle m \over (F-D)} In this equation, capital letters are 2D matrices, and lowercase letters are scalars. All matrix operations are performed element-by-element. In order for an astrophotographer to capture a light frame, they must place a light source over the imaging instrument's objective lens such that the light source emanates evenly through the users optics. The photographer must then adjust the exposure of their imaging device (charge-coupled device (CCD) or digital single-lens reflex camera (DSLR) ) so that when the histogram of the image is viewed, a peak reaching about 40–70% of the dynamic range (maximum range of pixel values) of the imaging device is seen. The photographer typically takes 15–20 light frames and performs median stacking. Once the desired light frames are acquired, the objective lens is covered so that no light is allowed in, then 15–20 dark frames are taken, each of equal exposure time as a light frame. These are called Dark-Flat frames. == In X-ray imaging == In X-ray imaging, the acquired projection images generally suffer from fixed-pattern noise, which is one of the limiting factors of image quality. It may stem from beam inhomogeneity, gain variations of the detector response due to inhomogeneities in the photon conversion yield, losses in charge transport, charge trapping, or variations in the performance of the readout. Also, the scintillator screen may accumulate dust and/or scratches on its surface, resulting in systematic patterns in every acquired X-ray projection image. In X-ray computed tomography (CT), fixed-pattern noise is known to significantly degrade the achievable spatial resolution and generally leads to ring or band artifacts in the reconstructed images. Fixed pattern noise can be easily removed using flat field correction. In conventional flat field correction, projection images without sample are acquired with and without the X-ray beam turned on, which are referred to as flat fields (F) and dark fields (D). Based on the acquired flat and dark fields, the measured projection images (P) with sample are then normalized to new images (N) according to: N = ( P − D ) ( F − D ) {\displaystyle N={\frac {(P-D)}{(F-D)}}} == Dynamic flat field correction == While conventional flat field correction is an elegant and easy procedure that largely reduces fixed-pattern noise, it heavily relies on the stationarity of the X-ray beam, scintillator response and CCD sensitivity. In practice, however, this assumption is only approximately met. Indeed, detector elements are characterized by intensity dependent, nonlinear response functions and the incident beam often shows time dependent non-uniformities, which render conventional FFC inadequate. In synchrotron X-ray tomography, many factors may cause flat field variations: instability of the bending magnets of the synchrotron, temperature variations due to the water cooling in mirrors and the monochromator, or vibrations of the scintillator and other beamline components. The latter is responsible for the biggest variations in the flat fields. To deal with such variations, a dynamic flat field correction procedure can be employed that estimates a flat field for each individual projection. Through principal component analysis of a set of flat fields, which are acquired prior and/or posterior to the actual scan, eigen flat fields can be computed. A linear combination of the most important eigen flat fields can then be used to individually normalize each X-ray projection: N j = P j − D ¯ F ¯ + ∑ k w j k u k − D ¯ {\displaystyle N_{j}={\frac {P_{j}-{\bar {D}}}{{\bar {F}}+\sum _{k}w_{jk}u_{k}-{\bar {D}}}}} where N j {\displaystyle N_{j}} = intensity normalized X-ray projection P j {\displaystyle P_{j}} = raw X-ray projection F ¯ {\displaystyle {\bar {F}}} = mean flat field image (average of flat fields) u k {\displaystyle u_{k}} = k-th eigen flat field w j k {\displaystyle w_{jk}} = weight of the eigen flat field u k {\displaystyle u_{k}} D ¯ {\displaystyle {\bar {D}}} = mean dark field (average of dark fields)

    Read more →
  • Gemini Enterprise Agent Platform

    Gemini Enterprise Agent Platform

    Gemini Enterprise Agent Platform (formerly known as Vertex AI) is a managed machine learning (ML) and artificial intelligence (AI) platform developed by Google Cloud. It provides a unified environment for building, training, deploying, and scaling ML models and generative AI applications. The platform integrates tools for the full ML lifecycle, including data preparation, model training, evaluation, deployment, and monitoring, under a single API and user interface. Vertex AI was announced at Google I/O and released as a generally available product on May 18, 2021. At launch, Google described Vertex AI as unifying its AutoML offerings with its prior Cloud AI Platform capabilities, and as adding operational features intended to help teams move models from experimentation into production use. On April 22, 2026, Google announced Gemini Enterprise Agent Platform as the replacement evolution of Vertex AI. == History == Google Cloud announced the general availability of Vertex AI on May 18, 2021, at the Google I/O developer conference. The platform was designed to consolidate Google Cloud's previously separate ML offerings, including AutoML and the legacy AI Platform, into a single system. At launch, Google claimed that Vertex AI required roughly 80% fewer lines of code to train a model compared to competing platforms. In June 2023, Google made generative AI support in Vertex AI generally available, giving developers access to foundation models including PaLM 2, Imagen, and Codey through the platform's Model Garden and the newly launched Generative AI Studio. At the time of this launch, Model Garden included over 60 models from Google and its partners. In August 2023, at the Google Cloud Next conference, Google announced further updates to Vertex AI, including the addition of third-party models such as Claude 2 from Anthropic and Llama 2 from Meta to the Model Garden, as well as new tools called Vertex AI Extensions for connecting models to APIs for real-time data retrieval. At the same event, Vertex AI Search and Conversation were made generally available, providing enterprise search and chatbot capabilities powered by foundation models. In April 2024, at Google Cloud Next, the company introduced Vertex AI Agent Builder, a no-code tool for creating AI-powered conversational agents built on top of Gemini large language models. This brought together the existing Vertex AI Search and Conversation products with new developer tools for building generative AI experiences. == Features == === Model training === Vertex AI supports both AutoML, which enables code-free model training on tabular, image, text, or video data, and custom training, which gives users full control over the ML framework, training code, and hyperparameter tuning. The platform provides serverless training as well as dedicated training clusters with GPU and TPU accelerators. Vertex AI Vizier handles automatic hyperparameter tuning, and Vertex AI Experiments allows comparison and tracking of training runs. === Model Garden === The Vertex AI Model Garden is a curated catalog of over 200 enterprise-ready models, including Google's own foundation models (such as Gemini, Imagen, and Veo), third-party models (such as Anthropic's Claude and Mistral AI models), and popular open-source models (such as Llama and Gemma). Models are accessible as fully managed model-as-a-service APIs. === Pipelines (workflow orchestration) === Vertex AI Pipelines provides managed orchestration of ML workflows and supports pipelines built with the Kubeflow Pipelines SDK, among other options described in Google Cloud documentation. === Vertex AI Studio === Vertex AI Studio provides tools for prompt design, testing, and model management, allowing developers to prototype and build generative AI applications using natural language, code, images, or video. === Agent Builder and Agent Engine === Vertex AI Agent Builder is a suite of products for building, deploying, and governing AI agents in production environments. It supports development with the open-source Agent Development Kit (ADK) and other frameworks. Vertex AI Agent Engine provides the underlying infrastructure for deploying and scaling agents, with support for enterprise security features including HIPAA compliance, customer-managed encryption keys (CMEK), and VPC Service Controls. === Generative AI tooling and model access === Google markets Vertex AI as providing access to Google foundation models (including the Gemini family) and developer tools such as Vertex AI Studio, along with a model catalog that includes Google and selected open source models (marketed as "Model Garden"). Google has also offered products within Vertex AI aimed at building generative search and conversational applications, including offerings named "Vertex AI Search" and "Vertex AI Conversation" as reported in 2023 coverage of platform updates. === MLOps tools === The platform includes a range of MLOps capabilities: Vertex AI Pipelines for orchestrating and automating ML workflows as reusable pipelines. Vertex AI Feature Store for serving, sharing, and reusing ML features across projects. Vertex AI Model Registry for storing, versioning, and managing trained models. Vertex AI Model Monitoring for detecting training-serving skew and inference drift in deployed models. Vertex Explainable AI for interpreting model predictions. Vertex AI Workbench for managed JupyterLab notebook environments integrated with Google Cloud Storage and BigQuery. == Industry recognition == Google was named a Leader for the fifth consecutive year in the 2024 Gartner Magic Quadrant for Cloud AI Developer Services, a recognition that encompasses Vertex AI and its related offerings. Google was also recognized as a Leader in the 2024 Gartner Magic Quadrant for Data Science and Machine Learning Platforms and was named a Leader in the Forrester Wave for AI/ML Platforms, Q3 2024. In October 2025, Google was also named a Leader in the 2025 IDC (International Data Corporation) MarketScape for Worldwide GenAI Life-Cycle Foundation Model Software. == Pricing == Vertex AI uses a pay-as-you-go pricing model, with costs determined by the specific services consumed, including model training, prediction serving, and data storage. For generative AI tasks, pricing is based on a per-token model, with rates varying depending on the specific model used and whether tokens are input or output. Google offers a free tier for new users, which includes limited custom training hours and online prediction usage, along with an introductory US$300 in Google Cloud credits valid for 90 days. == Adoption == In the year following its 2021 launch, Google reported that usage of Vertex AI and BigQuery had driven 2.5 times more machine learning predictions compared to the prior year, and that active customers of Vertex AI Workbench had grown 25-fold over a six-month period. Early enterprise adopters included Ford, Wayfair, and Seagate, among others. Wayfair reported that it was able to run large model training jobs 5 to 10 times faster using the platform.

    Read more →
  • Digital omnivore

    Digital omnivore

    A digital omnivore is a person who uses multiple modalities (devices) to access the Internet and other media content in their daily life. As people increasingly own mobile devices, cross-platform multimedia consumption has continued to shape the digital landscape, both in terms of the type of media content they consume and how they consume it. As of 2021, at least half of all global digital traffic is generated by mobile devices. == Connected devices and digital consumption == A 2015 study of digital media consumption showed that smartphones were primarily used for communication, and tablets were primarily used for entertainment – additionally, both were frequently used in conjuncture with other devices, like televisions. An earlier 2011 analysis of the way consumers in the U.S. viewed news content on their devices throughout the day demonstrated how people use different mobile devices for different functions. On a typical weekend morning, digital omnivores accessed their news using their tablet, favored their computer during the working day, and returned to tablet use in the evening, peaking between the hours of 9pm and midnight. Mobile phones were used for web-browsing throughout the day when users were away from their personal computer. Increased Wi-Fi availability and mobile broadband adoption have changed the way people are going online. In August 2011, more than a third (37.2%) of U.S. digital traffic coming from mobile phones occurred via a Wi-Fi connection while tablets, which traditionally required a Wi-Fi connection to access the Internet, are increasingly driving traffic using mobile broadband access. As of 2021, LTE, 5G, and other forms of mobile broadband access are available on the majority of mobile devices. Tablets contributed nearly 2% of all web browsing traffic in the United States in 2011. During this period, iPads also began to account for a higher share of Internet traffic than iPhones (46.8% vs. 42.6% of all iOS device traffic. == Implications for marketing, advertisers and publishers == As of 2021, the average amount of time spent daily consuming digital media was eight hours, an increase from 2020 and a further increase from 2019, partially as a result of the COVID-19 pandemic. Social media platforms such as Instagram, Facebook, Twitter, and TikTok, as well as other online platforms like YouTube, incorporate advertisements into the in-app or online experience, with some offering the ability to shop for and sell items through the app or website.

    Read more →
  • Orleans (software framework)

    Orleans (software framework)

    Orleans is a cross-platform software framework for building scalable and robust distributed interactive applications based on the .NET Framework or on the more recent .NET. == Overview == Orleans was originally created by the eXtreme Computing Group at Microsoft Research and introduced the virtual actor model as a new approach to building distributed systems for the cloud. Orleans scales from a single on-premises server to highly-available and globally distributed applications in the cloud. The virtual actor model is based on the actor model but has several differences: A virtual actor always exists, it cannot be explicitly created or destroyed. Virtual actors are automatically instantiated. If a server hosting an actor crashes, the next message sent to the actor causes it to be reinstantiated automatically. The server that an actor is on is transparent to the application code. Orleans can automatically create multiple instances of the same stateless actor. Starting with cloud services for the Halo franchise, the framework has been used by a number of cloud services at Microsoft and other companies since 2011. The core Orleans technology was transferred to 343 Industries and is available as open source since January 2015. The source code is licensed under MIT License and hosted on GitHub. Orleans runs on Microsoft Windows, Linux, and macOS and is compatible with .NET Standard 2.0 and above. == Features == Some Orleans features include: Persistence Distributed ACID transactions Streams Timers & Reminders Fault tolerance == Related implementations == The Electronic Arts BioWare division created Project Orbit. It is a Java implementation of virtual actors that was heavily inspired by the Orleans project.

    Read more →
  • Color layout descriptor

    Color layout descriptor

    In digital image and video processing, a color layout descriptor (CLD) is designed to capture the spatial distribution of color in an image. The feature extraction process consists of two parts: grid based representative color selection and discrete cosine transform with quantization. Color is the most basic quality of the visual contents, therefore it is possible to use colors to describe and represent an image. The MPEG-7 standard has tested the most efficient procedure to describe the color and has selected those that have provided more satisfactory results. This standard proposes different methods to obtain these descriptors, and one tool defined to describe the color is the CLD, that permits describing the color relation between sequences or group of images. The CLD captures the spatial layout of the representative colors on a grid superimposed on a region or image. Representation is based on coefficients of the discrete cosine transform (DCT). This is a very compact descriptor being highly efficient in fast browsing and search applications. It can be applied to still images as well as to video segments. == Definition == The CLD is a very compact and resolution-invariant representation of color for high-speed image retrieval and it has been designed to efficiently represent the spatial distribution of colors. This feature can be used for a wide variety of similarity-based retrieval, content filtering and visualization. It is especially useful for spatial structure-based retrieval applications. This descriptor is obtained by applying the DCT transformation on a 2-D array of local representative colors in Y or Cb or Cr color space. The functionalities of the CLD are basically the matching: – Image-to-image matching – Video clip-to-video clip matching Remark that the CLD is one of the most precise and fast color descriptor. == Extraction == The extraction process of this color descriptor consists of four stages: Image partitioning Representative color selection DCT transformation Zigzag scanning The standard MPEG-7 recommends using the YCbCr color space for the CLD. === Image partitioning === In the image partitioning stage, the input picture (on RGB color space) is divided into 64 blocks to guarantee the invariance to resolution or scale. The inputs and outputs of this step are summarized in the following table: === Representative color selection === After the image partitioning stage, a single representative color is selected from each block. Any method to select the representative color can be applied, but the standard recommends the use of the average of the pixel colors in a block as the corresponding representative color, since it is simpler and the description accuracy is sufficient in general. The selection results in a tiny image icon of size 8x8. The next figure shows this process. Note that in the image of the figure, the resolution of the original image has been maintained only in order to facilitate its representation. The inputs and outputs of this stage are summarized in the next table: Once the tiny image icon is obtained, the color space conversion between RGB and YCbCr is applied. === DCT transformation === In the fourth stage, the luminance (Y) and the blue and red chrominance (Cb and Cr) are transformed by 8x8 DCT, so three sets of 64 DCT coefficients are obtained. To calculate the DCT in a 2D array, the formulas below are used. B p q = α p α q ∑ m = 0 M − 1 ∑ n = 0 N − 1 A m n cos ⁡ π ( 2 m + 1 ) p 2 M cos ⁡ π ( 2 n + 1 ) q 2 N , 0 ≤ p ≤ M − 1 , 0 ≤ q ≤ N − 1 {\displaystyle B_{pq}=\alpha _{p}\alpha _{q}\sum _{m=0}^{M-1}\sum _{n=0}^{N-1}A_{mn}\cos {\frac {\pi (2m+1)p}{2M}}\cos {\frac {\pi (2n+1)q}{2N}},\qquad 0\leq p\leq M-1,\;0\leq q\leq N-1} α p = { 1 M , p = 0 2 M , 1 ≤ p ≤ M − 1 α q = { 1 N , q = 0 2 N , 1 ≤ q ≤ N − 1 {\displaystyle \alpha _{p}={\begin{cases}{\frac {1}{\sqrt {M}}},&p=0\\{\sqrt {\frac {2}{M}}},&1\leq p\leq M-1\end{cases}}\qquad \alpha _{q}={\begin{cases}{\frac {1}{\sqrt {N}}},&q=0\\{\sqrt {\frac {2}{N}}},&1\leq q\leq N-1\end{cases}}} The inputs and outputs of this stage are summarized in the next table: === Zigzag scanning === A zigzag scanning is performed with these three sets of 64 DCT coefficients, following the schema presented in the figure. The purpose of the zigzag scan is to group the low frequency coefficients of the 8x8 matrix into a vector. The inputs and outputs of this stage are summarized in the next table: Finally, these three set of matrices correspond to the CLD of the input image. == Matching == The matching process helps to evaluate if two elements are equal comparing both elements and calculating the distance between them. In the case of color descriptors the matching process helps to evaluate if two images are similar. Its procedure is the following: – Given an image as an input, the application attempts to find an image with a similar descriptor in a data base of images. If we consider two CLDs: {DY, DCb, DCr} { DY‟, DCb‟, DCr‟ }, The distance between the two descriptors can be computed as: D = ∑ i w y i ( D Y i − D Y i ′ ) 2 + ∑ i w b i ( D C b i − D C b i ′ ) 2 + ∑ i w r i ( D C r i − D C r i ′ ) 2 {\displaystyle D={\sqrt {\sum _{i}w_{yi}(DY_{i}-DY_{i}')^{2}}}+{\sqrt {\sum _{i}w_{bi}(DCb_{i}-DCb_{i}')^{2}}}+{\sqrt {\sum _{i}w_{ri}(DCr_{i}-DCr_{i}')^{2}}}} The subscript i represents the zigzag-scanning order of the coefficients. Furthermore, notice that is possible to weight the coefficients (w) in order to adjust the performance of the matching process. These weights let us give to some components of the descriptor more importance than others. Observing the formula, it can be extracted that: – 2 images are the same if the distance is 0 – 2 images are similar if the distance is near to 0 Therefore, this matching process will let to identify images with similar color descriptors. Since the complexity of the similarity matching process shown above is low, high-speed image matching can be achieved.

    Read more →
  • Color layout descriptor

    Color layout descriptor

    In digital image and video processing, a color layout descriptor (CLD) is designed to capture the spatial distribution of color in an image. The feature extraction process consists of two parts: grid based representative color selection and discrete cosine transform with quantization. Color is the most basic quality of the visual contents, therefore it is possible to use colors to describe and represent an image. The MPEG-7 standard has tested the most efficient procedure to describe the color and has selected those that have provided more satisfactory results. This standard proposes different methods to obtain these descriptors, and one tool defined to describe the color is the CLD, that permits describing the color relation between sequences or group of images. The CLD captures the spatial layout of the representative colors on a grid superimposed on a region or image. Representation is based on coefficients of the discrete cosine transform (DCT). This is a very compact descriptor being highly efficient in fast browsing and search applications. It can be applied to still images as well as to video segments. == Definition == The CLD is a very compact and resolution-invariant representation of color for high-speed image retrieval and it has been designed to efficiently represent the spatial distribution of colors. This feature can be used for a wide variety of similarity-based retrieval, content filtering and visualization. It is especially useful for spatial structure-based retrieval applications. This descriptor is obtained by applying the DCT transformation on a 2-D array of local representative colors in Y or Cb or Cr color space. The functionalities of the CLD are basically the matching: – Image-to-image matching – Video clip-to-video clip matching Remark that the CLD is one of the most precise and fast color descriptor. == Extraction == The extraction process of this color descriptor consists of four stages: Image partitioning Representative color selection DCT transformation Zigzag scanning The standard MPEG-7 recommends using the YCbCr color space for the CLD. === Image partitioning === In the image partitioning stage, the input picture (on RGB color space) is divided into 64 blocks to guarantee the invariance to resolution or scale. The inputs and outputs of this step are summarized in the following table: === Representative color selection === After the image partitioning stage, a single representative color is selected from each block. Any method to select the representative color can be applied, but the standard recommends the use of the average of the pixel colors in a block as the corresponding representative color, since it is simpler and the description accuracy is sufficient in general. The selection results in a tiny image icon of size 8x8. The next figure shows this process. Note that in the image of the figure, the resolution of the original image has been maintained only in order to facilitate its representation. The inputs and outputs of this stage are summarized in the next table: Once the tiny image icon is obtained, the color space conversion between RGB and YCbCr is applied. === DCT transformation === In the fourth stage, the luminance (Y) and the blue and red chrominance (Cb and Cr) are transformed by 8x8 DCT, so three sets of 64 DCT coefficients are obtained. To calculate the DCT in a 2D array, the formulas below are used. B p q = α p α q ∑ m = 0 M − 1 ∑ n = 0 N − 1 A m n cos ⁡ π ( 2 m + 1 ) p 2 M cos ⁡ π ( 2 n + 1 ) q 2 N , 0 ≤ p ≤ M − 1 , 0 ≤ q ≤ N − 1 {\displaystyle B_{pq}=\alpha _{p}\alpha _{q}\sum _{m=0}^{M-1}\sum _{n=0}^{N-1}A_{mn}\cos {\frac {\pi (2m+1)p}{2M}}\cos {\frac {\pi (2n+1)q}{2N}},\qquad 0\leq p\leq M-1,\;0\leq q\leq N-1} α p = { 1 M , p = 0 2 M , 1 ≤ p ≤ M − 1 α q = { 1 N , q = 0 2 N , 1 ≤ q ≤ N − 1 {\displaystyle \alpha _{p}={\begin{cases}{\frac {1}{\sqrt {M}}},&p=0\\{\sqrt {\frac {2}{M}}},&1\leq p\leq M-1\end{cases}}\qquad \alpha _{q}={\begin{cases}{\frac {1}{\sqrt {N}}},&q=0\\{\sqrt {\frac {2}{N}}},&1\leq q\leq N-1\end{cases}}} The inputs and outputs of this stage are summarized in the next table: === Zigzag scanning === A zigzag scanning is performed with these three sets of 64 DCT coefficients, following the schema presented in the figure. The purpose of the zigzag scan is to group the low frequency coefficients of the 8x8 matrix into a vector. The inputs and outputs of this stage are summarized in the next table: Finally, these three set of matrices correspond to the CLD of the input image. == Matching == The matching process helps to evaluate if two elements are equal comparing both elements and calculating the distance between them. In the case of color descriptors the matching process helps to evaluate if two images are similar. Its procedure is the following: – Given an image as an input, the application attempts to find an image with a similar descriptor in a data base of images. If we consider two CLDs: {DY, DCb, DCr} { DY‟, DCb‟, DCr‟ }, The distance between the two descriptors can be computed as: D = ∑ i w y i ( D Y i − D Y i ′ ) 2 + ∑ i w b i ( D C b i − D C b i ′ ) 2 + ∑ i w r i ( D C r i − D C r i ′ ) 2 {\displaystyle D={\sqrt {\sum _{i}w_{yi}(DY_{i}-DY_{i}')^{2}}}+{\sqrt {\sum _{i}w_{bi}(DCb_{i}-DCb_{i}')^{2}}}+{\sqrt {\sum _{i}w_{ri}(DCr_{i}-DCr_{i}')^{2}}}} The subscript i represents the zigzag-scanning order of the coefficients. Furthermore, notice that is possible to weight the coefficients (w) in order to adjust the performance of the matching process. These weights let us give to some components of the descriptor more importance than others. Observing the formula, it can be extracted that: – 2 images are the same if the distance is 0 – 2 images are similar if the distance is near to 0 Therefore, this matching process will let to identify images with similar color descriptors. Since the complexity of the similarity matching process shown above is low, high-speed image matching can be achieved.

    Read more →
  • List of Java software and tools

    List of Java software and tools

    This is a list of software and programming tools for the Java programming language, which includes frameworks, libraries, IDEs, build tools, application servers, and related projects. == Java frameworks == == Libraries == Apache Ant – build automation tool Apache Batik – SVG processing Apache Cayenne – object-relational mapping Apache Xerces – collection of software libraries for parsing, validating, serializing and manipulating XML. Applet – applet API Ardor3D – 3D graphics engine Bonita BPM – workflow engine Cassowary – constraint solving Checkstyle – static code analysis GNU Classpath – standard library implementation Colt – scientific computing and technical computing Commons Daemon – manages applications as daemons DESMO-J – discrete event simulation Diagrams.net – diagramming Disruptor – high-performance messaging Dom4j – XML processing Dynamic Languages Toolkit – support for dynamic programming languages on the JVM Echo – GUI Flying Saucer – XHTML/CSS rendering Formatting Objects Processor – XSL-FO to PDF H2 Database Engine – relational database IAIK-JCE – cryptography Internet Foundation Classes – legacy GUI JavaBeans – reusable component architecture for enabling encapsulation, events, and properties for software components JavaCC – open-source parser generator and lexical analyzer Java Class Library – standard library of Java and other JVM languages Java Native Access – provides Java programs easy access to native shared libraries without using the Java Native Interface Javolution – real-time computing Jblas – linear algebra JDBCFacade – simplifies JDBC use JExcel – Excel API JFugue – music programming JMusic – music programming Joget Workflow – workflow engine JOOQ Object Oriented Querying – fluent API for SQL JPOS – financial messaging JUNG – open-source graph modeling and visualization LanguageWare – language processing LibGDX – game development Modular Audio Recognition Framework – collection of voice, sound, speech, text and natural language processing algorithms. ASM – bytecode manipulation Open Inventor – 3D graphics OpenPDF – PDF Parallel Colt – parallel computing Parboiled – parser PlayN – game development QOCA – constraint solving QtJambi – Qt bindings SLF4J – logging StableUpdate – update management SWT – GUI SuanShu – numerical computing SwingLabs – GUI extensions UBY – natural language processing Undecimber – calendar XDoclet – attribute-oriented programming XINS – XML network services XStream – object serialization == Machine learning and AI == Apache Mahout – scalable machine learning library focused on clustering, classification, and collaborative filtering Apache MXNet – deep learning framework with Java API support Apache OpenNLP – machine learning based toolkit for natural language processing of text Deeplearning4j – distributed deep learning library Deep Java Library – open-source deep learning framework developed by Amazon Web Services Encog – framework for neural networks, genetic algorithms, Hidden Markov model, and Bayesian networks. LIBSVM – Support Vector Machine implementation Mallet – machine learning toolkit for classification, clustering, and topic modeling. MLlib – distributed machine-learning framework on top of Apache Spark Core Neuroph – lightweight neural network framework Weka – collection of machine learning algorithms for data mining Yooreeka – machine learning == Data mining == Java Data Mining (JDM) – standard Java API for data mining Massive Online Analysis (MOA) – data stream mining with concept drift == Math and scientific libraries == Apache Commons Math – general-purpose mathematics library including statistics, linear algebra, and optimization. Colt – high-performance scientific computing, including linear algebra and random numbers. Efficient Java Matrix Library (EJML) – dense and sparse matrix computations and linear algebra Easy Java Simulations – Open Source Physics project designed to create discrete computer simulations Exp4j – evaluates mathematical expressions at runtime GroovyLab – numerical computational environment Hipparchus – fork of Apache Commons Math with updated algorithms for statistics, linear algebra, and optimization. JAMA – numerical linear algebra library Jblas: Linear Algebra for Java (Jblas) – linear algebra library using native BLAS/LAPACK bindings Java Astrodynamics Toolkit – numerical library of software components for use in spaceflight applications for Java or MATLAB Matrix Toolkit Java (MTJ) – linear algebra library with BLAS and LAPACK support OjAlgo – optimization, linear algebra, and financial calculations. OptimJ – extension for mathematical optimization and constraint programming Parallel Colt – A parallel extension of Colt SuanShu – numerical analysis, linear algebra, statistics, and optimization. == Integrated development environments == See also: Java IDEs on Wikibooks Android Studio – IDE for Google's Android operating system BlueJ – educational IDE for teaching Java DrJava – lightweight Java IDE for beginners Eclipse IDE – open-source IDE with extensive plugin ecosystem Greenfoot – educational IDE IntelliJ IDEA – commercial and community editions from JetBrains JDeveloper – freeware IDE supplied by Oracle Corporation jGRASP – software visualizations MyEclipse – Java EE IDE NetBeans IDE – Apache NetBeans Visual Studio Code – general-purpose editor with Java extensions === Online IDEs === Eclipse Che GitHub Codespaces JDoodle Replit == Text editors with Java support == == Build tools and package managers == Apache Ant – automating software build Apache Ivy – subproject of Apache Ant Apache Maven – build automation and dependency management Boot – build automation for Clojure CMake – build tool with limited support for java Gradle – modern build automation tool Go continuous delivery (GoCD) – continuous delivery and build automation server Jenkins – automation server continuous delivery JitPack – package repository for Git projects Leiningen – build automation for Clojure Simple build tool (sbt) – open-source build tool Spring Roo – rapid application development of Java-based enterprise software WaveMaker – low-code development platform == Java runtimes, compilers and virtual machines == Android Runtime – runtime environment javac – Java programming language compiler Java Virtual Machine (JVM) – virtual machine that executes Java bytecode JD Decompiler JEB decompiler – disassembler and decompiler software for Android applications GraalVM – Just-in-time compilation HotSpot – JVM implementation included in OpenJDK == JVM languages and dialects == Clojure – Lisp dialect Groovy JRuby – Ruby implementation Jython – Python implementation Kotlin – popular for Android app development Renjin – R implementation Scala == Application servers and containers == Apache Geronimo – open source application server Apache MINA – event-driven asynchronous network application framework Apache Tomcat – web container and web server Apache TomEE – Apache Tomcat with Java EE features Borland Enterprise Server – discontinued application server by Borland ColdFusion – commercial application server by Adobe Systems GlassFish – application server for Jakarta EE IBM WebSphere Application Server – enterprise application server by IBM IBM WebSphere Application Server Community Edition – open source edition of WebSphere (discontinued) JBoss Enterprise Application Platform – Red Hat's supported distribution of JBoss/WildFly JEUS – commercial Java EE application server from TmaxSoft Jetty – HTTP server and web container Lucee (formerly Railo) – open source CFML application server Netty – non-blocking I/O client–server framework for network applications Oracle Containers for J2EE – discontinued application server by Oracle Oracle WebLogic Server – enterprise application server by Oracle Orion Application Server – early commercial Java EE server by IronFlare Payara Server – fork of GlassFish for production use Resin – Java application server by Caucho (open source and professional editions) SAP NetWeaver Application Server – enterprise application server by SAP WildFly – application server == Debugging and profiling tools == jdb – Java debugger bundled with the JDK JConsole – JMX-compliant monitoring tool JDK Flight Recorder – method profiling, allocation profiling, and garbage collection related events. JProfiler – commercial Java profiler VisualVM – visual tool integrating commandline JDK tools for profiling and monitoring == Testing and quality assurance == Apache JMeter – load testing tool JaCoCo – Java code coverage library JArchitect – analyzes code quality, architecture, and dependencies. Jtest – software testing and static analysis JUnit – unit testing framework Mockito – open-source testing framework for Java PMD – static program analysis source code analyzer Selenium – browser automation for web app testing Spock – test framework SpotBugs (formerly FindBugs) – static analysis tool TestNG – testing framework inspired by JUnit and NUnit == Other == Apache XMLBeans –

    Read more →
  • Alipay

    Alipay

    Alipay (simplified Chinese: 支付宝; traditional Chinese: 支付寶; pinyin: zhīfùbǎo) is a third-party mobile and online payment platform, established in Hangzhou, China, in February 2004 by Alibaba Group and its founder Jack Ma. In 2015, Alipay moved its headquarters to Pudong, Shanghai, although its parent company Ant Financial remains Hangzhou-based. Alipay overtook PayPal as the world's largest mobile (digital) payment platform in 2013. As of June 2020, Alipay serves over 1.3 billion users and 80 million merchants. According to the statistics of the fourth quarter of 2018, Alipay has a 55.32% share of the third-party payment market in mainland China, and it continues to grow. Along with WeChat, Alipay has been described to be China's super-app with a wide range of functionalities including ridesharing, travel booking and medical appointments. == History == The service was first launched in 2003, by Taobao. The People's Bank of China, China's central bank, issued licensing regulations in June 2010 for third-party payment providers. It also issued separate guidelines for foreign-funded payment institutions. Because of this, Alipay, which accounted for half of China's non-bank online payment market, was restructured as a domestic company controlled by Alibaba CEO Jack Ma in order to facilitate the regulatory approval for the license. The 2010 transfer of Alipay's ownership was controversial, with media reports in 2011 that Yahoo! and Softbank (Alibaba Group's controlling shareholders) were not informed of the sale for nominal value. Chinese business publication Century Weekly criticised Ma, who stated that Alibaba Group's board of directors was aware of the transaction. The incident was criticised in foreign and Chinese media as harming foreign trust in making Chinese investments. The ownership dispute was resolved by Alibaba Group, Yahoo!, and Softbank in July 2011. In 2013, Alipay launched a financial product platform called Yu'e Bao. Alipay partnered with Tianhong Asset Management to launch the it. Yu'e Bao offers an online money market account in which Alipay customers can deposit money and receive a higher interest rate than that available from banks. It soon became China's largest online money market fund and prompted competitors like Baidu and Tencent to introduce alternatives. Alibaba (the parent company of Alipay) reported having 152 million Yu'e Bao users in mid-2016, with 810 billion RMB (US$117 billion) in funds under management. In 2015, Alipay's parent company was re-branded as Ant Financial Services Group. In 2017, Alipay unveiled their facial recognition payment service. In 2020, Alipay upgraded from a payment financial instrument to an open platform for digital life. In 2021, the mandate by the Ministry of Industry and Information Technology (MIIT) to open up the "walled garden" ecosystems of the major tech companies has led to the introduction of interoperability of payment QR codes of Alipay and competing WeChat Pay and UnionPay's Cloud QuickPass platforms. In response to the increase in Alipay's payment volume due to use on Alibaba's e-commerce sites and others, Chinese regulators introduced new rules in 2020. The new rules focused on Alipay because the payment volume exploded due to its use on Alibaba's e-commerce sites and other platforms. By the second quarter in 2020, Alipay held 55.6% of China's third party mobile payment market. The People's Bank of China made rules that required payment firms to place money with regulators and anti-monopoly reviews would be triggered if the amount exceeded 50% market share. The rules included that the People's Bank of China mandate an online-payment clearing route through the NetsUnion Clearing Corporation, a centralized, state-overseen clearing body, and that unused consumer funds be held by a third-party payment provider in a non-interest-bearing account. These measures increased transparency and reduced systemic risk. When Alipay operates outside of China, it must comply with local financial regulations, which may treat specific functions such as money-market funds or investment-linked products. In Singapore, such services may require prior authorization from securities or financial-services regulators before they can be offered to residents. == Services == Alipay states that it operates with more than 65 financial institutions including Visa and MasterCard to provide payment services for Taobao and Tmall as well as more than 460,000 online and local Chinese businesses. Alipay is used in smartphones with their Alipay Wallet app. QR code payment codes are used for local in-store payments. The Alipay app also provides features such as credit card bill payments, bank account managements, P2P transfer, prepay mobile phone top-up, bus and train ticket purchases, food orders, vehicles for hire, insurance selections and a digital identification document storage. Alipay also allows online check-out on most Chinese-based websites such as Taobao and Tmall. The Alipay app allows users to add their own services provided from different companies to create a more personalised experience. Since late 2008, Alipay has promoted public service payment services and has covered more than 300 cities nationwide, supporting more than 1,200 partner organizations. In addition to utility bills such as water and electricity, Alipay also extends their services to areas such as paying transportation fines, property fees, and cable television fees. Common online payment services also include hydropower coal payment, tuition payment and traffic fine. On 15 January 2009, Alipay launched a credit card repayment service, supporting 39 domestic bank-issued credit cards. It is currently the most popular third-party repayment platform. The main advantages are free credit card bills checking, repayments with no administrative fee, as well as automatic repayment, repayment reminders and other value-added services. In the first quarter of 2014, 76% of credit cards were also paid by Alipay Wallet. From December 2013, several chain convenience store companies, including Meiyijia, Hongqi Chain, and Qishiduo C-STORE and 7-Eleven, have successively supported Alipay payment; in December, Beijing taxi drivers began to accept Alipay to pay the fare. Subsequently, Wanda Cinema, Joy City, Wangfujing and other large-scale retail companies as well as movie theaters, KTV, and catering companies have access to Alipay. From 26 March 2019, the service fee will be charged for the payment of credit card through Alipay. Customers only pay the portion of the payment that exceeds 2,000 yuan at 0.1%. In addition to this, in 2019, Walgreens accepted Alipay as payment in 3,000 US stores. Walgreen's products are available to Chinese customers through Alibaba's Tmall online marketplace. The payment application can also be used on Alibaba.com's site and Taobao as a means of payment. A Nielsen report suggests that over 90% of Chinese tourists would be willing to use mobile payment overseas if given the option. Many Chinese tourists do not have international credit cards, and so Alipay is a payment option. Digital payments have become the norm in China as the government pushes a cashless system even in rural and village areas. In November 2019, Alipay introduced Tourpass, a service component that allows non-Chinese users to use its mobile payment feature by pre-loading Chinese Yuan equivalent foreign currency into the app. In 2020, Alipay used a QR code system to help in containing the COVID-19 outbreak. The health code system tags users one of three colors according to their location, basic health information and travel history. "Beauty filters" were included to Alipay's face-scan payment system in a new upgrade that was released in July 2019. The market has responded well to the "beauty filters," which make users seem better when they use the program to make payments. Alipay Tap is a payment function launched by Alipay in July 2024. Alipay+ NFC enables wallets to offer tap-to-pay acceptance across Mastercard's global contactless network, all within your existing wallet infrastructure. == Foreign expansion == Outside of China, more than 300 worldwide merchants use Alipay to sell directly to consumers in China. It currently supports transactions in 18 foreign currencies. Since the launch of Alipay in the Mainland China, Ant Financial introduced a series of expansion of the services to other countries. Other than expanding into individual countries, the system would also be integrated with online payment platform providers. Ant Group had acquired a majority stake into 2C2P, a Singapore-based provider used by merchants worldwide in April 2022, and would eventually integrate Alipay with 2C2P. === Asia === ==== Bangladesh ==== In 2018, Alipay bought 20% shares in Bangladeshi mobile financial service provider bKash Limited. ==== Hong Kong ==== In 2017, Ant Financial expanded to Hong Kong. In a joint venture with CK Hutchison, as Alipay Payment Ser

    Read more →
  • List of performance analysis tools

    List of performance analysis tools

    This is a list of performance analysis tools for use in software development. == General purpose, language independent == The following tools work based on log files that can be generated from various systems. time (Unix) - can be used to determine the run time of a program, separately counting user time vs. system time, and CPU time vs. clock time. timem (Unix) - can be used to determine the wall-clock time, CPU time, and CPU utilization similar to time (Unix) but supports numerous extensions. Supports reporting peak resident set size, major and minor page faults, priority and voluntary context switches via getrusage. Supports sampling procfs on supporting systems to report metrics such as page-based resident set size, virtual memory size, read-bytes, and write-bytes, etc. Supports collecting hardware counters when built with PAPI support. == Multiple languages == The following tools work for multiple languages or binaries. == C and C++ == Arm MAP, a performance profiler supporting Linux platforms. AppDynamics, an application performance management service for C/C++ applications via SDK. AQtime Pro, a performance profiler and memory allocation debugger that can be integrated into Microsoft Visual Studio, and Embarcadero RAD Studio, or can run as a stand-alone application. IBM Rational Purify was a memory debugger allowing performance analysis. Instruments (bundled with Xcode) is used to profile an executable's memory allocations, time usage, filesystem activity, GPU activity etc. Intel Parallel Studio contains Intel VTune Amplifier, which tunes both serial and parallel programs. It also includes Intel Advisor and Intel Inspector. Intel Advisor optimizes vectorization (use of SIMD instructions) and prototypes threading implementations. Intel Inspector detects and debugs races, deadlocks and memory errors. Parasoft Insure++ provides a graphical tool that displays and animates memory allocations in real time to expose memory blowout, fragmentation, overuse, bottlenecks and leaks. Visual Studio Team System Profiler, commercial profiler by Microsoft. == Java == inspectIT is an open-source application performance management (APM) service for monitoring and analyzing software applications, available under the Apache License, Version 2.0 (ALv2). JConsole is the profiler which comes with the Java Development Kit JProfiler JRockit Mission Control, a profiler with low overhead. Netbeans Profiler, a profiler integrated into the NetBeans IDE (internally uses jvisualvm profiler) Plumbr, Java application performance monitoring with automated root cause detection. Links memory leaks, GC inefficiency, slow database and external web service calls, locked threads, and other performance problems to the line in source code that causes them. OverOps, Continuous reliability for the modern software supply chain, automatically detect and deliver root cause automation for all errors. VisualVM is a visual tool integrating several commandline JDK tools and lightweight profiling capabilities. It is bundled with the Java Development Kit since version 6, update 7. == JavaScript == The Firefox web browser's developer tools contain a Performance tool, which gives insight into JavaScript performance of a website. Microsoft Visual Studio AJAX Profiling Extensions is a free profiling tool for JavaScript by Microsoft Research. == .NET == CLR Profiler is a free memory profiler provided by Microsoft for CLR applications. GlowCode is a performance and memory profiler for .NET applications using C# and other .NET languages. It identifies time-intensive functions and detects memory leaks and errors in native, managed and mixed Windows x64 and x86 applications. Visual Studio == PHP == BlackFire.io Dbg Xdebug is a PHP extension which provides debugging and profiling capabilities.

    Read more →
  • OpenIO

    OpenIO

    OpenIO offered object storage for a wide range of high-performance applications. OpenIO was founded in 2015 by Laurent Denel (CEO), Jean-François Smigielski (CTO) and five other co-founders; it leveraged open source software, developed since 2006, based on a grid technology that enabled dynamic behaviour and supported heterogenous hardware. In October 2017 OpenIO was completed a $5 million funding rounds. In July 2020 OpenIO had been acquired by OVH and withdrawn from the market to become the core technology of OVHcloud object storage offering. == Software == OpenIO is a software-defined object store that supports S3 and can be deployed on-premises, cloud-hosted or at the edge, on any hardware mix. It has been designed from the beginning for performance and cost-efficiency at any scale, and it has been optimized for Big Data, HPC and AI. OpenIO stores objects within a flat structure within a massively distributed directory with indirections, which allows the data query path to be independent of the number of nodes and the performance not to be affected by the growth of capacity. Servers are organized as a grid of nodes massively distributed, where each node takes part in directory and storage services, which ensures that there is no single point of failure and that new nodes are automatically discovered and immediately available without the need to rebalance data. The software is built on top of a technology that ensures optimal data placement based on real-time metrics and allows the addition or removal of storage devices with automatic performance and load impact optimization. For data protection OpenIO has synchronous and asynchronous replication with multiple copies, and an erasure coding implementation based on Reed-Solomon that can be deployed in one data center or geo-distributed or stretched clusters. The software has a feature that catches all events that occur in the cluster and can pass them up in the stack or to applications running on OpenIO nodes. This enables event-driven computing directly into the storage infrastructure. The open source code is available on Github and it is licensed under AGPL3 for server code and LGPL3 for client code. == Performance == OpenIO claimed in 2019 to have reached 1.372 Tbit/s write speed (171 GB/s) on a cluster of 350 physical machines. The benchmark scenario, conducted under production conditions with standard hardware (commodity servers with 7200 rpm HDDs), consisted in backing up a 38 PB Hadoop datalake via the DistCp command. This level of performance marked, according to analysts, the arrival of a new generation of object storage technologies oriented toward high performance and hyper-scalability.

    Read more →