A Bayesian network (also known as a Bayes network, Bayes net, belief network, or decision network) is a probabilistic graphical model that represents a set of variables and their conditional dependencies via a directed acyclic graph (DAG). While it is one of several forms of causal notation, causal networks are special cases of Bayesian networks. Bayesian networks are ideal for taking an event that occurred and predicting the likelihood that any one of several possible known causes was the contributing factor. For example, a Bayesian network could represent the probabilistic relationships between diseases and symptoms. Given symptoms, the network can be used to compute the probabilities of the presence of various diseases. Efficient algorithms can perform inference and learning in Bayesian networks. Bayesian networks that model sequences of variables (e.g. speech signals or protein sequences) are called dynamic Bayesian networks. Generalizations of Bayesian networks that can represent and solve decision problems under uncertainty are called influence diagrams. == Graphical model == Formally, Bayesian networks are directed acyclic graphs (DAGs) whose nodes represent variables in the Bayesian sense: they may be observable quantities, latent variables, unknown parameters or hypotheses. Each edge represents a direct conditional dependency. Any pair of nodes that are not connected (i.e. no path connects one node to the other) represent variables that are conditionally independent of each other. Each node is associated with a probability function that takes, as input, a particular set of values for the node's parent variables, and gives (as output) the probability (or probability distribution, if applicable) of the variable represented by the node. For example, if m {\displaystyle m} parent nodes represent m {\displaystyle m} Boolean variables, then the probability function could be represented by a table of 2 m {\displaystyle 2^{m}} entries, one entry for each of the 2 m {\displaystyle 2^{m}} possible parent combinations. Similar ideas may be applied to undirected, and possibly cyclic, graphs such as Markov networks. == Example == Suppose we want to model the dependencies between three variables: the sprinkler (or more appropriately, its state - whether it is on or not), the presence or absence of rain and whether the grass is wet or not. Observe that two events can cause the grass to become wet: an active sprinkler or rain. Rain has a direct effect on the use of the sprinkler (namely that when it rains, the sprinkler usually is not active). This situation can be modeled with a Bayesian network (shown to the right). Each variable has two possible values, T (for true) and F (for false). The joint probability function is, by the chain rule of probability, Pr ( G , S , R ) = Pr ( G ∣ S , R ) Pr ( S ∣ R ) Pr ( R ) {\displaystyle \Pr(G,S,R)=\Pr(G\mid S,R)\Pr(S\mid R)\Pr(R)} where G = "Grass wet (true/false)", S = "Sprinkler turned on (true/false)", and R = "Raining (true/false)". The model can answer questions about the presence of a cause given the presence of an effect (so-called inverse probability) like "What is the probability that it is raining, given the grass is wet?" by using the conditional probability formula and summing over all nuisance variables: Pr ( R = T ∣ G = T ) = Pr ( G = T , R = T ) Pr ( G = T ) = ∑ x ∈ { T , F } Pr ( G = T , S = x , R = T ) ∑ x , y ∈ { T , F } Pr ( G = T , S = x , R = y ) {\displaystyle \Pr(R=T\mid G=T)={\frac {\Pr(G=T,R=T)}{\Pr(G=T)}}={\frac {\sum _{x\in \{T,F\}}\Pr(G=T,S=x,R=T)}{\sum _{x,y\in \{T,F\}}\Pr(G=T,S=x,R=y)}}} Using the expansion for the joint probability function Pr ( G , S , R ) {\displaystyle \Pr(G,S,R)} and the conditional probabilities from the conditional probability tables (CPTs) stated in the diagram, one can evaluate each term in the sums in the numerator and denominator. For example, Pr ( G = T , S = T , R = T ) = Pr ( G = T ∣ S = T , R = T ) Pr ( S = T ∣ R = T ) Pr ( R = T ) = 0.99 × 0.01 × 0.2 = 0.00198. {\displaystyle {\begin{aligned}\Pr(G=T,S=T,R=T)&=\Pr(G=T\mid S=T,R=T)\Pr(S=T\mid R=T)\Pr(R=T)\\&=0.99\times 0.01\times 0.2\\&=0.00198.\end{aligned}}} Then the numerical results (subscripted by the associated variable values) are Pr ( R = T ∣ G = T ) = 0.00198 T T T + 0.1584 T F T 0.00198 T T T + 0.288 T T F + 0.1584 T F T + 0.0 T F F = 891 2491 ≈ 35.77 % . {\displaystyle \Pr(R=T\mid G=T)={\frac {0.00198_{TTT}+0.1584_{TFT}}{0.00198_{TTT}+0.288_{TTF}+0.1584_{TFT}+0.0_{TFF}}}={\frac {891}{2491}}\approx 35.77\%.} To answer an interventional question, such as "What is the probability that it would rain, given that we wet the grass?" the answer is governed by the post-intervention joint distribution function Pr ( S , R ∣ do ( G = T ) ) = Pr ( S ∣ R ) Pr ( R ) {\displaystyle \Pr(S,R\mid {\text{do}}(G=T))=\Pr(S\mid R)\Pr(R)} obtained by removing the factor Pr ( G ∣ S , R ) {\displaystyle \Pr(G\mid S,R)} from the pre-intervention distribution. The do operator forces the value of G to be true. The probability of rain is unaffected by the action: Pr ( R ∣ do ( G = T ) ) = Pr ( R ) . {\displaystyle \Pr(R\mid {\text{do}}(G=T))=\Pr(R).} To predict the impact of turning the sprinkler on: Pr ( R , G ∣ do ( S = T ) ) = Pr ( R ) Pr ( G ∣ R , S = T ) {\displaystyle \Pr(R,G\mid {\text{do}}(S=T))=\Pr(R)\Pr(G\mid R,S=T)} with the term Pr ( S = T ∣ R ) {\displaystyle \Pr(S=T\mid R)} removed, showing that the action affects the grass but not the rain. These predictions may not be feasible given unobserved variables, as in most policy evaluation problems. The effect of the action do ( x ) {\displaystyle {\text{do}}(x)} can still be predicted, however, whenever the back-door criterion is satisfied. It states that, if a set Z of nodes can be observed that d-separates (or blocks) all back-door paths from X to Y then Pr ( Y , Z ∣ do ( x ) ) = Pr ( Y , Z , X = x ) Pr ( X = x ∣ Z ) . {\displaystyle \Pr(Y,Z\mid {\text{do}}(x))={\frac {\Pr(Y,Z,X=x)}{\Pr(X=x\mid Z)}}.} A back-door path is one that ends with an arrow into X. Sets that satisfy the back-door criterion are called "sufficient" or "admissible." For example, the set Z = R is admissible for predicting the effect of S = T on G, because R d-separates the (only) back-door path S ← R → G. However, if S is not observed, no other set d-separates this path and the effect of turning the sprinkler on (S = T) on the grass (G) cannot be predicted from passive observations. In that case P(G | do(S = T)) is not "identified". This reflects the fact that, lacking interventional data, the observed dependence between S and G is due to a causal connection or is spurious (apparent dependence arising from a common cause, R). (see Simpson's paradox) To determine whether a causal relation is identified from an arbitrary Bayesian network with unobserved variables, one can use the three rules of "do-calculus" and test whether all do terms can be removed from the expression of that relation, thus confirming that the desired quantity is estimable from frequency data. Using a Bayesian network can save considerable amounts of memory over exhaustive probability tables, if the dependencies in the joint distribution are sparse. For example, a naive way of storing the conditional probabilities of 10 two-valued variables as a table requires storage space for 2 10 = 1024 {\displaystyle 2^{10}=1024} values. If no variable's local distribution depends on more than three parent variables, the Bayesian network representation stores at most 10 ⋅ 2 3 = 80 {\displaystyle 10\cdot 2^{3}=80} values. One advantage of Bayesian networks is that it is intuitively easier for a human to understand (a sparse set of) direct dependencies and local distributions than complete joint distributions. == Inference and learning == Bayesian networks perform three main inference tasks: Inferring unobserved variables Parameter learning for the probability distributions of each node in the network Structure learning of the graphical network === Inferring unobserved variables === Because a Bayesian network is a complete model for its variables and their relationships, it can be used to answer probabilistic queries about them. For example, the network can be used to update knowledge of the state of a subset of variables when other variables (the evidence variables) are observed. This process of computing the posterior distribution of variables given evidence is called probabilistic inference. The posterior gives a universal sufficient statistic for detection applications, when choosing values for the variable subset that minimize some expected loss function, for instance the probability of decision error. A Bayesian network can thus be considered a mechanism for automatically applying Bayes' theorem to complex problems. The most common exact inference methods are: variable elimination, which eliminates (by integration or summation) the non-observed non-query variables one by one by distributing the sum over the prod
Tradeshift
Tradeshift is a cloud based business network and platform for purchase-to-pay automation, supply chain payments, marketplaces, virtual cards and supply chain financing. Its 2018 round of funding, led by Goldman Sachs, raised US$250 million at a valuation of $1.1 billion, giving the company unicorn status. Tradeshift is headquartered in San Francisco, California and has offices in London, Copenhagen, Bucharest and Kuala Lumpur. Tradeshift has reprocessed over $1 trillion USD through transactions on its network. == History == Tradeshift was founded in 2010 by Christian Lanng, Mikkel Hippe Brun, and Gert Sylvest. Inspiration for Tradeshift came after they created the world's first large scale peer-to-peer infrastructure for an e-business called NemHandel. The founders also had leading roles (Governing board member, Technical Director) in the European Commission project PEPPOL inside the European Union. In 2010, the Tradeshift platform launched in May in Copenhagen. Tradeshift won the European Startup Awards in the category of "Best Business or Enterprise Startup." In 2011, Tradeshift made its app marketplace available. In 2012, Tradeshift moved their headquarters from Copenhagen to San Francisco. In 2013, Tradeshift opened an R&D center in Suzhou, China. Tradeshift opened an additional office in London. And LATAM e-invoicing capabilities were added through partnership with Invoiceware. In 2014, Tradeshift expanded with offices in Tokyo, Paris, and Munich. The EU Commission officially approved the Universal Business Language (UBL) data format – a format Tradeshift supports – as eligible for referencing in tenders from public administrations. In 2015, Tradeshift won the Circulars "Digital Disruptor" Award at the WEF conference in Davos, Switzerland. Tradeshift also acquired product information management company Merchantry, and launched e-procurement and supplier risk management solutions. In 2016, Tradeshift acquired Hyper Travel and secured a $75 million series-D round funding. In 2017, Tradeshift acquired IBX Business Network and launches Tradeshift Ada. In 2018, Tradeshift secured a $250 million series-E round funding. and launched Blockchain Payments, the latter as part of Tradeshift Pay. In December 2018 Tradeshift acquired Babelway, an online B2B integration platform. The acquisition added three new office locations to Tradeshift (Salt Lake City, Louvain-la-neuve, Belgium, Cairo Egypt). In Q3 2018, Tradeshift reported year-over-year revenue growth of 400%, new bookings growth of 284%, and gross merchandise volume (GMV) growth of 262%. New total contract value also grew by US$47 million. Additionally, it added 27 new customers including Hertz, Shiseido, ECU and multiple Fortune 500 companies. In July 2023, HSBC and Tradeshift announced an agreement to launch a new, jointly owned business focused on the development of embedded finance solutions and financial services apps. As part of the agreement, HSBC made a $35 million investment into Tradeshift and joined its board. The agreement was part of a funding round which is expected to raise a minimum of $70 million from HSBC and other investors. The new joint venture will allow HSBC and Tradeshift to deploy a range of digital solutions across Tradeshift and other platforms. This includes payment and fintech services embedded into trade, e-commerce and marketplace experiences. In September 2023, CEO Lanng was fired for "gross misconduct on multiple grounds," including "allegations of sexual assault and harassment." Tradeshift was alleged to have fired his accuser after she complained to the company's human resources department, its co-founders and members of its board of directors about his abuse. == Financials == The company's valuation as of May 2018 was $1.1 billion. Tradeshift is now considered a unicorn, and, according to Bloomberg, will not need any further funding. Jan 14, 2020, Tradeshift announced that they had raised $240 million in Series F finance. == Acquisitions == In 2015, Tradeshift acquired product information management company Merchantry. Merchantry is a retail product information management (PIM) software for multi-vendor ecommerce retailers. In 2016, Tradeshift acquired Hyper Travel. Hyper Travel is a travel management service that allows customers to access travel agents via its native messaging apps, SMS, and email. In 2017, Tradeshift acquired IBX Group. In 2018, Tradeshift acquired Babelway, an online B2B integration platform.
Data lake
A data lake is a system or repository of data stored in its natural/raw format, usually object blobs or files. A data lake is usually a single store of data including raw copies of source system data, sensor data, social data etc., and transformed data used for tasks such as reporting, visualization, advanced analytics, and machine learning. A data lake can include structured data from relational databases (rows and columns), semi-structured data (CSV, logs, XML, JSON), unstructured data (emails, documents, PDFs), and binary data (images, audio, video). A data lake can be established on premises (within an organization's data centers) or in the cloud (using cloud services). == Background == James Dixon, then chief technology officer at Pentaho, coined the term by 2011 to contrast it with data mart, which is a smaller repository of interesting attributes derived from raw data. In promoting data lakes, he argued that data marts have several inherent problems, such as information siloing. PricewaterhouseCoopers (PwC) said that data lakes could "put an end to data silos". In their study on data lakes, they noted that enterprises were "starting to extract and place data for analytics into a single, Hadoop-based repository." == Examples == Many companies use cloud storage services such as Google Cloud Storage and Amazon S3 or a distributed file system such as Apache Hadoop distributed file system (HDFS). There is a gradual academic interest in the concept of data lakes. For example, Personal DataLake at Cardiff University is a new type of data lake which aims at managing big data of individual users by providing a single point of collecting, organizing, and sharing personal data. Early data lakes, such as Hadoop 1.0, had limited capabilities because it only supported batch-oriented processing (Map Reduce). Interacting with it required expertise in Java, map reduce and higher-level tools like Apache Pig, Apache Spark and Apache Hive (which were also originally batch-oriented). == Criticism == Poorly managed data lakes have been facetiously called data swamps. In June 2015, David Needle characterized "so-called data lakes" as "one of the more controversial ways to manage big data". PwC was also careful to note in their research that not all data lake initiatives are successful. They quote Sean Martin, CTO of Cambridge Semantics: We see customers creating big data graveyards, dumping everything into Hadoop distributed file system (HDFS) and hoping to do something with it down the road. But then they just lose track of what’s there. The main challenge is not creating a data lake, but taking advantage of the opportunities it presents. They describe companies that build successful data lakes as gradually maturing their lake as they figure out which data and metadata are important to the organization. Another criticism is that the term data lake is used with many different meanings. It may be used to refer to, for example: any tools or data management practices that are not data warehouses; a particular technology for implementation; a raw data reservoir; a hub for ETL offload; or a central hub for self-service analytics. While critiques of data lakes are warranted, in many cases they apply to other data projects as well. For example, the definition of data warehouse is also changeable, and not all data warehouse efforts have been successful. In response to various critiques, McKinsey noted that the data lake should be viewed as a service model for delivering business value within the enterprise, not a technology outcome. == Data lakehouses == Data lakehouses are a hybrid approach that can ingest a variety of raw data formats like a data lake, while also providing ACID transactions and enforced data quality like a data warehouse.
Multistage interconnection networks
Multistage interconnection networks (MINs) are a class of high-speed computer networks usually composed of processing elements (PEs) on one end of the network and memory elements (MEs) on the other end, connected by switching elements (SEs). The switching elements themselves are usually connected to each other in stages, hence the name. MINs are typically used in high-performance or parallel computing as a low-latency interconnection (as opposed to traditional packet switching networks), though they could be implemented on top of a packet switching network. Though the network is typically used for routing purposes, it could also be used as a co-processor to the actual processors for such uses as sorting; cyclic shifting, as in a perfect shuffle network; and bitonic sorting. == Background == Interconnection network are used to connect nodes, where nodes can be a single processor or group of processors, to other nodes. Interconnection networks can be categorized on the basis of their topology. Topology is the pattern in which one node is connected to other nodes. There are two main types of topology: static and dynamic. Static interconnect networks are hard-wired and cannot change their configurations. A regular static interconnect is mainly used in small networks made up of loosely couple nodes. The regular structure signifies that the nodes are arranged in specific shape and the shape is maintained throughout the networks. Some examples of static regular interconnections are: Completely connected network In a mesh network, multiple nodes are connected with each other. Each node in the network is connected to every other node in the network. This arrangement allows proper communication of the data between the nodes. But, there are a lot of communication overheads due to the increased number of node connections. Shared busThis network topology involves connection of the nodes with each other over a bus. Every node communicates with every other node using the bus. The bus utility ensures that no data is sent to the wrong node. But, the bus traffic is an important parameter which can affect the system. RingThis is one of the simplest ways of connecting nodes with each other. The nodes are connected with each other to form a ring. For a node to communicate with some other node, it has to send the messages to its neighbor. Therefore, the data message passes through a series of other nodes before reaching the destination. This involves increased latency in the system. TreeThis topology involves connection of the nodes to form a tree. The nodes are connected to form clusters and the clusters are in-turn connected to form the tree. This methodology causes increased complexity in the network. Hypercube This topology consists of connections of the nodes to form cubes. The nodes are also connected to the nodes on the other cubes. ButterflyThis is one of the most complex connections of the nodes. As the figure suggests, there are nodes which are connected and arranged in terms of their ranks. They are arranged in the form of a matrix. In dynamic interconnect networks, the nodes are interconnected via an array of simple switching elements. This interconnection can then be changed by use of routing algorithms, such that the path from one node to other nodes can be varied. Dynamic interconnections can be classified as: Single stage Interconnect Network Multistage interconnect Network Crossbar switch connections == Crossbar Switch Connections == In crossbar switch, there is a dedicated path from one processor to other processors. Thus, if there are n inputs and m outputs, we will need nm switches to realize a crossbar. As the number of outputs increases, the number of switches increases by factor of n. For large network this will be a problem. An alternative to this scheme is staged switching. == Single Stage Interconnect Network == In a single stage interconnect network, the input nodes are connected to output via a single stage of switches. The figure shows 88 single stage switch using shuffle exchange. As one can see, from a single shuffle, not all input can reach all output. Multiple shuffles are required for all inputs to be connected to all the outputs. == Multistage Interconnect Network == A multistage interconnect network is formed by cascading multiple single stage switches. The switches can then use their own routing algorithm, or be controlled by a centralized router, to form a completely interconnected network. Multistage Interconnect Network can be classified into three types: Non-blocking: A non-blocking network can connect any idle input to any idle output, regardless of the connections already established across the network. Crossbar is an example of this type of network. Rearrangeable non-blocking: This type of network can establish all possible connections between inputs and outputs by rearranging its existing connections. Blocking: This type of network cannot realize all possible connections between inputs and outputs. This is because a connection between one free input to another free output is blocked by an existing connection in the network. The number of switching elements required to realize a non-blocking network in highest, followed by rearrangeable non-blocking. Blocking network uses least switching elements. == Examples == Multiple types of multistage interconnection networks exist. === Omega network === An Omega network consists of multiple stages of 22 switching elements. Each input has a dedicated connection to an output. An NN omega network has log2(N) stages and N/2 switching elements in each stage for a perfect shuffle between stages. Thus the network has complexity of 0(N log(N)). Each switching element can employ its own switching algorithm. Consider an 88 omega network. There are 8! = 40320 1-to-1 mappings from input to output. There are 12 switching element for a total permutation of 2^12 = 4096. Thus, it is a blocking network. === Clos network === A Clos network uses 3 stages to switch from N inputs to N outputs. In the first stage, there are r= N/n crossbar switches and each switch is of size nm. In the second stage there are m switches of size rr and finally the last stage is a mirror of the first stage with r switches of size mn. A clos network will be completely non-blocking if m >= 2n-1. The number of connections, though more than omega network is much less than that of a crossbar network. === Beneš network === A Beneš network is a rearrangeably non-blocking network derived from the clos network by initializing n = m = 2. There are (2log2(N) - 1) stages, with each stage containing N/2 22 crossbar switches. An 88 Beneš network has 5 stages of switching elements, and each stage has 4 switching elements. The center three stages has two 44 benes network. The 44 Beneš network, can connect any input to any output recursively.
Social media stock bubble
The social media bubble is a hypothesis stating that there was a speculative boom and bust phenomenon in the field of social media in the 2010s, particularly in the United States. The Wall Street Journal defined a bubble as stocks "priced above a level that can be justified by economic fundamentals," but this bubble includes social media. Social networking services (SNS) have seen huge growth since 2006, but some investors believed around 2014-2015, that the "bubble" was similar to the dot-com bubble of the late 1990s and early 2000s. In 2015, Mark Cuban, owner of the Dallas Mavericks NBA team and star of the TV show, Shark Tank, sounded an alarm on his personal blog over the social media bubble, calling it worse than the tech bubble in 2000 due to the lack of liquidity in social media stocks. A year prior, however, Cuban told CNBC that he did not believe social media stocks were on the verge of a bubble. In a letter to investors in 2014, David Einhorn, who runs the hedge-fund Greenlight Capital, wrote that "we are witnessing our second tech bubble in 15 years." He went on to write, "What is uncertain is how much further the bubble can expand, and what might pop it." Einhorn cited several factors supporting the existence an over-exuberance including "rejection of conventional valuation methods" and "huge first day IPO pops for companies that have done little more than use the right buzzwords and attract the right venture capital." Since those claims, services like Facebook, Twitter, Instagram, and Snapchat have grown to become multi-billion-dollar corporations generating enormous revenues, though some continue to lose money. == History of social networking services == Social networking services have grown and evolved with time since the launch of SixDegrees.com in 1997. Cutting edge at its time, SixDegrees.com allowed users to create a profile, invite friends, and connect within its platform. At its peak, SixDegrees.com had more than 3.5 million users. Between 1997 and 2001 more social sites aimed at allowing users to connect with others for personal, professional, or dating reasons. Friendster and MySpace were next to enter the social SNS arena, followed by Facebook in 2004. Even though MySpace had a following of more than 300 million users, it could not compete with Facebook, which now has overtaken the social networking world. However, as development of SNS started to emerge, a market saturation began to take effect. Some classrooms have begun to incorporate technology in daily learning as well as social channels specific to student's course work. Traditional social media sites are used, as are educational oriented sites such as ShowMe and Educreations Interactive Whiteboard. == Controversies == While SNS continue to play an influential role in helping people form real-world connections via the Internet, renewed concerns over the social media bubble have surfaced due to recent controversies. These threats include growing concerns about breaches in data, the rise of bot accounts, and the sharing of fake news on SNS platforms. There are also concerns that big data figures associated with these SNS are inflated or fake, as well as worries about the role the platforms played in national elections (see Russian interference in the 2016 United States elections). These issues have resulted in a lack of trust among the sites' users.
CatDV
CatDV is a media asset manager program for handling multimedia production workflows developed by Square Box Systems. Quantum Corporation acquired Square Box Systems in 2020. == Versions == The full family of CatDV Products is as follows: CatDV Standalone Products CatDV Professional Edition CatDV Pegasus CatDV Networked Products CatDV Essential - entry level server product CatDV Enterprise Server - for MySQL databases and most common server platforms including Linux, Windows and Mac OS X CatDV Pegasus Server - adds features such as high performance full-text indexing, access control lists, and more CatDV Worker Node - automated workflow and transcoding engine CatDV Web Client - provides access to the CatDV database via a web browser. There is no need to install special software on the desktop, making it easy to deploy to a large number of users. CatDV Professional Edition & Pegasus Clients - designed to support the multi-user capabilities of the CatDV Enterprise and Workgroup Servers from the desktop Using plugins and scripting, which often require additional professional services support to set up, complex integrations with a wide variety of third party systems (including archive, cloud storage, and artificial intelligence) are possible. == Awards == CatDV won two awards in 2010, a blue ribbon from Creative COW Magazine and a "Best of Show Vidy Award" from Videography. In April 2012 Square Box won a Queen's Award for Enterprise for CatDV.
Signatures with efficient protocols
Signatures with efficient protocols are a form of digital signature invented by Jan Camenisch and Anna Lysyanskaya in 2001. In addition to being secure digital signatures, they need to allow for the efficient implementation of two protocols: A protocol for computing a digital signature in a secure two-party computation protocol. A protocol for proving knowledge of a digital signature in a zero-knowledge protocol. In applications, the first protocol allows a signer to possess the signing key to issue a signature to a user (the signature owner) without learning all the messages being signed or the complete signature. The second protocol allows the signature owner to prove that he has a signature on many messages without revealing the signature and only a (possibly) empty subset of the messages. The combination of these two protocols allows for the implementation of digital credential and ecash protocols.