Morphological antialiasing (MLAA) is a spatial anti-aliasing technique used in real-time computer graphics. It reduces artifacts, such as jaggies, when representing a high-resolution image at a lower resolution. MLAA is a post-process filtering which detects borders in the resulting image and then finds specific patterns in these. Anti-aliasing is achieved by blending pixels in these borders, according to the pattern they belong to and their position within the pattern. Introduced in 2009, MLAA was an early and influential example of anti-aliasing techniques done in post-processing, which makes them suitable for deferred shading. A similar method in this class is fast approximate anti-aliasing (FXAA). Temporal anti-aliasing, also a post-process, has become the most common anti-aliasing method for real-time rendering and video games. Enhanced subpixel morphological antialiasing, or SMAA, is an image-based GPU-based implementation of MLAA developed by Universidad de Zaragoza and Crytek.
ShareMethods
ShareMethods is a Web 2.0 document management and collaboration service with a focus on sales, marketing, and the extended selling network. It offers a software as a service (SaaS) subscription to companies and is available as a stand-alone application or as an integrated program with CRM tools such as Oracle CRM On Demand or salesforce.com. == History == ShareMethods was launched in 2004 to provide collaboration and communication services for sales and marketing teams, business partners, and customers. The founders have a background of building software-as-a-service applications and creating digital media applications. In September 2005, ShareMethods launched "ShareNow" as one of the first applications on the salesforce.com AppExchange. In September 2006, ShareMethods moved its operations into a SAS 70 Type II data center owned by SunGard. In March 2009, ShareMethods launched "ShareSpaces" to provide on-demand portals or workspaces. In 2013, ShareMethods announced that its platform is available in a private cloud (on-premises) version. == Products == ShareMethods: Combines document management, collaboration, analytics, and CRM integration into a single solution. Key content can be centrally managed and delivered to sales channels, while providing feedback to marketing. ShareMethods is often used as a sales portal for internal sales and a partner portal for external partners. ShareNow: Integrates ShareMethods with salesforce.com providing Single Sign On for salesforce.com users and access to files related to accounts opportunities, etc. including custom objects. Also facilitates collaboration between salesforce.com users and non-users. ShareMethods for Oracle CRM On Demand: Integrates ShareMethods with Oracle CRM On Demand providing Single Sign On for Oracle users and easy access to files related to accounts opportunities, etc. ShareOffice: An on-demand intranet/extranet solution. Features include full-text search, version history, server sync-up, email updates, audit trail/analytics, check-in/check-out, multilingual user interface. ShareSpaces: Independent workspaces or portals where users can collaborate with business partners, teammates, or individuals to work together on content and documents. == Integration and interoperability == ShareMethods is available on Salesforce.com's AppExchange platform. ShareMethods also integrates with Oracle CRM On Demand to provide document management within the CRM application. Customers also can integrate proprietary systems via single-sign-on and self-registration. In addition, developers can make use of the ShareMethods API based on WebDAV to integrate document management functionality.
Refik Anadol
Refik Anadol (born November 7, 1985) is a Turkish American media artist and the co-founder of Refik Anadol Studio and Dataland. Recognized as a pioneer in the aesthetics of data visualization and AI arts, his work merges art, technology, science, and architecture. Through media embedded into existing architecture, live audio-visual performances, immersive rooms, exhibitions, AI data paintings and sculptures, and digital collections, Anadol explores collective memories, humanity's relationship to nature, the perception of space and time, and human-machine collaborations. His work has been exhibited in more than seventy cities on six continents. == Early life and education == Anadol was born and raised in Istanbul and grew up in a family of teachers. He taught himself basic programming on a Commodore 64 when he was eight. His connection to machines began with coding and video games. Anadol saw Blade Runner for the first time when he was eight; his mother said the way he perceived his surroundings shifted the day after he saw the film. He was fascinated with its futuristic depiction of downtown Los Angeles, and transfixed by as a scene during which a replicant discovers that her memories are an implanted component of her machine mind, In a 2024 interview with the Financial Times, he said: "Since that moment, one of my inspirations has been that question: 'What can a machine do with someone else's memories?" Anadol attended Istanbul Bilgi University, where he received a BA in photography and video in 2009 and an MFA in visual communication in 2011. In 2014 he earned an MFA in design media arts at UCLA. He was mentored by Casey Reas, Jennifer Steinkamp, and Christian Moeller. == Career and selected works == === 2008–2012: Data painting, Quadrature and Quadrangle, Istanbul Biennial === As an undergraduate, Anadol read a paper by Lev Manovich on augmented space. Manovich's assertion that collaborations between architects and artists could make the "invisible flow of data visible" triggered Anadol's imagination, and in 2008, he altered built space for the first time. Bringing a projector outside, he projected large-scale images onto a concrete to create the illusion of movement. Coining the term "data painting," the piece inspired Anadol to use light as material and data as pigment. In 2010 he created Quadrature with Alican Aktürk, a fellow graduate student, at the SantralIstanbul Art and Culture Center's main gallery building. A live audio-visual performance that examined the relationship between architecture and media, Quadrature used video projection techniques to manipulate footage of quadrilaterals. He followed Quadrature with Quadrangle at SANAA School of Design in Essen, Germany, using the entire 360 degrees of the building as a canvas. In 2011, he was invited to create a media installation at the Istanbul Biennial on the heavily trafficked İstiklal Avenue. He created a site-specific large-scale interpretation of sounds he recorded during different times of day, and used nine projectors to project reinterpreted images. The work was titled Augmented Structures v1.0. Anadol's first solo exhibition, Sceptical Interventions, was held at the Piveneli Gallery in Istanbul in early 2012. Later that year he moved to Los Angeles to attend UCLA's Design Media Arts program. The first place he went after his arrival was downtown Los Angeles. [6] === 2013–2016: Visions of America: Amériques, Infinity Room, Google AMI === In 2013, at Microsoft Research's annual Design Expo, Anadol presented his idea to use the external walls of Walt Disney Concert Hall as a canvas. His presentation brought him to the attention of Gehry Technologies, and with the support of Gehry and his team, Anadol was offered the use of the original 3D model of the concert hall. For his 2014 thesis project, with assistance from architects and UCLA researchers, he created a site-specific architectural video installation inside the concert hall that accompanied a Los Angeles Philharmonic performance of Edgard Varèse's Amérique. Titled Visions of America: Amériques, Anadol used algorithmic sound analysis to listen and respond to the music in real-time. He tracked conductor Esa-Pekka Salonen's heartbeat with a sensor and used a 3-D camera system to integrate Salonen's movements. He created Infinity Room at the Zorlu PSM for the 2015 Istanbul Biennial. Rather than creating an illusion only with mirrors, Anadol used pixel and 3D projection mapping to transform every surface of the room into an abstract infinite moving space. A temporary immersive environment, Infinity Room was also exhibited at events including South by Southwest in Austin, Texas, the New Zealand Festival in Wellington, New Zealand, and Jeffrey Deitch in Los Angeles. In 2016, Anadol was awarded the first Google Artists and Machine Intelligence Artist Residency; it was just after a team at Google opened up the algorithm for DeepDream, a computer vision program that prompted Anadol's realization that if a machine could learn, it could remember, dream, and hallucinate. === 2017–2018: Winds of Boston, Archive Dreaming, Melting Memories, WDCH Dreams === In 2017, he created the data painting Winds of Boston, a 6' x 13' foot video installation in the lobby of a Boston office building, using software he created to read, analyze and visualize wind speed, direction, and gust patterns along with time and temperature at 20-second intervals recorded over a one-year period at Logan International Airport. Later in the year, he used AI to generate infinite new outputs based on a massive dataset for Archive Dreaming, an immersive installation at Salt Research, a contemporary gallery and library in Istanbul. Inspired by his idea of consciousness and its context within AI, as well as Jorge Luis Borges' The Library of Babel, Anadol used AI and machine learning to look at and discover interactions and correlations between 1.7 million items culled from 40,000 publications covering Turkish contemporary and modern art, architecture, and economics from 1997 to 2010. Archive Dreaming, which could be controlled by users with a joystick, dreamed of unexpected correlations among documents when idle. In 2018, after his uncle was diagnosed with Alzheimer's, Anadol created Melting Memories. Working with scientists from the neuroscape laboratory at the University of California, San Francisco, he used academic data from the neuroscience archives and EEG scans of an anonymous Alzheimer's disease dataset to create AI-generated visuals related to memory, health, degeneration, and decay.Melting Memories was projected on the walls of Pilevneli Gallery; visitors to the exhibition could watch as millions of pixels reconstructed people's memories. Anadol won the Lumen Prize Gold Award for Melting Memories. Anadol was commissioned by the Los Angeles Philharmonic to create an installation to celebrate the orchestra's centennial anniversary in 2018. He worked with Google's Kenric MacDowell to create WDCH Dreams, using algorithmic visualizations of data to mimic the process of human dreaming. Projected across the exterior walls of Walt Disney Concert Hall using 42 large-scale projectors with 50K visual resolution, 8-channel sound, and 1.2M luminance, Anadol painted with data points culled from the orchestra's archives, including 587,763 images, 1,880 videos, 1,483 metadata files, and 17,773 audio files. Because Gehry gave him access to the 3D architectural files of Walt Disney Concert Hall, Anadol knew the exact contours of the building. WDCH Dreams debuted in September 2018. A 12-minute performance in three parts staged every 30 minutes over ten nights, "Centennial Memories,” the first piece, used 44.5 terabytes of historical data from the Phil's archives. It was followed by "Consciousness", which processed every note the orchestra has ever recorded, using billions of data points to generate connections; and "Dream," which merged "Centennial Memories" and "Consciousness" to create hallucinations that were described in the New York Times as "a sort of combinatorial Fantasia. === 2019–2021: Machine Hallucinations: NYC, Machine Hallucinations: Nature Dreams, Machine Memories: Space, Quantum Memories === In 2019, Refik Anadol presented Latent History at Fotografiska Stockholm. The site specific installation transformed photographic archives of Stockholm into a large scale, machine generated visual projection displayed in the museum’s main exhibition hall. Drawing on thousands of archival images spanning approximately 150 years, the work used artificial intelligence to reinterpret the city’s historical imagery as a continuously evolving visual narrative.. Anadol began thinking about the work that would become the Machine Hallucinations series while in residence at Google. In 2019, he completed the first work in the series, Machine Hallucinations: NYC, which used 300 million photos of New York City and 113 million additional data points, including subway sounds, ra
Vagueness
In linguistics and philosophy, a vague predicate is one which gives rise to borderline cases. For example, the English adjective "tall" is vague since it is not clearly true or false for someone of middling height. By contrast, the word "prime" is not vague since every number is definitively either prime or not. Vagueness is commonly diagnosed by a predicate's ability to give rise to the sorites paradox. Vagueness is separate from ambiguity, in which an expression has multiple denotations. For instance the word "bank" is ambiguous since it can refer either to a river bank or to a financial institution, but there are no borderline cases between both interpretations. Vagueness is a major topic of research in philosophical logic, where it serves as a potential challenge to classical logic. Work in formal semantics has sought to provide a compositional semantics for vague expressions in natural language. Work in philosophy of language has addressed implications of vagueness for the theory of meaning, while metaphysicists have considered whether reality itself is vague. == Importance == The concept of vagueness has philosophical importance. Suppose one wants to come up with a definition of "right" in the moral sense. One wants a definition to cover actions that are clearly right and exclude actions that are clearly wrong, but what does one do with the borderline cases? Surely, there are such cases. Some philosophers say that one should try to come up with a definition that is itself unclear on just those cases. Others say that one has an interest in making his or her definitions more precise than ordinary language, or his or her ordinary concepts, themselves allow; they recommend one advances precising definitions. === In law === Vagueness is also a problem which arises in law, and in some cases, judges have to arbitrate regarding whether a borderline case does, or does not, satisfy a given vague concept. Examples include disability (how much loss of vision is required before one is legally blind?), human life (at what point from conception to birth is one a legal human being, protected for instance by laws against murder?), adulthood (most familiarly reflected in legal ages for driving, drinking, voting, consensual sex, etc.), race (how to classify someone of mixed racial heritage), etc. Even such apparently unambiguous concepts such as biological sex can be subject to vagueness problems, not just from transsexuals' gender transitions but also from certain genetic conditions which can give an individual mixed male and female biological traits (see intersex). In the common law system, vagueness is a possible legal defence against by-laws and other regulations. The legal principle is that delegated power cannot be used more broadly than the delegator intended. Therefore, a regulation may not be so vague as to regulate areas beyond what the law allows. Any such regulation would be "void for vagueness" and unenforceable. This principle is sometimes used to strike down municipal by-laws that forbid "explicit" or "objectionable" contents from being sold in a certain city; courts often find such expressions to be too vague, giving municipal inspectors discretion beyond what the law allows. In the US this is known as the vagueness doctrine and in Europe as the principle of legal certainty. === In science === Many scientific concepts are of necessity vague, for instance species in biology cannot be precisely defined, owing to unclear cases such as ring species. Nonetheless, the concept of species can be clearly applied in the vast majority of cases. As this example illustrates, to say that a definition is "vague" is not necessarily a criticism. Consider those animals in Alaska that are the result of breeding huskies and wolves: are they dogs? It is not clear: they are borderline cases of dogs. This means one's ordinary concept of doghood is not clear enough to let us rule conclusively in this case. == Approaches == The philosophical question of what the best theoretical treatment of vagueness is—which is closely related to the problem of the paradox of the heap, a.k.a. sorites paradox—has been the subject of much philosophical debate. === Fuzzy logic === One theoretical approach is that of fuzzy logic, developed by American mathematician Lotfi Zadeh. Fuzzy logic proposes a gradual transition between "perfect falsity", for example, the statement "Bill Clinton is bald", to "perfect truth", for, say, "Patrick Stewart is bald". In ordinary logics, there are only two truth-values: "true" and "false". The fuzzy perspective differs by introducing an infinite number of truth-values along a spectrum between perfect truth and perfect falsity. Perfect truth may be represented by "1", and perfect falsity by "0". Borderline cases are thought of as having a "truth-value" anywhere between 0 and 1 (for example, 0.6). Advocates of the fuzzy logic approach have included K. F. Machina (1976) and Dorothy Edgington (1993). === Supervaluationism === Another theoretical approach is known as "supervaluationism". This approach has been defended by Kit Fine and Rosanna Keefe. Fine argues that borderline applications of vague predicates are neither true nor false, but rather are instances of "truth value gaps". He defends an interesting and sophisticated system of vague semantics, based on the notion that a vague predicate might be "made precise" in many alternative ways. This system has the consequence that borderline cases of vague terms yield statements that are neither true, nor false. Given a supervaluationist semantics, one can define the predicate "supertrue" as meaning "true on all precisifications". This predicate will not change the semantics of atomic statements (e.g. "Frank is bald", where Frank is a borderline case of baldness), but does have consequences for logically complex statements. In particular, the tautologies of sentential logic, such as "Frank is bald or Frank is not bald", will turn out to be supertrue, since on any precisification of baldness, either "Frank is bald" or "Frank is not bald" will be true. Since the presence of borderline cases seems to threaten principles like this one (excluded middle), the fact that supervaluationism can "rescue" them is seen as a virtue. === Subvaluationism === Subvaluationism is the logical dual of supervaluationism, and has been defended by Dominic Hyde (2008) and Pablo Cobreros (2011). Whereas the supervaluationist characterises truth as 'supertruth', the subvaluationist characterises truth as 'subtruth', or "true on at least some precisifications". Subvaluationism proposes that borderline applications of vague terms are both true and false. It thus has "truth-value gluts". According to this theory, a vague statement is true if it is true on at least one precisification and false if it is false under at least one precisification. If a vague statement comes out true under one precisification and false under another, it is both true and false. Subvaluationism ultimately amounts to the claim that vagueness is a truly contradictory phenomenon. Of a borderline case of "bald man" it would be both true and false to say that he is bald, and both true and false to say that he is not bald. === Epistemicist view === A fourth approach, known as "the epistemicist view", has been defended by Timothy Williamson (1994), R. A. Sorensen (1988) and (2001), and Nicholas Rescher (2009). They maintain that vague predicates do, in fact, draw sharp boundaries, but that one cannot know where these boundaries lie. One's confusion about whether some vague word does or does not apply in a borderline case is due to one's ignorance. For example, in the epistemicist view, there is a fact of the matter, for every person, about whether that person is old or not old; some people are ignorant of this fact. === As a property of objects === One possibility is that one's words and concepts are perfectly precise, but that objects themselves are vague. Consider Peter Unger's example of a cloud (from his famous 1980 paper, "The Problem of the Many"): it is not clear where the boundary of a cloud lies; for any given bit of water vapor, one can ask whether it is part of the cloud or not, and for many such bits, one will not know how to answer. Hence, perhaps such a term as 'cloud' is not itself vague, but rather precisely denotes a vague object. This strategy has occasionally been poorly received; most notably, in Gareth Evans' short paper "Can There Be Vague Objects?" (1978), wherein an argument is examined which appears to show that vague identity-statements are impossible (i.e., result in logical incoherence). David Lewis explains that the reader is intended to conclude, with Evans, that—since there clearly are, in fact, meaningful vague identities—any purported proof to the contrary cannot be right; and as the proof relies upon the premise that vague terms precisely denote vague objects, but fails under the view that vague terms reflect a merel
Computing Machinery and Intelligence
"Computing Machinery and Intelligence" is a paper written by Alan Turing on the topic of artificial intelligence. The paper, published in 1950 in Mind, was the first to introduce his concept of what is now known as the Turing test to the general public. Turing's paper considers the question "Can machines think?" Turing says that since the words "think" and "machine" cannot clearly be defined, we should "replace the question by another, which is closely related to it and is expressed in relatively unambiguous words." To achieve this objective, Turing proposes a three-step approach. First, he identifies a simple and unambiguous concept to substitute for the term "think." Second, he delineates the specific "machines" under consideration. Third, armed with these tools, he poses a new question related to the first, which he believes he can answer in the affirmative. == Turing's test == Rather than trying to determine if a machine is thinking, Turing suggests we should ask if the machine can win a game, called the "Imitation Game". The original Imitation game, that Turing described, is a simple party game involving three players. Player A is a man, player B is a woman and player C (who plays the role of the interrogator) can be of either sex. In the Imitation Game, player C is unable to see either player A or player B (and knows them only as X and Y), and can communicate with them only through written notes or any other form that does not give away any details about their gender. By asking questions of player A and player B, player C tries to determine which of the two is the man and which is the woman. Player A's role is to trick the interrogator into making the wrong decision, while player B attempts to assist the interrogator in making the right one. Turing proposes a variation of this game that involves the computer: We now ask the question, "What will happen when a machine takes the part of A in this game?" Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, "Can machines think?" So the modified game becomes one that involves three participants in isolated rooms: a computer (which is being tested), a human, and a (human) judge. The human judge can converse with both the human and the computer by typing into a terminal. Both the computer and the human try to convince the judge that they are the human. If the judge cannot consistently tell which is which, then the computer wins the game. Researchers in the United Kingdom had been exploring "machine intelligence" for up to ten years prior to the founding of the field of artificial intelligence (AI) research in 1956. It was a common topic among the members of the Ratio Club, an informal group of British cybernetics and electronics researchers that included Alan Turing. Turing, in particular, had been running the notion of machine intelligence since at least 1941 and one of the earliest-known mentions of "computer intelligence" was made by him in 1947. As Stevan Harnad notes, the question has become "Can machines do what we (as thinking entities) can do?" In other words, Turing is no longer asking whether a machine can "think"; he is asking whether a machine can act indistinguishably from the way a thinker acts. This question avoids the difficult philosophical problem of pre-defining the verb "to think" and focuses instead on the performance capacities that being able to think makes possible, and how a causal system can generate them. Since Turing introduced his test, it has been both highly influential and widely criticised, and has become an important concept in the philosophy of artificial intelligence. Some of its criticisms, such as John Searle's Chinese room, are themselves controversial. Some have taken Turing's question to have been "Can a computer, communicating over a teleprinter, fool a person into believing it is human?" but it seems clear that Turing was not talking about fooling people but about generating human cognitive capacity. == Digital machines == Turing also notes that we need to determine which "machines" we wish to consider. He points out that a human clone, while man-made, would not provide a very interesting example. Turing suggested that we should focus on the capabilities of digital machinery—machines which manipulate the binary digits of 1 and 0, rewriting them into memory using simple rules. He gave two reasons. First, there is no reason to speculate whether or not they can exist. They already did in 1950. Second, digital machinery is "universal". Turing's research into the foundations of computation had proved that a digital computer can, in theory, simulate the behaviour of any other digital machine, given enough memory and time. (This is the essential insight of the Church–Turing thesis and the universal Turing machine.) Therefore, if any digital machine can "act like it is thinking", then every sufficiently powerful digital machine can. Turing writes, "all digital computers are in a sense equivalent." This allows the original question to be made even more specific. Turing now restates the original question as "Let us fix our attention on one particular digital computer C. Is it true that by modifying this computer to have an adequate storage, suitably increasing its speed of action, and providing it with an appropriate programme, C can be made to play satisfactorily the part of A in the imitation game, the part of B being taken by a man?" Hence, Turing states that the focus is not on "whether all digital computers would do well in the game nor whether the computers that are presently available would do well, but whether there are imaginable computers which would do well". What is more important is to consider the advancements possible in the state of our machines today regardless of whether we have the available resource to create one or not. == Nine common objections == Having clarified the question, Turing turned to answering it: he considered the following nine common objections, which include all the major arguments against artificial intelligence raised in the years since his paper was first published. Religious Objection: This states that thinking is a function of man's immortal soul; therefore, a machine cannot think. "In attempting to construct such machines," wrote Turing, "we should not be irreverently usurping His power of creating souls, any more than we are in the procreation of children: rather we are, in either case, instruments of His will providing mansions for the souls that He creates." 'Heads in the Sand' Objection: "The consequences of machines thinking would be too dreadful. Let us hope and believe that they cannot do so." This thinking is popular among intellectual people, as they believe superiority derives from higher intelligence and the possibility of being overtaken is a threat (as machines have efficient memory capacities and processing speed, machines exceeding the learning and knowledge capabilities are highly probable). This objection is a fallacious appeal to consequences, confusing what should not be with what can or cannot be (Wardrip-Fruin, 56). The Mathematical Objection: This objection uses mathematical theorems, such as Gödel's incompleteness theorem, to show that there are limits to what questions a computer system based on logic can answer. Turing suggests that humans are too often wrong themselves and pleased at the fallibility of a machine. (This argument would be made again by philosopher John Lucas in 1961 and physicist Roger Penrose in 1989, and later would be called Penrose–Lucas argument.) Argument From Consciousness: This argument, suggested by Professor Geoffrey Jefferson in his 1949 Lister Oration (acceptance speech for his 1948 award of Lister Medal) states that "not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain." Turing replies by saying that we have no way of knowing that any individual other than ourselves experiences emotions, and that therefore we should accept the test. He adds, "I do not wish to give the impression that I think there is no mystery about consciousness ... [b]ut I do not think these mysteries necessarily need to be solved before we can answer the question [of whether machines can think]." (This argument, that a computer can't have conscious experiences or understanding, would be made in 1980 by philosopher John Searle in his Chinese room argument. Turing's reply is now known as the "other minds reply". See also Can a machine have a mind? in the philosophy of AI.) Arguments from various disabilities. These arguments all have the form "a computer will never do X". Turing offers a selection:Be kind, resourceful, beautiful, friendly, have initiative, have a sense of humour, tell right from wrong, make mistakes, fall in love, enjo
Reparameterization trick
The reparameterization trick (aka "reparameterization gradient estimator") is a technique used in statistical machine learning, particularly in variational inference, variational autoencoders, and stochastic optimization. It allows for the efficient computation of gradients through random variables, enabling the optimization of parametric probability models using stochastic gradient descent, and the variance reduction of estimators. It was developed in the 1980s in operations research, under the name of "pathwise gradients", or "stochastic gradients". Its use in variational inference was proposed in 2013. == Mathematics == Let z {\displaystyle z} be a random variable with distribution q ϕ ( z ) {\displaystyle q_{\phi }(z)} , where ϕ {\displaystyle \phi } is a vector containing the parameters of the distribution. === REINFORCE estimator === Consider an objective function of the form: L ( ϕ ) = E z ∼ q ϕ ( z ) [ f ( z ) ] {\displaystyle L(\phi )=\mathbb {E} _{z\sim q_{\phi }(z)}[f(z)]} Without the reparameterization trick, estimating the gradient ∇ ϕ L ( ϕ ) {\displaystyle \nabla _{\phi }L(\phi )} can be challenging, because the parameter appears in the random variable itself. In more detail, we have to statistically estimate: ∇ ϕ L ( ϕ ) = ∇ ϕ ∫ d z q ϕ ( z ) f ( z ) {\displaystyle \nabla _{\phi }L(\phi )=\nabla _{\phi }\int dz\;q_{\phi }(z)f(z)} The REINFORCE estimator, widely used in reinforcement learning and especially policy gradient, uses the following equality: ∇ ϕ L ( ϕ ) = ∫ d z q ϕ ( z ) ∇ ϕ ( ln q ϕ ( z ) ) f ( z ) = E z ∼ q ϕ ( z ) [ ∇ ϕ ( ln q ϕ ( z ) ) f ( z ) ] {\displaystyle \nabla _{\phi }L(\phi )=\int dz\;q_{\phi }(z)\nabla _{\phi }(\ln q_{\phi }(z))f(z)=\mathbb {E} _{z\sim q_{\phi }(z)}[\nabla _{\phi }(\ln q_{\phi }(z))f(z)]} This allows the gradient to be estimated: ∇ ϕ L ( ϕ ) ≈ 1 N ∑ i = 1 N ∇ ϕ ( ln q ϕ ( z i ) ) f ( z i ) {\displaystyle \nabla _{\phi }L(\phi )\approx {\frac {1}{N}}\sum _{i=1}^{N}\nabla _{\phi }(\ln q_{\phi }(z_{i}))f(z_{i})} The REINFORCE estimator has high variance, and many methods were developed to reduce its variance. === Reparameterization estimator === The reparameterization trick expresses z {\displaystyle z} as: z = g ϕ ( ϵ ) , ϵ ∼ p ( ϵ ) {\displaystyle z=g_{\phi }(\epsilon ),\quad \epsilon \sim p(\epsilon )} Here, g ϕ {\displaystyle g_{\phi }} is a deterministic function parameterized by ϕ {\displaystyle \phi } , and ϵ {\displaystyle \epsilon } is a noise variable drawn from a fixed distribution p ( ϵ ) {\displaystyle p(\epsilon )} . This gives: L ( ϕ ) = E ϵ ∼ p ( ϵ ) [ f ( g ϕ ( ϵ ) ) ] {\displaystyle L(\phi )=\mathbb {E} _{\epsilon \sim p(\epsilon )}[f(g_{\phi }(\epsilon ))]} Now, the gradient can be estimated as: ∇ ϕ L ( ϕ ) = E ϵ ∼ p ( ϵ ) [ ∇ ϕ f ( g ϕ ( ϵ ) ) ] ≈ 1 N ∑ i = 1 N ∇ ϕ f ( g ϕ ( ϵ i ) ) {\displaystyle \nabla _{\phi }L(\phi )=\mathbb {E} _{\epsilon \sim p(\epsilon )}[\nabla _{\phi }f(g_{\phi }(\epsilon ))]\approx {\frac {1}{N}}\sum _{i=1}^{N}\nabla _{\phi }f(g_{\phi }(\epsilon _{i}))} == Examples == For some common distributions, the reparameterization trick takes specific forms: Normal distribution: For z ∼ N ( μ , σ 2 ) {\displaystyle z\sim {\mathcal {N}}(\mu ,\sigma ^{2})} , we can use: z = μ + σ ϵ , ϵ ∼ N ( 0 , 1 ) {\displaystyle z=\mu +\sigma \epsilon ,\quad \epsilon \sim {\mathcal {N}}(0,1)} Exponential distribution: For z ∼ Exp ( λ ) {\displaystyle z\sim {\text{Exp}}(\lambda )} , we can use: z = − 1 λ log ( ϵ ) , ϵ ∼ Uniform ( 0 , 1 ) {\displaystyle z=-{\frac {1}{\lambda }}\log(\epsilon ),\quad \epsilon \sim {\text{Uniform}}(0,1)} Discrete distribution can be reparameterized by the Gumbel distribution (Gumbel-softmax trick or "concrete distribution") and diffusion models. In general, any distribution that is differentiable with respect to its parameters can be reparameterized by inverting the multivariable CDF function, then apply the implicit method. See for an exposition and application to the Gamma, Beta, Dirichlet, and von Mises distributions. == Applications == === Variational autoencoder === In Variational Autoencoders (VAEs), the VAE objective function, known as the Evidence Lower Bound (ELBO), is given by: ELBO ( ϕ , θ ) = E z ∼ q ϕ ( z | x ) [ log p θ ( x | z ) ] − D KL ( q ϕ ( z | x ) | | p ( z ) ) {\displaystyle {\text{ELBO}}(\phi ,\theta )=\mathbb {E} _{z\sim q_{\phi }(z|x)}[\log p_{\theta }(x|z)]-D_{\text{KL}}(q_{\phi }(z|x)||p(z))} where q ϕ ( z | x ) {\displaystyle q_{\phi }(z|x)} is the encoder (recognition model), p θ ( x | z ) {\displaystyle p_{\theta }(x|z)} is the decoder (generative model), and p ( z ) {\displaystyle p(z)} is the prior distribution over latent variables. The gradient of ELBO with respect to θ {\displaystyle \theta } is simply E z ∼ q ϕ ( z | x ) [ ∇ θ log p θ ( x | z ) ] ≈ 1 L ∑ l = 1 L ∇ θ log p θ ( x | z l ) {\displaystyle \mathbb {E} _{z\sim q_{\phi }(z|x)}[\nabla _{\theta }\log p_{\theta }(x|z)]\approx {\frac {1}{L}}\sum _{l=1}^{L}\nabla _{\theta }\log p_{\theta }(x|z_{l})} but the gradient with respect to ϕ {\displaystyle \phi } requires the trick. Express the sampling operation z ∼ q ϕ ( z | x ) {\displaystyle z\sim q_{\phi }(z|x)} as: z = μ ϕ ( x ) + σ ϕ ( x ) ⊙ ϵ , ϵ ∼ N ( 0 , I ) {\displaystyle z=\mu _{\phi }(x)+\sigma _{\phi }(x)\odot \epsilon ,\quad \epsilon \sim {\mathcal {N}}(0,I)} where μ ϕ ( x ) {\displaystyle \mu _{\phi }(x)} and σ ϕ ( x ) {\displaystyle \sigma _{\phi }(x)} are the outputs of the encoder network, and ⊙ {\displaystyle \odot } denotes element-wise multiplication. Then we have ∇ ϕ ELBO ( ϕ , θ ) = E ϵ ∼ N ( 0 , I ) [ ∇ ϕ log p θ ( x | z ) + ∇ ϕ log q ϕ ( z | x ) − ∇ ϕ log p ( z ) ] {\displaystyle \nabla _{\phi }{\text{ELBO}}(\phi ,\theta )=\mathbb {E} _{\epsilon \sim {\mathcal {N}}(0,I)}[\nabla _{\phi }\log p_{\theta }(x|z)+\nabla _{\phi }\log q_{\phi }(z|x)-\nabla _{\phi }\log p(z)]} where z = μ ϕ ( x ) + σ ϕ ( x ) ⊙ ϵ {\displaystyle z=\mu _{\phi }(x)+\sigma _{\phi }(x)\odot \epsilon } . This allows us to estimate the gradient using Monte Carlo sampling: ∇ ϕ ELBO ( ϕ , θ ) ≈ 1 L ∑ l = 1 L [ ∇ ϕ log p θ ( x | z l ) + ∇ ϕ log q ϕ ( z l | x ) − ∇ ϕ log p ( z l ) ] {\displaystyle \nabla _{\phi }{\text{ELBO}}(\phi ,\theta )\approx {\frac {1}{L}}\sum _{l=1}^{L}[\nabla _{\phi }\log p_{\theta }(x|z_{l})+\nabla _{\phi }\log q_{\phi }(z_{l}|x)-\nabla _{\phi }\log p(z_{l})]} where z l = μ ϕ ( x ) + σ ϕ ( x ) ⊙ ϵ l {\displaystyle z_{l}=\mu _{\phi }(x)+\sigma _{\phi }(x)\odot \epsilon _{l}} and ϵ l ∼ N ( 0 , I ) {\displaystyle \epsilon _{l}\sim {\mathcal {N}}(0,I)} for l = 1 , … , L {\displaystyle l=1,\ldots ,L} . This formulation enables backpropagation through the sampling process, allowing for end-to-end training of the VAE model using stochastic gradient descent or its variants. === Variational inference === More generally, the trick allows using stochastic gradient descent for variational inference. Let the variational objective (ELBO) be of the form: ELBO ( ϕ ) = E z ∼ q ϕ ( z ) [ log p ( x , z ) − log q ϕ ( z ) ] {\displaystyle {\text{ELBO}}(\phi )=\mathbb {E} _{z\sim q_{\phi }(z)}[\log p(x,z)-\log q_{\phi }(z)]} Using the reparameterization trick, we can estimate the gradient of this objective with respect to ϕ {\displaystyle \phi } : ∇ ϕ ELBO ( ϕ ) ≈ 1 L ∑ l = 1 L ∇ ϕ [ log p ( x , g ϕ ( ϵ l ) ) − log q ϕ ( g ϕ ( ϵ l ) ) ] , ϵ l ∼ p ( ϵ ) {\displaystyle \nabla _{\phi }{\text{ELBO}}(\phi )\approx {\frac {1}{L}}\sum _{l=1}^{L}\nabla _{\phi }[\log p(x,g_{\phi }(\epsilon _{l}))-\log q_{\phi }(g_{\phi }(\epsilon _{l}))],\quad \epsilon _{l}\sim p(\epsilon )} === Dropout === The reparameterization trick has been applied to reduce the variance in dropout, a regularization technique in neural networks. The original dropout can be reparameterized with Bernoulli distributions: y = ( W ⊙ ϵ ) x , ϵ i j ∼ Bernoulli ( α i j ) {\displaystyle y=(W\odot \epsilon )x,\quad \epsilon _{ij}\sim {\text{Bernoulli}}(\alpha _{ij})} where W {\displaystyle W} is the weight matrix, x {\displaystyle x} is the input, and α i j {\displaystyle \alpha _{ij}} are the (fixed) dropout rates. More generally, other distributions can be used than the Bernoulli distribution, such as the gaussian noise: y i = μ i + σ i ⊙ ϵ i , ϵ i ∼ N ( 0 , I ) {\displaystyle y_{i}=\mu _{i}+\sigma _{i}\odot \epsilon _{i},\quad \epsilon _{i}\sim {\mathcal {N}}(0,I)} where μ i = m i ⊤ x {\displaystyle \mu _{i}=\mathbf {m} _{i}^{\top }x} and σ i 2 = v i ⊤ x 2 {\displaystyle \sigma _{i}^{2}=\mathbf {v} _{i}^{\top }x^{2}} , with m i {\displaystyle \mathbf {m} _{i}} and v i {\displaystyle \mathbf {v} _{i}} being the mean and variance of the i {\displaystyle i} -th output neuron. The reparameterization trick can be applied to all such cases, resulting in the variational dropout method.
Fuzzy concept
A fuzzy concept is an idea of which the boundaries of application can vary considerably according to context or conditions, instead of being fixed once and for all. That means the idea is somewhat vague or imprecise. Yet it is not unclear or meaningless. It has a definite meaning, which can often be made more exact with further elaboration and specification — including a closer definition of the context in which the concept is used. The inverse of a "fuzzy concept" is a "crisp concept" (i.e. a precise concept). Fuzzy concepts are often used to navigate imprecision in the real world, when precise information is not available and an approximate indication is sufficient to be helpful. Although the linguist George Philip Lakoff already defined the semantics of a fuzzy concept in 1973 (inspired by an unpublished 1971 paper by Eleanor Rosch,) the term "fuzzy concept" rarely received a standalone entry in dictionaries, handbooks and encyclopedias. Sometimes it was defined in encyclopedia articles on fuzzy logic, or it was simply equated with a mathematical “fuzzy set”. A fuzzy concept can be "fuzzy" for many different reasons in different contexts. This makes it harder to provide a precise definition that covers all cases. Paradoxically, the definition of fuzzy concepts may itself be somewhat "fuzzy". Lotfi A. Zadeh, known as "the father of fuzzy logic", claimed that "vagueness connotes insufficient specificity, whereas fuzziness connotes unsharpness of class boundaries". Not all scholars agree. With increasing academic literature on the subject, the term "fuzzy concept" is now more widely recognized as a philosophical, linguistic or scientific category, and the study of the characteristics of fuzzy concepts and fuzzy language is known as fuzzy semantics. “Fuzzy logic” has become a generic term for many different kinds of many-valued logics, and is applied in many different areas of research, computer programming and industrial design. For engineers, "Fuzziness is imprecision or vagueness of definition." For computer scientists, a fuzzy concept is an idea which is "to an extent applicable" in a situation. It means that the concept can have gradations of significance or unsharp (variable) boundaries of application — a "fuzzy statement" is a statement which is true "to some extent", and that extent can often be represented by a scaled value (a score). For mathematicians, a "fuzzy concept" is usually a fuzzy set or a combination of such sets (see fuzzy mathematics and fuzzy set theory). In cognitive linguistics, the things that belong to a "fuzzy category" exhibit gradations of family resemblance, and the borders of the category are not clearly defined. Through most of the 20th century, the idea of reasoning with fuzzy concepts faced considerable resistance from Western academic elites. They did not want to endorse the use of imprecise concepts in research or argumentation, and they often regarded fuzzy logic with suspicion, derision or even hostility. That may partly explain why the idea of a "fuzzy concept" did not get a separate entry in encyclopedias, handbooks and dictionaries. Yet although people might not be aware of it, the use of fuzzy concepts has risen gigantically in all walks of life from the 1970s onward. That is mainly due to advances in electronic engineering, fuzzy mathematics and digital computer programming. The new technology allows very complex inferences about "variations on a theme" to be anticipated and fixed in a program. The Perseverance Mars rover, a driverless NASA vehicle used to explore the Jezero crater on the planet Mars, features fuzzy logic programming that steers it through rough terrain. Similarly, to the North, the Chinese Mars rover Zhurong used fuzzy logic algorithms to calculate its travel route in Utopia Planitia from sensor data. New neuro-fuzzy computational methods make it possible for machines to identify, measure, adjust and respond to fine gradations of significance with great precision. It means that practically useful concepts can be coded, sharply defined, and applied to all kinds of tasks, even if ordinarily these concepts are never exactly defined. Nowadays engineers, statisticians and programmers often represent fuzzy concepts mathematically, using fuzzy logic, fuzzy values, fuzzy variables and fuzzy sets (see also fuzzy set theory). Fuzzy logic is not "woolly thinking", but a "precise logic of imprecision" which reasons with graded concepts and gradations of truth. Fuzzy concepts and fuzzy logic often play a significant role in artificial intelligence programming, for example because they can model human cognitive processes more easily than other methods. == Origins == Vagueness and fuzziness have probably always been a part of human experience. In the West, ancient texts show that philosophers and scientists were already thinking critically about this in classical antiquity. Most often, they regarded vagueness as a problem: as an obstacle to clear thinking, as a source of confusion, or as an evasive tactic. It got in the way of providing clear orientation, guidance, direction and leadership. Therefore, vagueness became associated with a hermeneutic of suspicion — it was considered as something to avoid, as something undesirable. By contrast, in the ancient Chinese tradition of Daoist thought of Laozi and Zhuang Zhou, "vagueness is not regarded with suspicion, but is simply an acknowledged characteristic of the world around us" — a subject for meditation and a source of insight. === Sorites paradox === The ancient Sorites paradox raised the logical problem, of how we could exactly define the threshold at which a change in quantitative gradation turns into a qualitative or categorical difference. With some physical processes, this threshold seems relatively easy to identify. For example, water turns into steam at 100 °C or 212 °F. Of course, the boiling point depends partly on atmospheric pressure, which decreases at higher altitudes; it is also affected by the level of humidity — in that sense, the boiling point is "somewhat fuzzy", because it can vary under different conditions. Nevertheless, for every altitude, level of air pressure and degree of humidity, we can predict accurately what the boiling point will be, if we know the relevant conditions. With many other processes and gradations, however, the point of change is much more difficult to locate, and remains somewhat vague. Thus, the boundaries between qualitatively different things may be unsharp: we know that there are boundaries, but we cannot define them exactly. For example, to identify "the oldest city in the world", we have to define what counts as a city, and at what point a growing human settlement becomes a city. === The continuum fallacy and Loki's wager === According to the modern idea of the continuum fallacy, the fact that a statement is to an extent vague, does not automatically mean that it has no validity. The question then arises, of how (by what method or approach) we could ascertain and define the validity that the fuzzy statement does have. The Nordic myth of Loki's wager suggested that concepts that lack precise meanings or lack precise boundaries of application cannot be operated with, because they evade any clear definition. However, the 20th-century idea of "fuzzy concepts" proposes that "somewhat vague terms" can be operated with, because we can explicate and define the variability of their application — by assigning numbers to gradations of applicability. This idea sounds simple enough, but it had large implications. === Precursors and pioneers === In Western civilization, the intellectual recognition of fuzzy concepts has been traced back to a diversity of famous and less well-known thinkers, including (among many others) Eubulides, Epicurus, Plato, Cicero, William Ockham and John Buridan, Georg Wilhelm Friedrich Hegel, Karl Marx and Friedrich Engels, Friedrich Nietzsche, William James, Hugh MacColl, Charles S. Peirce, Hans Reichenbach, Carl Gustav Hempel, Max Black, Arto Salomaa, Ludwig Wittgenstein, Jan Łukasiewicz, Emil Leon Post, Alfred Tarski, Georg Cantor, Nicolai A. Vasiliev, Kurt Gödel, Stanisław Jaśkowski, Willard Van Orman Quine, George J. Klir, Petr Hájek, Joseph Goguen, Ronald R. Yager, Enrique Héctor Ruspini, Jan Pavelka, Didier Dubois, Bernadette Bouchon-Meunier, and Donald Knuth. Across at least two and a half millennia, all of them had something to say about graded concepts with unsharp boundaries. This suggests at least that the awareness of the existence of concepts with "fuzzy" characteristics, in one form or another, has a very long history in human thought. Quite a few 20th century logicians, mathematicians and philosophers also tried to analyze the characteristics of fuzzy concepts as a recognized species, sometimes with the aid of some kind of many-valued logic or substructural logic. An early attempt in the post-WW2 era to create a mathematical theory of sets with gradations of