Trying to pick the best AI video generator? An AI video generator is software that uses machine learning to help you get more done — it scales effortlessly from a single task to thousands. The best picks balance beginner-friendly simplicity with the depth power users need, and they ship updates often. Whether you are a beginner or a pro, the right AI video generator slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.
Cross-entropy method
The cross-entropy (CE) method is a Monte Carlo method for importance sampling and optimization. It is applicable to both combinatorial and continuous problems, with either a static or noisy objective. The method approximates the optimal importance sampling estimator by repeating two phases: Draw a sample from a probability distribution. Minimize the cross-entropy between this distribution and a target distribution to produce a better sample in the next iteration. Reuven Rubinstein developed the method in the context of rare-event simulation, where tiny probabilities must be estimated, for example in network reliability analysis, queueing models, or performance analysis of telecommunication systems. The method has also been applied to the traveling salesman, quadratic assignment, DNA sequence alignment, max-cut and buffer allocation problems. == Estimation via importance sampling == Consider the general problem of estimating the quantity ℓ = E u [ H ( X ) ] = ∫ H ( x ) f ( x ; u ) d x {\displaystyle \ell =\mathbb {E} _{\mathbf {u} }[H(\mathbf {X} )]=\int H(\mathbf {x} )\,f(\mathbf {x} ;\mathbf {u} )\,{\textrm {d}}\mathbf {x} } , where H {\displaystyle H} is some performance function and f ( x ; u ) {\displaystyle f(\mathbf {x} ;\mathbf {u} )} is a member of some parametric family of distributions. Using importance sampling this quantity can be estimated as ℓ ^ = 1 N ∑ i = 1 N H ( X i ) f ( X i ; u ) g ( X i ) {\displaystyle {\hat {\ell }}={\frac {1}{N}}\sum _{i=1}^{N}H(\mathbf {X} _{i}){\frac {f(\mathbf {X} _{i};\mathbf {u} )}{g(\mathbf {X} _{i})}}} , where X 1 , … , X N {\displaystyle \mathbf {X} _{1},\dots ,\mathbf {X} _{N}} is a random sample from g {\displaystyle g\,} . For positive H {\displaystyle H} , the theoretically optimal importance sampling density (PDF) is given by g ∗ ( x ) = H ( x ) f ( x ; u ) / ℓ {\displaystyle g^{}(\mathbf {x} )=H(\mathbf {x} )f(\mathbf {x} ;\mathbf {u} )/\ell } . This, however, depends on the unknown ℓ {\displaystyle \ell } . The CE method aims to approximate the optimal PDF by adaptively selecting members of the parametric family that are closest (in the Kullback–Leibler sense) to the optimal PDF g ∗ {\displaystyle g^{}} . == Generic CE algorithm == Choose initial parameter vector v ( 0 ) {\displaystyle \mathbf {v} ^{(0)}} ; set t = 1. Generate a random sample X 1 , … , X N {\displaystyle \mathbf {X} _{1},\dots ,\mathbf {X} _{N}} from f ( ⋅ ; v ( t − 1 ) ) {\displaystyle f(\cdot ;\mathbf {v} ^{(t-1)})} Solve for v ( t ) {\displaystyle \mathbf {v} ^{(t)}} , where v ( t ) = argmax v 1 N ∑ i = 1 N H ( X i ) f ( X i ; u ) f ( X i ; v ( t − 1 ) ) log f ( X i ; v ) {\displaystyle \mathbf {v} ^{(t)}=\mathop {\textrm {argmax}} _{\mathbf {v} }{\frac {1}{N}}\sum _{i=1}^{N}H(\mathbf {X} _{i}){\frac {f(\mathbf {X} _{i};\mathbf {u} )}{f(\mathbf {X} _{i};\mathbf {v} ^{(t-1)})}}\log f(\mathbf {X} _{i};\mathbf {v} )} If convergence is reached then stop; otherwise, increase t by 1 and reiterate from step 2. In several cases, the solution to step 3 can be found analytically. Situations in which this occurs are When f {\displaystyle f\,} belongs to the natural exponential family When f {\displaystyle f\,} is discrete with finite support When H ( X ) = I { x ∈ A } {\displaystyle H(\mathbf {X} )=\mathrm {I} _{\{\mathbf {x} \in A\}}} and f ( X i ; u ) = f ( X i ; v ( t − 1 ) ) {\displaystyle f(\mathbf {X} _{i};\mathbf {u} )=f(\mathbf {X} _{i};\mathbf {v} ^{(t-1)})} , then v ( t ) {\displaystyle \mathbf {v} ^{(t)}} corresponds to the maximum likelihood estimator based on those X k ∈ A {\displaystyle \mathbf {X} _{k}\in A} . == Continuous optimization—example == The same CE algorithm can be used for optimization, rather than estimation. Suppose the problem is to maximize some function S {\displaystyle S} , for example, S ( x ) = e − ( x − 2 ) 2 + 0.8 e − ( x + 2 ) 2 {\displaystyle S(x)={\textrm {e}}^{-(x-2)^{2}}+0.8\,{\textrm {e}}^{-(x+2)^{2}}} . To apply CE, one considers first the associated stochastic problem of estimating P θ ( S ( X ) ≥ γ ) {\displaystyle \mathbb {P} _{\boldsymbol {\theta }}(S(X)\geq \gamma )} for a given level γ {\displaystyle \gamma \,} , and parametric family { f ( ⋅ ; θ ) } {\displaystyle \left\{f(\cdot ;{\boldsymbol {\theta }})\right\}} , for example the 1-dimensional Gaussian distribution, parameterized by its mean μ t {\displaystyle \mu _{t}\,} and variance σ t 2 {\displaystyle \sigma _{t}^{2}} (so θ = ( μ , σ 2 ) {\displaystyle {\boldsymbol {\theta }}=(\mu ,\sigma ^{2})} here). Hence, for a given γ {\displaystyle \gamma \,} , the goal is to find θ {\displaystyle {\boldsymbol {\theta }}} so that D K L ( I { S ( x ) ≥ γ } ‖ f θ ) {\displaystyle D_{\mathrm {KL} }({\textrm {I}}_{\{S(x)\geq \gamma \}}\|f_{\boldsymbol {\theta }})} is minimized. This is done by solving the sample version (stochastic counterpart) of the KL divergence minimization problem, as in step 3 above. It turns out that parameters that minimize the stochastic counterpart for this choice of target distribution and parametric family are the sample mean and sample variance corresponding to the elite samples, which are those samples that have objective function value ≥ γ {\displaystyle \geq \gamma } . The worst of the elite samples is then used as the level parameter for the next iteration. This yields the following randomized algorithm that happens to coincide with the so-called Estimation of Multivariate Normal Algorithm (EMNA), an estimation of distribution algorithm. === Pseudocode === // Initialize parameters μ := −6 σ2 := 100 t := 0 maxits := 100 N := 100 Ne := 10 // While maxits not exceeded and not converged while t < maxits and σ2 > ε do // Obtain N samples from current sampling distribution X := SampleGaussian(μ, σ2, N) // Evaluate objective function at sampled points S := exp(−(X − 2) ^ 2) + 0.8 exp(−(X + 2) ^ 2) // Sort X by objective function values in descending order X := sort(X, S) // Update parameters of sampling distribution via elite samples μ := mean(X(1:Ne)) σ2 := var(X(1:Ne)) t := t + 1 // Return mean of final sampling distribution as solution return μ == Related methods == Simulated annealing Genetic algorithms Harmony search Estimation of distribution algorithm Tabu search Natural Evolution Strategy Ant colony optimization algorithms
R.U.R.
R.U.R. is a 1920 science fiction play by the Czech writer Karel Čapek. "R.U.R." stands for Rossumovi Univerzální Roboti (Rossum's Universal Robots, a phrase that has been used as a subtitle in English versions). The play had its world premiere on 2 January 1921 in Hradec Králové. It introduced the word "robot" to the English language and to science fiction as a whole. R.U.R. became influential soon after its publication. By 1923, it had been translated into thirty languages. R.U.R. was successful in its time in Europe and North America. Čapek later took a different approach to the same theme in his 1936 novel War with the Newts, in which non-humans become a servant-class in human society. == Characters == Parentheses indicate names which vary according to translation. On the meaning of the names, see Ivan Klíma: Karel Čapek: Life and Work (2002). == Plot == === Synopsis === The play begins in a factory that makes artificial workers from synthetic organic matter. (As living creatures of artificial flesh and blood, that later terminology would call androids, the playwright's 'roboti' differ from later fictional and scientific concepts of inorganic constructs.) Robots may be mistaken for humans but have no original thoughts. Though most are content to work for humans, eventually a rebellion causes the extinction of the human race. === Prologue (Act I in the Selver translation) === Helena, the daughter of the president of a major industrial power, arrives at the island factory of Rossum's Universal Robots. Here, she meets Domin, the General Manager of R.U.R., who relates to her the history of the company. Rossum had come to the island in 1920 to study marine biology. In 1932, Rossum had invented a substance like organic matter, though with a different chemical composition. He argued with his nephew about their motivations for creating artificial life. While the elder wanted to create animals to prove or disprove the existence of God, his nephew only wanted to become rich. Young Rossum finally locked away his uncle in a lab to play with the monstrosities he had created and created thousands of robots. By the time the play takes place (circa the year 2000), robots are cheap and available all over the world. They have become essential for industry. After meeting the heads of R.U.R., Helena reveals that she is a representative of the League of Humanity, an organization that wishes to liberate the robots. The managers of the factory find this absurd. They see robots as appliances. Helena asks that the robots be paid, but according to R.U.R. management, the robots do not "like" anything. Eventually Helena is convinced that the League of Humanity is a waste of money, but still argues robots have a "soul". Later, Domin confesses that he loves Helena and forces her into an engagement. === Act I (Act II in Selver) === Ten years have passed. Helena and her nurse Nana discuss current events, the decline in human births in particular. Helena and Domin reminisce about the day they met and summarize the last ten years of world history, which has been shaped by the new worldwide robot-based economy. Helena meets Dr. Gall's new experiment, Radius. Dr. Gall describes his experimental robotess, also named Helena. Both are more advanced, fully-featured robots. In secret, Helena burns the formula required to create robots. The revolt of the robots reaches Rossum's island as the act ends. === Act II (Act III in Selver) === The characters sense that the very universality of the robots presents a danger. Echoing the story of the Tower of Babel, the characters discuss whether creating national robots who were unable to communicate beyond their languages would have been a good idea. As robot forces lay siege to the factory, Helena reveals she has burned the formula necessary to make new robots. The characters lament the end of humanity and defend their actions, despite the fact that their imminent deaths are a direct result of their choices. Busman is killed while attempting to negotiate a peace with the robots. The robots storm the factory and kill all the humans except for Alquist, the company's Clerk of the Works (Head of Construction). The robots spare him because they recognize that "He works with his hands like a robot. He builds houses. He can work." === Act III (Epilogue in Selver) === Years have passed. Alquist, who still lives, attempts to recreate the formula that Helena destroyed. He is a mechanical engineer, though, with insufficient knowledge of biochemistry, so he has made little progress. The robot government has searched for surviving humans to help Alquist and found none alive. Officials from the robot government beg him to complete the formula, even if it means he will have to kill and dissect other robots for it. Alquist yields. He will kill and dissect robots, thus completing the circle of violence begun in Act Two. Alquist is disgusted. Robot Primus and Helena develop human feelings and fall in love. Playing a hunch, Alquist threatens to dissect Primus and then Helena; each begs him to take him- or herself and spare the other. Alquist now realizes that Primus and Helena are the new Adam and Eve, and gives the charge of the world to them. == Čapek's conception of robots == The robots described in Čapek's play are not robots in the popularly understood sense of an automaton. They are not mechanical devices, but rather artificial biological organisms that may be mistaken for humans. A comic scene at the beginning of the play shows Helena arguing with her future husband, Harry Domin, because she cannot believe his secretary is a robotess: His robots resemble more modern conceptions of man-made life forms, such as the Replicants in Blade Runner, the "hosts" in the Westworld TV series and the humanoid Cylons in the re-imagined Battlestar Galactica, but in Čapek's time there was no conception of modern genetic engineering (DNA's role in heredity was not confirmed until 1952). There are descriptions of kneading-troughs for robot skin, great vats for liver and brains, and a factory for producing bones. Nerve fibers, arteries, and intestines are spun on factory bobbins, while the robots themselves are assembled like automobiles. Čapek's robots are living biological beings, but they are still assembled, as opposed to grown or born. One critic has described Čapek's robots as epitomizing "the traumatic transformation of modern society by the First World War and the Fordist assembly line". === Origin of the word robot === The play introduced the word robot, which displaced older words such as "automaton" or "android" in languages around the world. In an article in Lidové noviny, Karel Čapek named his brother Josef as the true inventor of the word. In Czech, robota means forced labour of the kind that serfs had to perform on their masters' lands and is derived from rab, meaning "slave". The name Rossum is an allusion to the Czech word rozum, meaning "reason", "wisdom", "intellect" or "common sense". It has been suggested that the allusion might be preserved by translating "Rossum" as "Reason" but only the Majer/Porter version translates the word as "Reason". == Production history and translations == The work was published in two differing versions in Prague by Aventinum, first in 1920, followed by a revised version in 1921. After being postponed, it premiered at the city's National Theatre on 25 January 1921, although an amateur group had by then already presented a production. By 1921, Paul Selver translated either the original 1920 edition of R.U.R. or a manuscript copy close to this version into English. He probably translated the play freelance, and sold it to St Martin's Theatre in London. Selver's translation was adapted for the British stage by Nigel Playfair in 1922, but it was not produced straight away. Later that year performance rights for the U.S. and Canada were sold to the New York Theatre Guild, perhaps during Lawrence Langner's visit to Britain. Playfair's version included several changes to Čapek's original play, such as renaming the acts (the prologue became act one, and the heavily abridged final act became the epilogue), omitting around sixty lines (including most of Alquist's final speech), adding several more lines, and removing the robot character Damon (giving his lines to Radius). The omission of some lines may have been censorship from the Lord Chamberlain's Office, or self-censorship in anticipation of this, while some other changes might have been made by Čapek himself if Selver was working from a manuscript copy. An edition of Playfair's adaptation was published by the Oxford University Press in 1923, and Selver went on to write a satiric novel One, Two, Three (1926) based on his experiences getting R.U.R. staged. The American première was produced by the Theatre Guild at the Garrick Theatre in New York City in October 1922, where it ran for 184 performances. In the first performance, Domin was portrayed by Basil Sydney,
Tales from the Loop (role-playing game)
Tales from the Loop (Swedish: Ur Varselklotet), subtitled "Roleplaying in the '80s That Never Was", is an alternative history science fiction tabletop role-playing game published in 2017 by Free League Publishing, the international arm of Swedish game and book publisher Fria Ligan AB, and Modiphius Entertainment. The game, based on the art of Simon Stålenhag, envisions an alternative world where a group of bored and ignored preteens and teens solve mysteries caused by new technology near their hometown. == Description == === Setting === Tales from the Loop is set in an alternative history world taken from the artwork of Simon Stålenhag. According to this alternative timeline, back in the 1940s, research began on particle accelerators. In the 1960s, two massive underground particle accelerators were built in Sweden and Colorado with the promise of a harvest of technological marvels that would change everyone's lives. Tales from the Loop is set twenty years later, in the late 1980s, and the better life has not materialized. Although the particle accelerators have created robots and large skyships, the detritus of failed experiments and the ruins of abandoned high tech company buildings litter the landscape. Generally the life of the average family has not changed for the better. A campaign can either be set in the Mälaren Islands, west of the Swedish capital of Stockholm, or in a city in the Southwest United States that resembles Boulder City, Nevada. There is also a step-by-step guide for the gamemaster to use their own hometown. === Character generation === Player characters are preteens and young teenagers age 10–15 who live in a society where they are bored and largely left to themselves. Players can choose archetypes for their characters including Bookworm, Jock, Troublemaker, Popular Kid and Weirdo. Unlike most role-playing games, characters in Tales from the Loop cannot be killed, although in an ongoing campaign or due to an in-game effect, they are removed from the game if they reach the age of sixteen. === Game system === The game uses the Year Zero Engine first developed by Tomas Härenstam for the post-apocalyptic role-playing game Mutant: Year Zero. (Härenstam served as the editor and project manager for Tales from the Loop.) Problems are resolved by rolling a pool of six-sided dice, with any 6 rolled marking success. Attributes and skills (Sneak, Force, Move, Build, Tinker, Calculate, Contact, Charm, Lead, Investigate, Comprehend, and Empathize) may allow the player to add more dice to the dice pool, increasing the chances of success. However, if a character has earned a condition such as Scared or Injured, dice are removed from the dice pool. === Gameplay === The game principles are that life for the characters is dull and boring, but the area around the town is full of wonderful, mysterious things. An adventure is set up as a Mystery, and in order to successfully resolve the Mystery, characters must overcome a series of Troubles, which can range from having to be home by a certain time to dealing with a bully to disarming or otherwise overcoming a booby-trap on a door that must be opened. Each Mystery is played as a series of scenes, much like a TV drama. Although the gamemaster leads the players into the Mystery, each scene is set collaboratively with the players before action continues. As critic Jukka Kauppinen noted, "The players and the gamemaster take turns verbally staging a new scene — where we are, what it's like there — and only then what we do." === Campaign === The book presents a chronologically-linked set of four Mysteries called "The Four Seasons of Mad Science" that take place over a calendar year: "Summer Break and Killer Birds": The Kids hears pigeons having a conversation and investigate "Grown-Up Attraction": Adults start disappearing without any sign of struggle. "Creatures from the Cretaceous": The search for a missing dog leads to the discovery of creatures that don't belong in our time "I, Wagner": The Kids discover a body in a stream, and are drawn into a Mystery with robots and humans that may affect them closely. == Publication history == In 2017, Swedish artist Simon Stålenhag was raising money on Kickstarter to publish a book of his art titled Tales from the Loop. One of the stretch goals offered was the creation of a role-playing game. A second Kickstarter campaign to publish the role-playing game was initiated by Fria Ligan AB, who surpassed their crowdfunding goal and raised a total of 3,745,896 kr from 5,600 backers. The role-playing game Tales from the Loop was subsequently published as a 184-page hardcover book in 2017 by Free League Publishing, the international arm of Swedish game and book publisher Fria Ligan AB, and Modiphius Entertainment. Cover art and interior art were by Stålenhag, and cartography was by Christian Granath. A stand-alone expansion, Things from the Flood (Swedish: Flodskörden), based on Stålenhag's art book of the same name, was created by Nils Hintze, Rickard Antroia, and Tomas Härenstam. The 216-page hardcover book was published in 2019 with cover art by Stålenhag, interior art by Stålenhag and Reine Rosenberg, and cartography by Christian Granath. In 2020, the setting of the role-playing game was transferred to the TV series Tales from the Loop developed by Nathanial Halpern and Simon Stålenhag. The series tells eight stories of children's encounters with strange technology. == Reception == Shut Up & Sit Down praised Tales from the Loop for its comfortable, contemporary setting, simple rules that make the game easy to run, and the alternation between sci-fi and the kids' lives, but criticized the Type system for characters, noting "a suggested 'Pride' for the Weirdo involved being homosexual –– the only mention of queerness in the entire game. Those of us who identify as GLBTQ bristled at that: why was only the Weirdo queer, with queerness as a (possibly secret) Pride? Why not more fully address being a GLBTQ kid in the 1980s?" The review concluded, "For new RPG players, Tales is a decent game that you'll enjoy and that will make your heart burst. But you need an experienced GM who’s able to either alter the book’s mysteries or create their own, and who can put in work when poor dice rolls hold the players back." Rob Weiland of Geek & Sundry named Tales from the Loop 2017's best RPG release and praised Stålenhag's art, the collaborative nature between the GM and players, and the simplicity of running the game. Weiland concluded, "It has a simple system that is easy to explain but holds up under several plays. It has a setting that’s immediately evocative but also leaves plenty of room for GMs to build out their own world. It offers players a chance to experience the rush of memory, the pain of childhood and the wonder of movies." In a review of Tales from the Loop in Black Gate, Andrew Zimmerman Jones said, "Though not based directly on an established franchise, it draws richly from elements of popular culture that will make it resonate with many players. The focus on narrative play also means it’s a good game for people who aren’t necessarily big into learning a ton of new rules." Jukka Kauppinen, writing for the Finnish games magazine Skrolli, called the game, "downright delicious in its diversity. The science fiction world created by the Swedish artist Simon Stälenhag is, after all, both delightful vintage and tickling novelty." Kauppinen concluded, "This mutual storytelling and interaction makes this game more of a campfire circle than a traditional role-playing game. At the same time, its setting in the real world, tinged with science fiction and even horror, creates a delicious and unique adventure environment." In his 2023 book Monsters, Aliens, and Holes in the Ground, RPG historian Stu Horvath noted that the game system "pushes the players to constantly reevaluate their characters' relationships with the everyday world, for better or worse. It won't be long before navigating entanglements with parents, teachers, siblings and bullies proves just as risky to the characters, and central to the players' experience, as trying to find out what happened with the time portal or dealing with a rampaging robot." Horvath concluded, "The appeal of Tales from the Loop is Stålenhag's deep shadows and purple dusks. They hide the dangers and mysteries that often act [as] an escape hatch, a way to avoid prosaic problems." == Awards == At the 2017 Golden Geek Awards, Tales of the Loop won "RPG of the Year", and was a finalist for " Best RPG Artwork/Presentation" At the 2017 ENnie Awards, Tales from the Loops won five Gold Medals: Product of the Year Best Writing Best Setting Best Game Best Art, Interior
Herbrand Award
The Herbrand Award for Distinguished Contributions to Automated Reasoning is an award given by the Conference on Automated Deduction (CADE), Inc., (although it predates the formal incorporation of CADE) to honour persons or groups for important contributions to the field of automated deduction. The award is named after the French scientist Jacques Herbrand and given at most once per CADE or International Joint Conference on Automated Reasoning (IJCAR). It comes with a prize of US$1,000. Anyone can be nominated, the award is awarded after a vote among CADE trustees and former recipients, usually with input from the CADE/IJCAR programme committee. == Recipients == Past award recipients are: === 1990s === Larry Wos (1992) Woody Bledsoe (1994) John Alan Robinson (1996) Wu Wenjun (1997) Gérard Huet (1998) Robert S. Boyer and J Strother Moore (1999) === 2000s === William W. McCune (2000) Donald W. Loveland (2001) Mark E. Stickel (2002). Peter B. Andrews (2003) Harald Ganzinger (2004) Martin Davis (2005) Wolfgang Bibel (2006) Alan Bundy (2007) Edmund M. Clarke (2008) Deepak Kapur (2009) === 2010s === David Plaisted (2010) Nachum Dershowitz (2011) Melvin Fitting (2012) C. Greg Nelson (2013) Robert L. Constable (2014) Andrei Voronkov (2015) Zohar Manna and Richard Waldinger (2016) Lawrence C. Paulson (2017) Bruno Buchberger (2018) Nikolaj Bjørner and Leonardo de Moura (2019) === 2020s === Franz Baader (2020) Tobias Nipkow (2021) Natarajan Shankar (2022) Moshe Vardi (2023) Armin Biere (2024) Aart Middeldorp (2025)
Highway network
In machine learning, the Highway Network was the first working very deep feedforward neural network with hundreds of layers, much deeper than previous neural networks. It uses skip connections modulated by learned gating mechanisms to regulate information flow, inspired by long short-term memory (LSTM) recurrent neural networks. The advantage of the Highway Network over other deep learning architectures is its ability to overcome or partially prevent the vanishing gradient problem, thus improving its optimization. Gating mechanisms are used to facilitate information flow across the many layers ("information highways"). Highway Networks have found use in text sequence labeling and speech recognition tasks. In 2014, the state of the art was training deep neural networks with 20 to 30 layers. Stacking too many layers led to a steep reduction in training accuracy, known as the "degradation" problem. In 2015, two techniques were developed to train such networks: the Highway Network (published in May), and the residual neural network, or ResNet (December). ResNet behaves like an open-gated Highway Net. == Model == The model has two gates in addition to the H ( W H , x ) {\displaystyle H(W_{H},x)} gate: the transform gate T ( W T , x ) {\displaystyle T(W_{T},x)} and the carry gate C ( W C , x ) {\displaystyle C(W_{C},x)} . The latter two gates are non-linear transfer functions (specifically sigmoid by convention). The function H {\displaystyle H} can be any desired transfer function. The carry gate is defined as: C ( W C , x ) = 1 − T ( W T , x ) {\displaystyle C(W_{C},x)=1-T(W_{T},x)} while the transform gate is just a gate with a sigmoid transfer function. == Structure == The structure of a hidden layer in the Highway Network follows the equation: y = H ( x , W H ) ⋅ T ( x , W T ) + x ⋅ C ( x , W C ) = H ( x , W H ) ⋅ T ( x , W T ) + x ⋅ ( 1 − T ( x , W T ) ) {\displaystyle {\begin{aligned}y=H(x,W_{H})\cdot T(x,W_{T})+x\cdot C(x,W_{C})\\=H(x,W_{H})\cdot T(x,W_{T})+x\cdot (1-T(x,W_{T}))\end{aligned}}} == Related work == Sepp Hochreiter analyzed the vanishing gradient problem in 1991 and attributed to it the reason why deep learning did not work well. To overcome this problem, Long Short-Term Memory (LSTM) recurrent neural networks have residual connections with a weight of 1.0 in every LSTM cell (called the constant error carrousel) to compute y t + 1 = F ( x t ) + x t {\textstyle y_{t+1}=F(x_{t})+x_{t}} . During backpropagation through time, this becomes the residual formula y = F ( x ) + x {\textstyle y=F(x)+x} for feedforward neural networks. This enables training very deep recurrent neural networks with a very long time span t. A later LSTM version published in 2000 modulates the identity LSTM connections by so-called "forget gates" such that their weights are not fixed to 1.0 but can be learned. In experiments, the forget gates were initialized with positive bias weights, thus being opened, addressing the vanishing gradient problem. As long as the forget gates of the 2000 LSTM are open, it behaves like the 1997 LSTM. The Highway Network of May 2015 applies these principles to feedforward neural networks. It was reported to be "the first very deep feedforward network with hundreds of layers". It is like a 2000 LSTM with forget gates unfolded in time, while the later Residual Nets have no equivalent of forget gates and are like the unfolded original 1997 LSTM. If the skip connections in Highway Networks are "without gates," or if their gates are kept open (activation 1.0), they become Residual Networks. The residual connection is a special case of the "short-cut connection" or "skip connection" by Rosenblatt (1961) and Lang & Witbrock (1988) which has the form x ↦ F ( x ) + A x {\displaystyle x\mapsto F(x)+Ax} . Here the randomly initialized weight matrix A does not have to be the identity mapping. Every residual connection is a skip connection, but almost all skip connections are not residual connections. The original Highway Network paper not only introduced the basic principle for very deep feedforward networks, but also included experimental results with 20, 50, and 100 layers networks, and mentioned ongoing experiments with up to 900 layers. Networks with 50 or 100 layers had lower training error than their plain network counterparts, but no lower training error than their 20 layers counterpart (on the MNIST dataset, Figure 1 in ). No improvement on test accuracy was reported with networks deeper than 19 layers (on the CIFAR-10 dataset; Table 1 in ). The ResNet paper, however, provided strong experimental evidence of the benefits of going deeper than 20 layers. It argued that the identity mapping without modulation is crucial and mentioned that modulation in the skip connection can still lead to vanishing signals in forward and backward propagation (Section 3 in ). This is also why the forget gates of the 2000 LSTM were initially opened through positive bias weights: as long as the gates are open, it behaves like the 1997 LSTM. Similarly, a Highway Net whose gates are opened through strongly positive bias weights behaves like a ResNet. The skip connections used in modern neural networks (e.g., Transformers) are dominantly identity mappings.
AI-assisted targeting in the Gaza Strip
As part of the Gaza war, the Israel Defense Forces (IDF) have used artificial intelligence to rapidly and automatically perform much of the process of determining what to bomb. Israel has greatly expanded the bombing of the Gaza Strip, which in previous wars had been limited by the Israeli Air Force running out of targets. These tools include the Gospel, an AI which automatically reviews surveillance data looking for buildings, equipment and people thought to belong to the enemy, and upon finding them, recommends bombing targets to a human analyst who may then decide whether to pass it along to the field. Another is Lavender, an "AI-powered database" which lists tens of thousands of Palestinian men linked by AI to Hamas or Palestinian Islamic Jihad, and which is also used for target recommendation. Critics have argued the use of these AI tools puts civilians at risk, blurs accountability, and results in militarily disproportionate violence in violation of international humanitarian law. == The Gospel == Israel uses an AI system dubbed "Habsora", "the Gospel", to determine which targets the Israeli Air Force would bomb. It automatically provides a targeting recommendation to a human analyst, who decides whether to pass it along to soldiers in the field. The recommendations can be anything from individual fighters, rocket launchers, Hamas command posts, to private homes of suspected Hamas or Islamic Jihad members. AI can process military intelligence far faster than humans. Retired Lt Gen. Aviv Kohavi, head of the IDF until 2023, stated that the system could produce 100 bombing targets in Gaza a day, with real-time recommendations which ones to attack, where human analysts might produce 50 a year. A lecturer interviewed by NPR estimated these figures as 50–100 targets in 300 days for 20 intelligence officers, and 200 targets within 10–12 days for the Gospel. === Technological background === The Gospel uses machine learning, where an AI is tasked with identifying commonalities in vast amounts of data (e.g. scans of cancerous tissue, photos of a facial expression, surveillance of Hamas members identified by human analysts), then looking for those commonalities in new material. What information the Gospel uses is not known, but it is thought to combine surveillance data from diverse sources in enormous amounts. Recommendations are based on pattern-matching. A person with enough similarities to other people labeled as enemy combatants may be labelled a combatant themselves. Regarding the suitability of AIs for the task, NPR cited Heidy Khlaaf, engineering director of AI Assurance at the technology security firm Trail of Bits, as saying "AI algorithms are notoriously flawed with high error rates observed across applications that require precision, accuracy, and safety." Bianca Baggiarini, lecturer at the Australian National University's Strategic and Defence Studies Centre wrote AIs are "more effective in predictable environments where concepts are objective, reasonably stable, and internally consistent." She contrasted this with telling the difference between a combatant and non-combatant, which even humans frequently can't do. Khlaaf went on to point out that such a system's decisions depend entirely on the data it's trained on, and are not based on reasoning, factual evidence or causation, but solely on statistical probability. === Operation === The IAF ran out of targets to strike in the 2014 war and 2021 crisis. In an interview on France 24, investigative journalist Yuval Abraham of +972 Magazine stated that to maintain military pressure, and due to political pressure to continue the war, the military would bomb the same places twice. Since then, the integration of AI tools has significantly sped up the selection of targets. In early November, the IDF stated more than 12,000 targets in Gaza had been identified by the target administration division that uses the Gospel. NPR wrote on December 14 that it was unclear how many targets from the Gospel had been acted upon, but that the Israeli military said it was currently striking as many as 250 targets a day. The bombing, too, has intensified to what the December 14 article called an astonishing pace: the Israeli military stated at the time it had struck more than 22,000 targets inside Gaza, at a daily rate more than double that of the 2021 conflict, more than 3,500 of them since the collapse of the truce on December 1. Early in the offensive the head of the Air Force stated his forces only struck military targets, but added: "We are not being surgical." Once a recommendation is accepted, another AI, Fire Factory, cuts assembling the attack down from hours to minutes by calculating munition loads, prioritizing and assigning targets to aircraft and drones, and proposing a schedule, according to a pre-war Bloomberg article that described such AI tools as tailored for a military confrontation and proxy war with Iran. One change that The Guardian noted is that since senior Hamas leaders disappear into tunnels at the start of an offensive, systems such as the Gospel have allowed the IDF to locate and attack a much larger pool of more junior Hamas operatives. It cited an official who worked on targeting decisions in previous Gaza operations as saying that while the homes of junior Hamas members had previously not been targeted for bombing, the official believes the houses of suspected Hamas operatives were now targeted regardless of rank. In the France 24 interview, Abraham, of +972 Magazine, characterized this as enabling the systematization of dropping a 2000 lb bomb into a home to kill one person and everybody around them, something that had previously been done to a very small group of senior Hamas leaders. NPR cited a report by +972 Magazine and its sister publication Local Call as asserting the system is being used to manufacture targets so that Israeli military forces can continue to bombard Gaza at an enormous rate, punishing the general Palestinian population. NPR noted it had not verified this; it was unclear how many targets are being generated by AI alone, but there had been a substantial increase in targeting, with an enormous civilian toll. In principle, the combination of a computer's speed to identify opportunities and a human's judgment to evaluate them can enable more precise attacks and fewer civilian casualties. Israeli military and media have emphasized this capacity to minimize harm to non-combatants. Richard Moyes, researcher and head of the NGO Article 36, pointed to "the widespread flattening of an urban area with heavy explosive weapons" to question these claims, while Lucy Suchman, professor emeritus at Lancaster University, described the bombing as "aimed at maximum devastation of the Gaza Strip". The Guardian wrote that when a strike was authorized on private homes of those identified as Hamas or Islamic Jihad operatives, target researchers knew in advance the expected number of civilians killed, each target had a file containing a collateral damage score stipulating how many civilians were likely to be killed in a strike, and according to a senior Israeli military source, operatives use a "very accurate" measurement of the rate of civilians evacuating a building shortly before a strike. "We use an algorithm to evaluate how many civilians are remaining. It gives us a green, yellow, red, like a traffic signal." ==== 2021 use ==== Kohavi compared the target division using the Gospel to a machine and stated that once the machine was activated in the war of May 2021, it generated 100 targets a day, with half of them being attacked, in contrast with 50 targets in Gaza per year beforehand. Approximately 200 targets came from the Gospel out of the 1,500 targets Israel struck in Gaza in the war, including both static and moving targets according to the military. The Jewish Institute for National Security of America's after action report identified an issue, stating the system had data on what was a target, but lacked data on what wasn't. The system depends entirely on training data, and intel that human analysts had examined and deemed didn't constitute a target had been discarded, risking bias. The vice president expressed his hopes this had since been rectified. === Organization === The Gospel is used by the military's target administration division (or Directorate of Targets or Targeting Directorate), which was formed in 2019 in the IDF's intelligence directorate to address the air force running out of targets to bomb, and which Kohavi described as "powered by AI capabilities" and including hundreds of officers of soldiers. In addition to its wartime role, The Guardian wrote it'd helped the IDF build a database of between 30,000 and 40,000 suspected militants in recent years, and that systems such as the Gospel had played a critical role in building lists of individuals authorized to be assassinated. The Gospel was developed by Unit 8200 of the Israeli Intelligence C