The exploration–exploitation dilemma, also known as the explore–exploit tradeoff, is a fundamental concept in decision-making that arises in many domains. It is depicted as the balancing act between two opposing strategies. Exploitation involves choosing the best option based on current knowledge of the system (which may be incomplete or misleading), while exploration involves trying out new options that may lead to better outcomes in the future at the expense of an exploitation opportunity. Finding the optimal balance between these two strategies is a crucial challenge in many decision-making problems whose goal is to maximize long-term benefits. == Application in machine learning == In the context of machine learning, the exploration–exploitation tradeoff is fundamental in reinforcement learning (RL), a type of machine learning that involves training agents to make decisions based on feedback from the environment. Crucially, this feedback may be incomplete or delayed. The agent must decide whether to exploit the current best-known policy or explore new policies to improve its performance. === Multi-armed bandit methods === The multi-armed bandit (MAB) problem was a classic example of the tradeoff, and many methods were developed for it, such as epsilon-greedy, Thompson sampling, and the upper confidence bound (UCB). See the page on MAB for details. In more complex RL situations than the MAB problem, the agent can treat each choice as a MAB, where the payoff is the expected future reward. For example, if the agent performs an epsilon-greedy method, then the agent will often "pull the best lever" by picking the action that had the best predicted expected reward (exploit). However, it would pick a random action with probability epsilon (explore). Monte Carlo tree search, for example, uses a variant of the UCB method. === Exploration problems === There are some problems that make exploration difficult. Sparse reward. If rewards occur only once a long while, then the agent might not persist in exploring. Furthermore, if the space of actions is large, then the sparse reward would mean the agent would not be guided by the reward to find a good direction for deeper exploration. A standard example is Montezuma's Revenge. Deceptive reward. If some early actions give immediate small reward, but other actions give later large reward, then the agent might be lured away from exploring the other actions. Noisy TV problem. If certain observations are irreducibly noisy (such as a television showing random images), then the agent might be trapped exploring those observations (watching the television). === Exploration reward === This section based on. The exploration reward (also called exploration bonus) methods convert the exploration-exploitation dilemma into a balance of exploitations. That is, instead of trying to get the agent to balance exploration and exploitation, exploration is simply treated as another form of exploitation, and the agent simply attempts to maximize the sum of rewards from exploration and exploitation. The exploration reward can be treated as a form of intrinsic reward. We write these as r t i , r t e {\displaystyle r_{t}^{i},r_{t}^{e}} , meaning the intrinsic and extrinsic rewards at time step t {\displaystyle t} . However, exploration reward is different from exploitation in two regards: The reward of exploitation is not freely chosen, but given by the environment, but the reward of exploration may be picked freely. Indeed, there are many different ways to design r t i {\displaystyle r_{t}^{i}} described below. The reward of exploitation is usually stationary (i.e. the same action in the same state gives the same reward), but the reward of exploration is non-stationary (i.e. the same action in the same state should give less and less reward). Count-based exploration uses N n ( s ) {\displaystyle N_{n}(s)} , the number of visits to a state s {\displaystyle s} during the time-steps 1 : n {\displaystyle 1:n} , to calculate the exploration reward. This is only possible in small and discrete state space. Density-based exploration extends count-based exploration by using a density model ρ n ( s ) {\displaystyle \rho _{n}(s)} . The idea is that, if a state has been visited, then nearby states are also partly-visited. In maximum entropy exploration, the entropy of the agent's policy π {\displaystyle \pi } is included as a term in the intrinsic reward. That is, r t i = − ∑ a π ( a | s t ) ln π ( a | s t ) + ⋯ {\displaystyle r_{t}^{i}=-\sum _{a}\pi (a|s_{t})\ln \pi (a|s_{t})+\cdots } . === Prediction-based === This section based on. The forward dynamics model is a function for predicting the next state based on the current state and the current action: f : ( s t , a t ) ↦ s t + 1 {\displaystyle f:(s_{t},a_{t})\mapsto s_{t+1}} . The forward dynamics model is trained as the agent plays. The model becomes better at predicting state transition for state-action pairs that had been done many times. A forward dynamics model can define an exploration reward by r t i = ‖ f ( s t , a t ) − s t + 1 ‖ 2 2 {\displaystyle r_{t}^{i}=\|f(s_{t},a_{t})-s_{t+1}\|_{2}^{2}} . That is, the reward is the squared-error of the prediction compared to reality. This rewards the agent to perform state-action pairs that had not been done many times. This is however susceptible to the noisy TV problem. Dynamics model can be run in latent space. That is, r t i = ‖ f ( s t , a t ) − ϕ ( s t + 1 ) ‖ 2 2 {\displaystyle r_{t}^{i}=\|f(s_{t},a_{t})-\phi (s_{t+1})\|_{2}^{2}} for some featurizer ϕ {\displaystyle \phi } . The featurizer can be the identity function (i.e. ϕ ( x ) = x {\displaystyle \phi (x)=x} ), randomly generated, the encoder-half of a variational autoencoder, etc. A good featurizer improves forward dynamics exploration. The Intrinsic Curiosity Module (ICM) method trains simultaneously a forward dynamics model and a featurizer. The featurizer is trained by an inverse dynamics model, which is a function for predicting the current action based on the features of the current and the next state: g : ( ϕ ( s t ) , ϕ ( s t + 1 ) ) ↦ a t {\displaystyle g:(\phi (s_{t}),\phi (s_{t+1}))\mapsto a_{t}} . By optimizing the inverse dynamics, both the inverse dynamics model and the featurizer are improved. Then, the improved featurizer improves the forward dynamics model, which improves the exploration of the agent. Random Network Distillation (RND) method attempts to solve this problem by teacher–student distillation. Instead of a forward dynamics model, it has two models f , f ′ {\displaystyle f,f'} . The f ′ {\displaystyle f'} teacher model is fixed, and the f {\displaystyle f} student model is trained to minimize ‖ f ( s ) − f ′ ( s ) ‖ 2 2 {\displaystyle \|f(s)-f'(s)\|_{2}^{2}} on states s {\displaystyle s} . As a state is visited more and more, the student network becomes better at predicting the teacher. Meanwhile, the prediction error is also an exploration reward for the agent, and so the agent learns to perform actions that result in higher prediction error. Thus, we have a student network attempting to minimize the prediction error, while the agent attempting to maximize it, resulting in exploration. The states are normalized by subtracting a running average and dividing a running variance, which is necessary since the teacher model is frozen. The rewards are normalized by dividing with a running variance. Exploration by disagreement trains an ensemble of forward dynamics models, each on a random subset of all ( s t , a t , s t + 1 ) {\displaystyle (s_{t},a_{t},s_{t+1})} tuples. The exploration reward is the variance of the models' predictions. === Noise === For neural network–based agents, the NoisyNet method changes some of its neural network modules by noisy versions. That is, some network parameters are random variables from a probability distribution. The parameters of the distribution are themselves learnable. For example, in a linear layer y = W x + b {\displaystyle y=Wx+b} , both W , b {\displaystyle W,b} are sampled from Gaussian distributions N ( μ W , Σ W ) , N ( μ b , Σ b ) {\displaystyle {\mathcal {N}}(\mu _{W},\Sigma _{W}),{\mathcal {N}}(\mu _{b},\Sigma _{b})} at every step, and the parameters μ W , Σ W , μ b , Σ b {\displaystyle \mu _{W},\Sigma _{W},\mu _{b},\Sigma _{b}} are learned via the reparameterization trick.
DABUS
DABUS (Device for the Autonomous Bootstrapping of Unified Sentience) is an artificial intelligence (AI) system created by Stephen Thaler. It reportedly conceived of two novel products — a food container constructed using fractal geometry, which enables rapid reheating, and a flashing beacon for attracting attention in an emergency. The filing of patent applications designating DABUS as inventor has led to decisions by patent offices and courts on whether a patent can be granted for an invention reportedly made by an AI system. == History in different jurisdictions == === Australia === On 17 September 2019, Thaler filed an application to patent a "Food container and devices and methods for attracting enhanced attention," naming DABUS as the inventor. On 21 September 2020, IP Australia found that section 15(1) of the Patents Act 1990 (Cth) is inconsistent with an artificial intelligence machine being treated as an inventor, and Thaler's application had lapsed. Thaler sought judicial review, and on 30 July 2021, the Federal Court set aside IP Australia's decision and ordered IP Australia to reconsider the application. On 13 April 2022, the Full Court of the Federal Court set aside that decision, holding that only a natural person can be an inventor for the purposes of the Patents Act 1990 (Cth) and the Patents Regulations 1991 (Cth), and that such an inventor must be identified for any person to be entitled to a grant of a patent. On 11 November 2022, Thaler was refused special leave to appeal to the High Court. === European Patent Office === On 17 October 2018 and 7 November 2018, Thaler filed two European patent applications with the European Patent Office. The first claimed invention was a "Food Container" and the second was "Devices and Methods for Attracting Enhanced Attention." On 27 January 2020, the EPO rejected the applications on the grounds that the application listed an AI system named DABUS, and not a human, as the inventor, based on Article 81 and Rule 19(1) of the European Patent Convention (EPC). On 21 December 2021, the Board of Appeal of the EPO dismissed Thaler's appeal from the EPO's primary decision. The Board of Appeal confirmed that "under the EPC the designated inventor has to be a person with legal capacity. This is not merely an assumption on which the EPC was drafted. It is the ordinary meaning of the term inventor." === United Kingdom === Similar applications were filed by Thaler to the United Kingdom Intellectual Property Office on 17 October and 7 November 2018. The Office asked Thaler to file statements of inventorship and of right of grant to a patent (Patent Form 7) in respect of each invention within 16 months of the filing date. Thaler filed those forms naming DABUS as the inventor and explaining in some detail why he believed that machines should be regarded as inventors in the circumstances. His application was rejected on the grounds that: (1) naming a machine as inventor did not meet the requirements of the Patents Act 1977; and (2) the IPO was not satisfied as to the manner in which Thaler had acquired rights that would otherwise vest in the inventor. Thaler was not satisfied with the decision and asked for a hearing before an official known as the "hearing officer". By a decision dated 4 December 2019 the hearing officer rejected Thaler's appeal. Thaler appealed against the hearing officer's decision to the Patents Court (a specialist court within the Chancery Division of the High Court of England and Wales that determines patent disputes). On 21 September 2020, Mr Justice Marcus Smith upheld the decision of the hearing officer. On 21 September 2021, Thaler's further appeal to the Court of Appeal was dismissed by Arnold LJ and Laing LJ (Birss LJ dissenting). On 20 December 2023, the UK Supreme Court dismissed a further appeal by Thaler. In its judgment, the court held that an "inventor" under the Patents Act 1977 must be a natural person. === United States === The patent applications on the inventions were refused by the USPTO, which held that only natural persons can be named as inventors in a patent application. Thaler first fought this result by filing a complaint under the Administrative Procedure Act alleging that the decision was "arbitrary, capricious, an abuse of discretion and not in accordance with the law; unsupported by substantial evidence, and in excess of Defendants’ statutory authority." A month later on August 19, 2019, Thaler filed a petition with the USPTO as allowed in 37 C.F.R. § 1.181 stating that DABUS should be the inventor. The judge and Thaler agreed in this case that Thaler himself is unable to receive the patent on behalf of DABUS. In their August 5, 2022, Thaler decision, the US Court of Appeals for the Federal Circuit affirmed that only a natural person could be an inventor, which means that the AI that invents any other type of invention is not addressed by the "who" mentioned in the legislation. === New Zealand === On January 31, 2022, the Intellectual Property Office of New Zealand (IPONZ) decided that a patent application (776029) filed by Stephen Thaler was void, on the basis that no inventor was identified on the patent application. IPONZ determined that DABUS could not be "an actual devisor of the invention" as required by the Patents Act 2013, and that this must be a natural person as held by the previous patent offices above. The High Court of New Zealand confirmed the decision in 2023. === South Africa === On 24 June 2021, the South African Companies and Intellectual Property Commission (CIPC) accepted Dr Thaler's Patent Cooperation Treaty, for a patent in respect of inventions generated by DABUS. In July 2021, the CIPC released a notice of issuance for the patent. It is the first patent granted for an AI invention. === Switzerland === On June 26, 2025, the Swiss Federal Administrative Court ruled that artificial intelligence systems such as DABUS cannot be listed as inventors in patent applications. The court upheld the existing practice of the Swiss Federal Institute of Intellectual Property (IPI), which requires that only natural persons can be recognized as inventors under Swiss patent law. The case concerned a patent application, which sought to designate DABUS as the sole inventor of a food container designed with a fractal geometry to enhance heat distribution. The IPI had rejected the application, arguing that both the absence of a human inventor and the attribution of inventorship to an AI system were inadmissible. While the court dismissed Thaler's main request, it accepted a subsidiary request: if a human applicant recognizes and files a patent based on an AI-generated invention, that person may be considered the inventor. As a result, the application may proceed with Thaler listed as the inventor. The decision (B-2532/2024) can still be appealed to the Swiss Federal Supreme Court.
Fling (social network)
Fling was a social media app available for IOS and Android. It was founded in 2014 by Marco Nardone and was taken offline in August 2016. == Overview == In 2012, Marco Nardone founded the startup Unii and launched Unii.com, a social network intended for students in the UK. While working on this service, Nardone had the idea for a messaging service where pictures could be sent to strangers in January 2014. The app Fling was then developed and released between March and July 2014. After a month, it already had 375,000 downloads and 180,000 active users on iOS. Users were able to take pictures inside the app and send them to 50 random people all over the world. The recipient could then choose to answer via chat or reply by sending a picture themselves. The app was used by many users as a medium to exchange sexually explicit pictures and for sexting with strangers. This led to the app being removed from the App Store in June 2015. In the 19 days that followed, flings developers rewrote the App almost completely from scratch, working around the clock. The feature to message random strangers was removed, and the app was readmitted into the App Store as a messenger App resembling Snapchat. But the redesigned Application did not have the success of its predecessor. The funding ran out and the parent company Unii went bankrupt. The company was not able to pay their content moderation team anymore, leading to a new surge of pornographic content on the App. Shortly after that, the Social Network was taken offline in August 2016. It has been inactive since. During the 2 years Fling was online, $21 million was raised from investors while generating no revenue at all. Of this $21 million (£16.5m), £5 million came from Nardone's father. == Allegations against CEO == Former employees made multiple allegations against Marco Nardone, the Founder and CEO of Unii and Fling. According to these claims, he behaved erratic and abusive, throwing "things across the office". He hired his girlfriend as the head of human resources to handle issues between him and his staff. Employees who left the company often had "some part of their pay held back". According to the reports, he also spent the money raised from investors irresponsibly, having no clear concept of a budget. Some of that money was used on expensive restaurants in London, a luxurious office for CEO Nardone and advertisements for Fling on Twitter and Facebook. Nardone also spent time partying in Ibiza with two employees, while the developer team in London frantically tried to get Fling back online after it being removed from the App Store. In December 2017 he pleaded guilty to assaulting his girlfriend at a domestic violence court.
AstroPay
AstroPay is a global digital wallet that provides users with a way to pay, send, and receive money. The app provides online payments, virtual and physical debit cards, peer-to-peer money transfers, and more. == History == AstroPay was founded in Uruguay in 2009 as a payment processing company. Over time, it expanded its services across Latin America, EMEA, and APAC. A significant milestone occurred in 2016, when AstroPay spun off dLocal, focusing on cross-border payments for emerging markets. dLocal became Uruguay's first unicorn and eventually went public through a successful IPO. In 2020, AstroPay spun off its payment processing services into a new entity, D24, to focus on mobile wallet for cross border. Between 2023 and 2024 the Company brought new leadership to guide its transition towards becoming a fully focused global digital multicurrency wallet where users save, send, and spend globally. This shift introduced enhanced features, including loyalty prepaid cards and multicurrency accounts. == Services == AstroPay offers three main products: AstroPay Wallet, AstroPay check-out, and AstroPay Platform. AstroPay Wallet is a digital wallet for consumers, where they have multicurrency accounts, prepaid card and marketplace. With AstroPay check-out, businesses can tap into AstroPay's wallet user base by accepting AstroPay as a payment method in their check-out options. Lastly, AstroPay Platform enables other businesses to use the AstroPay network to launch their own global wallet. == Brand endorsements, partnerships == AstroPay's marketing strategy has included the development of co-branded products with sports teams and other brand. The company sponsored Burnley Football Club during the 2018–19 Premier League season, renewing the partnership for the 2021–22 Premier League season when it became the club's official payment service partner. In August 2021, AstroPay entered into a partnership with the Wolverhampton Wanderers for the 2021-22 Premier League season, and the following year, became the team's shirt sponsor. Later, in September 2021, AstroPay expanded its partnership with Wolverhampton Wanderers, which included becoming the team's official payment partner and later, in 2023, co-launching a co-branded card. Other partnerships include Newcastle United in 2021 in the English Premier League. AstroPay made arrangements to ensure that branding and logo would be visible on the pitch-side LED advertising during Premier League matches. Furthermore, in June 2022, the company renewed it's partnership with Wolverhampton Wanderers for the 2022-23 Premier League season and launched its Wolves debit card in February 2023. Some other notable partnerships include: Universidad de Chile in 2024, Tottenham Hotspurs in 2023-25, and even a collaboration with Lionel Messi across all of Latin America. == Recent developments == AstroPay has refocused its strategy since 2023, pivoting from payment processing to concentrate on its global digital wallet. This move reflects a broader effort to redefine the company's market positioning by emphasizing global user-friendly financial services, while separating its identity from previous operations managed by dLocal and D24.
Flo (app)
Flo is a period-tracking app that provides menstrual cycle, ovulation and pregnancy tracking as well as perimenopause symptom tracking that was developed by Flo Health, Inc. It has over 380 million downloads worldwide and over 70 million monthly active users as of November 2024. In mid-2024, it reached unicorn status, and became Europe’s first femtech unicorn. The company has been accused of sharing users' sensitive health data with third parties without consent and misleading its users about data practices. == History == Flo Health, Inc. was co-founded in 2015 by Dmitry and Yuri Gurski, in Belarus. Their backgrounds helped build the first version of the software having experience in other fitness and health apps. Dmitry serves as the company's CEO. The company's development hubs are in London, Amsterdam and Vilnius. In 2016, the company raised $1 million in seed round funding from Flint Capital and Haxus Venture Fund. In 2017, Flo received an investment of $5 million from Flint Capital and model Natalia Vodianova with Vodianova helping develop an awareness campaign for the company. In 2018, Flo received an investment of $6 million from Mangrove Capital Partners, with participation from Flint Capital and Haxus, giving the company a valuation of $200 million. In mid-2019, Flo received an additional investment of $7.5 million led by Founders Fund. In 2020, the Federal Trade Commission alleged that Flo had misled users about its handling of health information to third parties including Google, Facebook, AppsFlyer, and Flurry since 2016. These allegations followed a 2019 report by The Wall Street Journal in reference to Facebook. The company reached a settlement in 2021 and was required to notify users of how their personal information was shared and obtain permission before any further information was shared. The agreement also required that Flo to undertake an independent privacy audit which it completed in March 2022. In early September 2021, Flo announced it closed $50M in a Series B financing, bringing the total capital raised to $65 million and company valuation to $800M led by VNV Global and Target Global. In March 2024, the Supreme Court of British Columbia certified a class action suit against Flo for sharing intimate data with Facebook and other third parties without user knowledge. In July 2024, Flo announced it raised more than $200M in Series C financing from General Atlantic bringing its valuation beyond $1 billion. As of November 2024, the app had over 380 million downloads world wide, and over 70 million monthly active users. In 2025, Flo adopted a data intelligence platform from Databricks to power its analytics and AI features, allowing users personalized cycle predictions. In 2025, a class action lawsuit in California was settled for $56 million with Flo paying $8 million and Google paying $48 million. == Features and privacy == Flo was initially created as a period and ovulation tracking application. It now provides reminders of upcoming menstrual cycles and a place to record various other health symptoms such as contraceptive methods, vaginal discharge (leukorrhea), water intake, pains, mood swings, and sexual activity. The application is available on iOS and Android. Flo is free to download and the free basic version gives you access to period and ovulation tracking and predictions, symptom tracking, cycle history, and anonymous mode. In Pregnancy mode, the app provides tracking features and educational material for pregnancy. In October 2023, Flo launched Flo for Partners, a feature that allows users to share their Flo data with their partner. In September 2022, as a response to Roe v. Wade being overturned, Flo sped up the release of a feature called "Anonymous Mode". Flo said this mode allows users to access the app without any personal identifiers such as name, email address, or technical identifiers being associated with their health data. Flo said it uses a technology called Oblivious HTTP to help protect user privacy in Anonymous Mode. == Recognition == Flo was named to Bloomberg’s Top 25 UK Startups to Watch for 2024. Flo's Anonymous Mode feature was recognized on both Fast Company's World Changing Ideas 2023 and TIME's Best Inventions List 2023. Flo is a CES 2019 Innovation Awards Honoree in the Software and Mobile Applications category.
Abiquo Enterprise Edition
Abiquo Hybrid Cloud Management Platform is a web-based cloud computing software platform developed by Abiquo. Written entirely in Java, it is used to build, integrate and manage public and private clouds in homogeneous environments. Users can deploy and manage servers, storage system and network and virtual devices. It also supports LDAP integration. == Hypervisors == Abiquo supports five hypervisor systems. VMware ESXi Microsoft Hyper-V Citrix XenServer Oracle VM Server for x86 KVM From version 3.1, it also supports multiple public cloud providers: Amazon AWS Rackspace Google Compute Engine HP Cloud ElasticHosts DigitalOcean Abiquo version 3.2 added: Microsoft Azure Abiquo version 3.4 added: Support for Docker hosts, adding multi-tenant networking, storage management and private registry management for Docker SoftLayer CloudSigma Later versions continued to add features including autoscaling on any cloud, integration to VMware NSX and OpenStack Neutron for software defined networking, guest config with cloud-init and integrated monitoring driving guest automation. == Storage services == Abiquo supports any vendor for hypervisor storage, and also supports tiered storage pools, enabling storage-as-a-service from specific vendors and technologies including: NFS Generic iSCSI NetApp Nexenta == SAAS version == In April 2014 Abiquo launched Abiquo anyCloud, a SAAS version of the Abiquo Hybrid Cloud Platform software. This version lets users manage public cloud resources from: Amazon AWS Microsoft Azure IBM SoftLayer DigitalOcean Rackspace Open Cloud (an OpenStack cloud) HP Public Cloud (an OpenStack cloud) Google Compute Engine ElasticHosts Additional security and process features include workflow, to have an enterprise administrator electronically sign off on changes, an audit trail of activity and the ability to share cloud accounts among and enterprise team in a secure way. == Reviews and awards == Finalist for the 2015 Cloud Awards Finalist for the 2015 UK Cloud Awards in the category Cloud Management Product of the Year EMA Radar for Private Cloud platforms 2013 Global Telecoms Business Innovation Summit and Awards 2013 (with Interoute) EuroCloud UK Awards
Pixelmator Pro
Pixelmator Pro is a photo, video, and vector graphic editor developed by Apple for macOS and iPadOS as part of its Pixelmator and pro apps platforms and as a part of their Apple Creator Studio suite of applications. Pixelmator Pro relies heavily on technologies from Apple platforms such as Metal, CoreML, Core Image, AVFoundation, GCD, and SwiftUI. == Features == GPU accelerated with Metal 50+ standard image editing tools Layer-based image editor Video editing support Vector graphic support (including SVG support) AI-powered editing features such as background removal ML Super Resolution and Smart Replace Supports a variety of media formats (JPEG, RAW, Apple ProRAW, PSD, PNG, GIF, MP4, HEIF, etc) == Reception == Pixelmator Pro was generally well-received by reviewers who praised its deep use of machine learning, fully macOS-native design, and relatively affordable one-time purchase compared to subscription software such as Adobe Photoshop. Some reviewers criticized that some features are hard to find or hard to use. It was awarded Apple's Mac App of the Year in 2018. Pixelmator Pro does not have support for panorama stitching. == Acquisition by Apple == On November 1, 2024, the Pixelmator Team announced that they were to be acquired by Apple, subject to regulatory approval. Their site promises "There will be no material changes to the Pixelmator Pro, Pixelmator for iOS, and Photomator apps at this time." The acquisition was completed in February 2025. On January 13, 2026, Apple announced that a new version of Pixelmator Pro with AI features would be included in its new Apple Creator Studio subscription, the app would be brought to the iPad and the Mac app would be redesigned with Liquid Glass. == Version history == == Applescript == In 2020 Pixelmator Pro added the ability to leverage Apple's automation language 'AppleScript' to automate many tasks in version 1.8 (Lynx). This enabled simple and advanced automation activities such as image resize, crop, color adjustments, format change, moving layers around, and more advanced actions like removing background, Gaussian blur, text replacement, shadows, color replacement, etc.