Within Active Learning
Why the Best Prediction May Be the Wrong Experiment
Active learning must sometimes test uncertain candidates rather than the predicted winner, or it may miss better discoveries elsewhere.
On this page
- How exploitation traps search in familiar regions
- How uncertainty and expected improvement encourage exploration
- When diversity based selection finds overlooked possibilities
Page outline Jump by section
Introduction
A successful active learning system does not always choose the experiment that currently looks most promising. Sometimes the most valuable experiment is one that seems merely plausible but lies in a poorly understood part of the search space. Testing these uncertain regions prevents the system from becoming trapped by its own early assumptions and gives it a chance to discover genuinely better solutions.
This balance between exploitation (improving the best-known candidate) and exploration (learning about uncertain possibilities) is one of the defining mechanisms behind AI-guided scientific discovery. It matters because many scientific search spaces—from drug molecules to battery materials and protein designs—contain multiple peaks of quality. If an AI repeatedly refines only today’s favourite candidate, it may never discover a superior region that initially appeared unremarkable. Well-designed active learning systems therefore deliberately invest some experiments in reducing uncertainty as well as chasing immediate gains.[NIST]nist.govfly closed loop materials discovery bayesian active learningOn-the-fly closed-loop materials discovery via Bayesian active learning | NISTNovember 24, 2020…
How exploitation traps search in familiar regions
Pure exploitation sounds sensible. If one candidate appears to outperform every alternative, why not continue improving it?
The problem is that early predictions are based on incomplete evidence. A machine learning model trained on only a handful of experiments inevitably has blind spots. Areas with abundant data often receive confident predictions, while unexplored regions remain uncertain. If the algorithm always chooses the highest predicted score, it repeatedly samples near the same part of the search space, making its existing beliefs even stronger without seriously testing whether those beliefs are correct.[Springer]link.springer.comActive Learning and Bayesian Optimization: A Unified Perspective to Learn with a Goal | Archives of Computational Methods in Engi…
Researchers often describe this as becoming trapped in a local optimum. Imagine hiking through fog while trying to reach the highest mountain. Climbing the nearest hill always feels like progress, but unless you occasionally explore beyond the surrounding valleys, you may never discover that a much taller peak exists elsewhere.
Scientific discovery presents the same challenge. A promising family of catalysts, proteins or chemical compounds may continue yielding modest improvements while an entirely different family—barely sampled so far—contains the true breakthrough. Without exploration, the algorithm has little chance of finding it.[OUP Academic]academic.oup.comOUP AcademicBenchmarking active learning strategies for materials optimization and discovery | Oxford Open Materials Science | Oxford Aca…
Within the broader vision of AI accelerating science, this distinction is crucial. The goal is not merely to optimise today’s best design faster, but to expand humanity’s ability to uncover discoveries that conventional trial-and-error or human intuition might never reach.
How uncertainty and expected improvement encourage exploration
Modern active learning systems avoid these traps by estimating not only what they expect to happen, but also how uncertain those expectations are.
Many systems use probabilistic surrogate models, such as Gaussian processes, which produce two outputs for every untested candidate:
- a predicted performance; and
- an estimate of prediction uncertainty.
These uncertainty estimates allow the algorithm to ask a richer question than “Which experiment looks best?” Instead it asks, “Which experiment is most likely to improve what we know or reveal something even better?”[Springer]link.springer.comActive Learning and Bayesian Optimization: A Unified Perspective to Learn with a Goal | Archives of Computational Methods in Engi…
One widely used acquisition strategy is Expected Improvement (EI). Rather than selecting only the highest predicted performer, EI weighs two competing possibilities:
- High predicted reward, favouring candidates likely to outperform the current best.
- High uncertainty, favouring candidates whose true value could be much better than current predictions suggest.
An experiment with only a moderate predicted score can therefore become the preferred choice if the uncertainty is large enough that it might reveal an unexpected advance. This gives exploration an explicit mathematical value rather than treating it as random curiosity.[OUP Academic]academic.oup.comOUP AcademicBenchmarking active learning strategies for materials optimization and discovery | Oxford Open Materials Science | Oxford Aca…
Other acquisition functions express the same principle differently. Upper Confidence Bound methods deliberately add an “optimism bonus” to uncertain predictions, while information-based methods seek experiments that reduce uncertainty across the wider search space. Although their mathematics differ, they all recognise that learning where the model is ignorant can be as valuable as confirming what it already believes.[Springer]link.springer.comActive Learning and Bayesian Optimization: A Unified Perspective to Learn with a Goal | Archives of Computational Methods in Engi…
When diversity-based selection finds overlooked possibilities
Exploration is not only about uncertainty. It is also about diversity.
Many candidate experiments resemble one another. Testing twenty nearly identical molecules or material compositions may produce only small refinements in understanding. A diversity-aware active learning strategy instead spreads experiments across distinct regions of the search space so that each new result teaches something genuinely different.[NIST]nist.govbenchmarking active learning strategies materials optimization and discoveryBenchmarking Active Learning Strategies for Materials Optimization and Discovery | NISTJuly 9, 2022…
This can produce discoveries that would otherwise remain invisible.
For example:
- In materials science, sampling compositions from different parts of a phase diagram can reveal entirely new material behaviours instead of endlessly tuning one composition.
- In molecular discovery, examining chemically distinct structures broadens understanding of how molecular features influence performance rather than simply optimising one successful scaffold.
- In protein engineering, evaluating varied sequence families helps reveal multiple functional solutions rather than converging prematurely on one evolutionary pathway.
The practical benefit is greater resilience against early modelling errors. If one promising region ultimately proves disappointing, the algorithm has already gathered information elsewhere instead of having spent its entire experimental budget refining a dead end.[NIST]nist.govbenchmarking active learning strategies materials optimization and discoveryBenchmarking Active Learning Strategies for Materials Optimization and Discovery | NISTJuly 9, 2022…
Exploration is calculated, not random
A common misconception is that exploration means performing arbitrary experiments.
Modern active learning rarely works that way. Instead, exploration is directed towards experiments expected to produce the greatest increase in useful knowledge.
An exploratory experiment is typically selected because it is expected to:
- reduce uncertainty over a large region of the search space;
- distinguish between competing scientific hypotheses;
- reveal whether an apparently mediocre region actually contains exceptional candidates; or
- improve future experiment selection even if the immediate result is unsuccessful.
In this sense, exploration is an investment. A single informative failure can prevent dozens of wasted experiments later by correcting the model’s understanding of the landscape. This is one reason autonomous laboratories emphasise learning efficiency rather than simply maximising immediate performance.[NIST]nist.govfly closed loop materials discovery bayesian active learningOn-the-fly closed-loop materials discovery via Bayesian active learning | NISTNovember 24, 2020…
Evidence from autonomous scientific discovery
Real autonomous research systems illustrate why exploration matters.
The National Institute of Standards and Technology (NIST) has demonstrated closed-loop materials discovery systems in which machine learning repeatedly recommends experiments, receives new measurements and updates its model before selecting the next candidate. Rather than following a fixed optimisation path, these systems continually balance searching promising regions with investigating uncertain ones, allowing them to discover previously unknown materials more efficiently than conventional sequential testing.[NIST]nist.govfly closed loop materials discovery bayesian active learningOn-the-fly closed-loop materials discovery via Bayesian active learning | NISTNovember 24, 2020…
Benchmark studies comparing different acquisition strategies likewise show that exploration-aware methods generally outperform purely exploitative approaches when searching complex scientific landscapes. In a benchmark based on real Fe–Co–Ni materials data, acquisition functions that balanced predicted performance with uncertainty consistently identified high-performing materials more efficiently than simpler strategies, with Expected Improvement achieving particularly strong overall performance.[NIST]nist.govbenchmarking active learning strategies materials optimization and discoveryBenchmarking Active Learning Strategies for Materials Optimization and Discovery | NISTJuly 9, 2022…
These results reinforce an important lesson: success depends not only on building accurate predictive models but also on asking the right experimental questions.
Why this mechanism matters for AI-enabled scientific progress
For the broader vision of AI accelerating discovery, exploration is more than a technical detail. It is one of the mechanisms that allows AI systems to search beyond established scientific intuition.
Many transformative discoveries have emerged from investigating ideas that initially appeared uncertain or unconventional. An active learning system that only refines today’s apparent winner risks reproducing existing assumptions at greater speed. One that systematically explores uncertainty has a better chance of uncovering new classes of drugs, materials, catalysts or biological designs that reshape what scientists consider possible.
That does not eliminate uncertainty or guarantee breakthroughs. Exploration always spends some experiments on ideas that fail. The challenge is choosing failures that are informative enough to make every subsequent experiment smarter. When active learning achieves that balance, it becomes a powerful tool for scientific acceleration—helping limited experimental resources search not just faster, but more intelligently.[springer.com]link.springer.comActive Learning and Bayesian Optimization: A Unified Perspective to Learn with a Goal | Archives of Computational Methods in Engi…
Amazon book picks
Further Reading
Books and field guides related to Why the Best Prediction May Be the Wrong Experiment. Use these as the next step if you want deeper reading beyond the article.
Algorithms to Live By
A fascinating exploration of how computer algorithms can be applied to our everyday lives. In this dazzlingly interdisciplinary work, acc...
Superforecasting
The international bestseller 'A manual for thinking clearly in an uncertain world. Read it.' Daniel Kahneman, author of Thinking, Fast an...
The Elements of Statistical Learning
Rating: 4.5/5 from 13 Google Books ratings
This book describes the important ideas in a common conceptual framework. While the approach is statistical, the emphasis is on concepts...
Thinking, Fast and Slow
Why is there more chance we'll believe something if it's in a bold type face? Why are judges more likely to deny parole before lunch? Why...
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Selected fromrobotics kit oneBay.co.uk.
Endnotes
1.
Source: nist.gov
Title: fly closed loop materials discovery bayesian active learning
Link:https://www.nist.gov/publications/fly-closed-loop-materials-discovery-bayesian-active-learning
Source snippet
On-the-fly closed-loop materials discovery via Bayesian active learning | NISTNovember 24, 2020...
Published: November 24, 2020
2.
Source: link.springer.com
Link:https://link.springer.com/article/10.1007/s11831-024-10064-z
Source snippet
Active Learning and Bayesian Optimization: A Unified Perspective to Learn with a Goal | Archives of Computational Methods in Engi...
3.
Source: academic.oup.com
Link:https://academic.oup.com/ooms/article/2/1/itac006/6637521
Source snippet
OUP AcademicBenchmarking active learning strategies for materials optimization and discovery | Oxford Open Materials Science | Oxford Aca...
4.
Source: nist.gov
Title: benchmarking active learning strategies materials optimization and discovery
Link:https://www.nist.gov/publications/benchmarking-active-learning-strategies-materials-optimization-and-discovery
Source snippet
Benchmarking Active Learning Strategies for Materials Optimization and Discovery | NISTJuly 9, 2022...
Published: July 9, 2022
5.
Source: nist.gov
Link:https://www.nist.gov/publications/active-learning-regression-structure-property-mapping-importance-sampling-and
6.
Source: nist.gov
Link:https://www.nist.gov/programs-projects/autonomous-systems-materials-research-and-metrology-accelerating-discovery-and
7.
Source: nist.gov
Link:https://www.nist.gov/programs-projects/machine-learning-high-throughput-materials-discovery-and-optimization-applications
8.
Source: ouci.dntb.gov.ua
Link:https://ouci.dntb.gov.ua/en/works/4gY0Z509/
Additional References
9.
Source: doi.org
Link:https://doi.org/10.1039/D6DD00081A
Source snippet
May 25, 2026 — Probability of improvement (PI) | Balanced; tunable via ξ | High | Identifying materials with low lattice thermal conducti...
Published: May 25, 2026
10.
Source: youtube.com
Title: Exploration vs. Exploitation
Link:https://www.youtube.com/watch?v=mo96Nqlo1L8
Source snippet
"Where and How of Exploration and Exploitation" by Ann Kowal Smith Case Western Reserve University · 1.7K views...
11.
Source: youtube.com
Title: Active Materials Exploration and Characterization with Bayesian Optimization
Link:https://www.youtube.com/watch?v=DsNR7czaVXI
Source snippet
Ensuring Exploration and Exploitation in Graph-Based Active Learning...
12.
Source: youtube.com
Title: Active Learning via Bayesian Optimization for Materials Discovery
Link:https://www.youtube.com/watch?v=yqspdqa0H-s
Source snippet
Active Materials Exploration and Characterization with Bayesian Optimization...
13.
Source: nature.com
Link:https://www.nature.com/articles/s41598-025-13972-7
14.
Source: youtube.com
Title: Ensuring Exploration and Exploitation in Graph-Based Active Learning
Link:https://www.youtube.com/watch?v=ayU-FO-XcJY
Source snippet
Bandits, Active Learning, Bayesian RL and Global Optimization...
15.
Source: youtube.com
Title: Active Learning for Materials Design with Kevin Tran
Link:https://www.youtube.com/watch?v=HXHKBeWWZPw
Source snippet
Exploration vs. Exploitation - Learning the Optimal Reinforcement Learning Policy...
16.
Source: youtube.com
Title: Bandits, Active Learning, Bayesian RL and Global Optimization
Link:https://www.youtube.com/watch?v=5rev-zVx1Ps
Source snippet
Active Learning for Materials Design with Kevin Tran...
17.
Source: pubs.rsc.org
Link:https://pubs.rsc.org/en/content/articlehtml/2026/dd/d6dd00081a
Source snippet
OPTIMIZATION-DRIVEN MATERIALS DISCOVERY 3.2.1 Interplay between surrogate models and sampling strategy. The dominant AL practices in both...
18.
Source: pubs.rsc.org
Link:https://pubs.rsc.org/en/content/articlehtml/2026/dd/d6dd00134c
Source snippet
rsc.orgNovelty-aware evolutionary Bayesian optimisation for multi-objective discovery science - Digital Discovery (RSC Publishing) DOI:10...