Who gets missed by medical AI?

Introduction

Artificial intelligence in medicine is often presented as a way to make diagnosis faster, cheaper, and more accurate. In the most optimistic versions of the AI abundance story, advanced systems could help detect disease earlier, personalise treatment, and extend healthy life for billions of people. But there is a quieter problem underneath many medical AI systems: they learn from healthcare records that already reflect unequal access to care.

If some populations are underrepresented in training data, receive fewer tests, or have less complete medical histories, AI systems can inherit those gaps. The result is not necessarily dramatic failure. More often, it is something subtler and potentially more dangerous: certain groups are missed more often, diagnosed later, or classified as lower risk than they really are. In a future where AI-guided healthcare becomes increasingly important, those errors could compound existing inequalities in life expectancy and healthy lifespan rather than reduce them.

Why healthcare records underrepresent some groups

Medical AI depends on data. Electronic health records, medical images, insurance claims, laboratory results, and genomic databases provide the raw material from which algorithms learn patterns. The problem is that these datasets are not neutral snapshots of human health.

Many healthcare datasets are built from large hospitals, academic medical centres, insured populations, or regions with stronger healthcare infrastructure. People who struggle to access care often generate fewer records in the first place. Those records may also be less complete because diagnoses occur later, follow-up visits are less frequent, and preventive screening is less common. Researchers have repeatedly warned that these patterns can make machine-learning systems less reliable for underrepresented populations.[PMC]pmc.ncbi.nlm.nih.govPMCAddressing bias in big data and AI for health careThis can result in misdiagnosing certain…Read more…[PMC]pmc.ncbi.nlm.nih.govPMCSources of bias in artificial intelligence that perpetuateby LA Celi · 2022 · Cited by 610 — The lack of diverse digital datasets for ML algorithms can amplify systematic underrepresentation o…

The bias can emerge through several pathways:

This means a model can achieve impressive average accuracy while still performing substantially worse for specific groups.

That distinction matters because healthcare is not like a recommendation system or advertising platform. Missing a diagnosis can delay treatment, increase complications, and shorten lives.

The chest X-ray warning case

One of the most influential demonstrations of this problem came from research on AI systems used to analyse chest radiographs.

In 2021, researchers examined multiple chest X-ray classification models across several major datasets. They focused on a particularly important failure mode: underdiagnosis. Instead of asking whether overall accuracy was high, they asked which patients were most likely to have disease incorrectly labelled as absent.[Nature]nature.comUnderdiagnosis bias of artificial intelligence algorithms…by L Seyyed-Kalantari · 2021 · Cited by 1052 — Here, we examine algori…

The results were troubling.

Across different datasets and institutions, the researchers found systematic underdiagnosis for historically underserved populations. Certain groups were more likely to receive false-negative results, meaning the AI system was more likely to classify a sick patient as healthy. The disparities appeared across multiple pathologies and persisted even when researchers examined different datasets and modelling approaches.[Nature]nature.comNat. Med. 27…Read more…

The significance of the study was not merely that bias existed. Researchers already suspected that. The striking finding was that the problem remained visible across different datasets and models, suggesting that unequal performance was deeply connected to the structure of the underlying data rather than a single defective algorithm.

For patients, false negatives can be especially damaging. A false positive may trigger extra testing. A false negative can delay diagnosis entirely.

In practical terms, a hospital could deploy an AI screening tool believing it improves efficiency and accuracy overall, while some populations quietly receive worse screening performance than the averages suggest.

How inequality becomes embedded in training data

The chest X-ray findings highlight a broader issue. Medical datasets often reflect not only disease but also the social conditions under which disease is observed.

Consider two patients with similar underlying illness:

The first patient’s condition is more likely to be identified, labelled, and recorded. The second patient’s condition may remain partially invisible in the dataset.

When machine-learning systems are trained on those records, they can learn patterns associated with healthcare access rather than health itself. Researchers describe this as a major source of bias in healthcare AI. PMC[PLOS]journals.plos.orgSources of bias in artificial intelligence that perpetuate…by LA Celi · 2022 · Cited by 614 — Here, we describe the landscape of A…

The problem becomes even more complicated when datasets use proxies.[nature.com]nature.comThe value of standards for health datasets in artificial…by A Arora · 2023 · Cited by 353 — However, a growing body of evidence has hi…

A famous example came from a widely used healthcare risk algorithm studied by Ziad Obermeyer and colleagues. The system attempted to identify patients needing additional care management. Instead of predicting illness directly, it used future healthcare spending as a proxy for health need. Because Black patients historically received less medical spending for equivalent illness levels, the algorithm learned to treat them as lower risk. Researchers estimated that correcting the bias would dramatically increase the proportion of Black patients identified for additional support.[Science]science.orgfind evidence of racial bias in one widely used algorithm, such that Black patients assigned the same level of risk by the algorithm are…

The algorithm was not explicitly programmed to discriminate. The bias emerged because historical spending patterns reflected unequal healthcare access.

That case became one of the clearest demonstrations that apparently neutral data can encode social inequality.

Skin tone, ancestry, and the limits of medical datasets

The problem extends beyond healthcare records.

Many medical imaging datasets contain far more examples from certain populations than others. Dermatology has become a frequently cited example because skin conditions can appear differently across skin tones.

Researchers have warned that machine-learning systems trained predominantly on lighter-skinned patients may perform less reliably for darker-skinned populations. Underrepresentation in dermatological datasets creates a risk that algorithms will miss or misclassify conditions that present differently across diverse populations.[probiologists.com]probiologists.comJournal of Biomed Researchby G Kleinberg · 2022 · Cited by 62 — This review explores the extent, causes, possible solutions, and overall…

Genomic medicine faces related challenges.

Large genetic databases remain heavily weighted toward people of European ancestry. As a result, predictive tools built from those datasets may generalise less effectively to other populations. Reviews of medical-device and health-equity research have repeatedly highlighted this concern, especially for polygenic risk scores that attempt to estimate future disease risk from genetic information.[ScienceDirect]sciencedirect.comWeighing the benefits and risks of collecting race…by A Fiske · 2025 · Cited by 13 — This Viewpoint weighs the risks of c…

These examples reveal a recurring pattern: when representation gaps exist in the underlying data, prediction gaps often follow.

Why average accuracy can hide unequal outcomes

A common misunderstanding is that fairness problems disappear once an AI system reaches high overall accuracy.

In reality, average performance can conceal substantial differences between groups.

Imagine a diagnostic model that achieves 95% accuracy overall. That figure sounds impressive. But if performance is 98% for one population and 85% for another, the average masks a clinically important disparity.

Researchers increasingly argue that healthcare AI should be evaluated not only on aggregate performance but also on subgroup outcomes. This includes examining results across race, sex, age, socioeconomic status, disability, language background, and combinations of those characteristics.[PMC]pmc.ncbi.nlm.nih.govPMCAddressing bias in big data and AI for health careThis can result in misdiagnosing certain…Read more…[ScienceDirect]sciencedirect.comWeighing the benefits and risks of collecting race…by A Fiske · 2025 · Cited by 13 — This Viewpoint weighs the risks of c…

Intersectional analysis is especially important.

The 2021 chest X-ray study found some of the highest underdiagnosis rates in intersectional groups that combined multiple forms of disadvantage. More recent research on medical imaging foundation models reported similar patterns, with elevated underdiagnosis rates for groups such as Black women compared with broader population averages.[Nature]nature.comFirst published in 1869, Nature is the world's leading multidisciplinary science journal. Nature publishes the finest peer-reviewed…

A model may therefore appear equitable when comparing broad categories while still failing particular subpopulations.

How fairness testing can reveal hidden failure

One lesson from recent research is that fairness cannot be treated as a final quality-control step.

Researchers increasingly recommend testing for unequal outcomes throughout development and deployment rather than after a system reaches clinical use.[ScienceDirect]sciencedirect.comWeighing the benefits and risks of collecting race…by A Fiske · 2025 · Cited by 13 — This Viewpoint weighs the risks of c…

Several approaches are becoming more common:

Auditing performance across groups

Instead of reporting only overall accuracy, developers examine false-positive and false-negative rates for different populations.

This helps identify situations where one group experiences substantially more missed diagnoses than another. The chest X-ray underdiagnosis studies are examples of this approach.[Nature]nature.commore…

Testing on multiple datasets

Models often perform well on the institution where they were trained but degrade elsewhere.

Evaluating systems across hospitals, regions, and patient populations can reveal whether apparent performance depends on narrow training conditions.[PLOS]journals.plos.orgSources of bias in artificial intelligence that perpetuate…by LA Celi · 2022 · Cited by 614 — Here, we describe the landscape of A…

Improving dataset diversity

Researchers increasingly argue that better representation during data collection is one of the most important fairness interventions.

That may involve recruiting more diverse participants, collecting data from underserved regions, or ensuring that demographic information is available for auditing.[ScienceDirect]sciencedirect.comWeighing the benefits and risks of collecting race…by A Fiske · 2025 · Cited by 13 — This Viewpoint weighs the risks of c…[ScienceDirect]sciencedirect.comTackling algorithmic bias and promoting transparency in…by JE Alderman · 2025 · Cited by 157 — Biases in the outputs of a…

Monitoring after deployment

Bias may emerge only after systems are introduced into real clinical environments.

Healthcare populations change over time, disease patterns shift, and hospital workflows evolve. Ongoing monitoring can identify performance drift that was not visible during development.[PMC]pmc.ncbi.nlm.nih.govPMCSources of bias in artificial intelligence that perpetuateby LA Celi · 2022 · Cited by 610 — The lack of diverse digital datasets for ML algorithms can amplify systematic underrepresentation o…

The deeper challenge for an AI longevity future

The concern is not simply that medical AI can make mistakes. Human clinicians make mistakes too.

The deeper issue is scale.

A biased doctor may affect hundreds or thousands of patients during a career. A biased diagnostic model can influence millions of decisions across health systems. If disparities are embedded into widely deployed AI infrastructure, unequal outcomes can become faster, more consistent, and harder to detect.

This matters directly to the wider question of AI-guided longevity. If future healthcare increasingly relies on predictive models, continuous monitoring, automated triage, and personalised prevention, then the quality of the underlying data becomes part of the distribution problem. The people whose lives are most improved by AI medicine may be the people most visible to AI systems.

That outcome is not inevitable. Better datasets, stronger auditing, transparent reporting, fairness testing, and broader healthcare access can reduce the risk. But the evidence from chest X-ray systems, healthcare risk algorithms, dermatology datasets, and genomic prediction tools suggests that unequal representation is not a peripheral technical issue.

It is one of the mechanisms through which a technology designed to extend healthy life could end up extending it unevenly.[ScienceDirect]sciencedirect.comWeighing the benefits and risks of collecting race…by A Fiske · 2025 · Cited by 13 — This Viewpoint weighs the risks of c…[Nature]nature.comBias recognition and mitigation strategies in artificial…by F Hasanzadeh · 2025 · Cited by 253 — This review examines the origins of b…[PubMed]pubmed.ncbi.nlm.nih.govbias of artificial intelligence algorithms…by L Seyyed-Kalantari · 2021 · Cited by 1051 — Here, we examine algorithmic underdiagnosis…

Amazon book picks

Further Reading

Books and field guides related to Who gets missed by medical AI?. Use these as the next step if you want deeper reading beyond the article.

Book

Algorithms of Oppression

By Safiya Umoja Noble

Frames how data systems can reproduce inequality.

See on Amazon

Book

Invisible Women

By Caroline Criado Perez

Explains how missing data can produce harmful design and policy failures.

See on Amazon

Book

Deep Medicine

By Eric Topol

Explains the promise and risks of AI in clinical care.

See on Amazon

Book

Weapons of Math Destruction

By Cathy O'Neil

Directly relevant to hidden bias and unequal algorithmic outcomes.

See on Amazon

Browse more on Amazon:Algorithms of OppressionInvisible WomenDeep Medicine

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Example marketplace items related to this page. Use the search link to explore similar finds on eBay.

Shop location

UsingUSA USA USAUKCanadaAustraliaIreland

Example eBay listing

Soviet Russian Poster First Aid Medical Kit AI Army Forces, Military AV 18x24"

SearcheBay.co.uk: medical AI poster

Browse similar oneBay.co.uk

Example eBay listing

Soviet Poster ORIGINAL First Aid kit Medical AI-2 NBC Chernobyl stalker USSR

SearcheBay.co.uk: medical AI poster

Browse similar oneBay.co.uk

Example eBay listing

Soviet Russian Poster First Aid Individual Medical Kit AI Military Army 18x24"

SearcheBay.co.uk: medical AI poster

Browse similar oneBay.co.uk

Example eBay listing

Soviet Russian Poster First Aid Individual Medical Kit AI Military 24x36"

SearcheBay.co.uk: medical AI poster

Browse similar oneBay.co.uk

Browse more oneBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: pmc.ncbi.nlm.nih.gov
Title: PMCAddressing bias in big data and AI for health care
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC8515002/

2. Source: pmc.ncbi.nlm.nih.gov
Title: PMCSources of bias in artificial intelligence that perpetuate
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC9931338/

3. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC11542778/

4. Source: journals.plos.org
Link:https://journals.plos.org/digitalhealth/article?id=10.1371%2Fjournal.pdig.0000022

5. Source: nature.com
Link:https://www.nature.com/articles/s41591-021-01595-0

6. Source: probiologists.com
Link:https://www.probiologists.com/article/racial-underrepresentation-in-dermatological-datasets-leads-to-biased-machine-learning-models-and-inequitable-healthcare

7. Source: sciencedirect.com
Link:https://www.sciencedirect.com/science/article/pii/S2589750025000032

8. Source: sciencedirect.com
Link:https://www.sciencedirect.com/science/article/pii/S2589750024002243

9. Source: sciencedirect.com
Link:https://www.sciencedirect.com/science/article/pii/S2514664524015674

10. Source: nature.com
Link:https://www.nature.com/articles/s41746-023-00858-z

11. Source: nature.com
Link:https://www.nature.com/

12. Source: nature.com
Link:https://www.nature.com/articles/s41467-024-52003-3

13. Source: nature.com
Link:https://www.nature.com/articles/s41746-025-01503-7

14. Source: nature.com
Link:https://www.nature.com/articles/s41591-023-02608-w

15. Source: nature.com
Link:https://www.nature.com/articles/s41591-024-02838-6

16. Source: nature.com
Link:https://www.nature.com/articles/d41586-019-03228-6

17. Source: sciencedirect.com
Link:https://www.sciencedirect.com/science/article/pii/S1386505625002680

18. Source: sciencedirect.com
Link:https://www.sciencedirect.com/science/article/pii/S2688115225002693

19. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/34893776/

20. Source: science.org
Link:https://www.science.org/doi/10.1126/science.aax2342

21. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/31649194/

22. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/39488857/

23. Source: Wikipedia
Title: Artificial intelligence
Link:https://en.wikipedia.org/wiki/Artificial_intelligence

24. Source: x.com
Link:https://x.com/NatureMedicine/status/1474047774417211401

Additional References

25. Source: onwork.edu.au
Link:https://onwork.edu.au/bibitem/2019-Obermeyer%2CZiad-Powers%2CBrian-etal-Dissecting%2Bracial%2Bbias%2Bin%2Ban%2Balgorithm%2Bused%2Bto%2Bmanage%2Bthe%2Bhealth%2Bof%2Bpopulations/

26. Source: researchgate.net
Link:https://www.researchgate.net/publication/356949367_Underdiagnosis_bias_of_artificial_intelligence_algorithms_applied_to_chest_radiographs_in_under-served_patient_populations

27. Source: linkedin.com
Link:https://www.linkedin.com/posts/jacreator_aiengineering-responsibleai-fairnessinml-activity-7436094639964573696-5JaE

28. Source: partners4healthequity.org
Link:https://www.partners4healthequity.org/resource-library/racial-bias-found-major-health-care-risk-algorithm

29. Source: delfina.com
Link:https://www.delfina.com/resource/racial-bias-in-healthcare-ai-applications

30. Source: rrapp.spia.princeton.edu
Link:https://rrapp.spia.princeton.edu/an-algorithm-designed-to-predict-health-care-costs-as-a-proxy-for-health-needs-critically-underestimates-the-needs-of-black-patients-with-life-threatening-consequences/

31. Source: racismandtechnology.center
Link:https://racismandtechnology.center/2023/02/18/racist-technology-in-action-the-underdiagnosis-bias-in-ai-algorithms-for-health-chest-radiographs/

32. Source: journalistsresource.org
Link:https://journalistsresource.org/home/research-artificial-intelligence-can-fuel-racial-bias-in-health-care-but-can-mitigate-it-too/

33. Source: healthyml.org
Link:https://healthyml.org/publication/seyyed-2021-underdiagnosis/

34. Source: rrapp.spia.princeton.edu
Link:https://rrapp.spia.princeton.edu/algorithms-can-replicate-or-remedy-racial-biases-in-healthcare-resource-allocation/