Lost autonomy

Introduction

An AI-driven economy could become dramatically more productive without making ordinary people more free. That is one of the central tensions inside the broader debate about AI abundance. GDP may rise because software systems optimise logistics, automate services, accelerate research, and reduce costs across entire industries. Yet the same systems can also concentrate power, weaken bargaining positions, and make individuals more dependent on institutions they cannot meaningfully understand or challenge.

Lost autonomy illustration 1 This matters because the optimistic case for “AI bloom” is not only about producing more goods and services. It is about expanding human capability and agency: the ability to choose meaningful work, participate in society, protect privacy, shape one’s environment, and retain control over important life decisions. A society can become richer while citizens lose practical autonomy over employment, information, finance, healthcare, or even their own attention.

The key question is therefore not simply whether AI increases output, but who controls the systems that increasingly organise daily life, and whether ordinary people retain genuine room to refuse, negotiate, switch providers, appeal decisions, or live outside tightly managed digital infrastructures.

Why fast AI growth can weaken personal agency

The clearest mechanism is dependency. Advanced AI systems often become more useful as they absorb more data, integrate more services, and operate across larger networks. That creates strong pressure toward centralisation.

A small number of firms may end up controlling the most capable models, the largest computing infrastructure, the dominant cloud platforms, and the interfaces through which people access education, work, healthcare, entertainment, or government services. Researchers and regulators increasingly warn that generative AI could reinforce “gatekeeper” dynamics already visible in digital platforms. [arXiv]arxiv.orgarXivAI and the EU Digital Markets Act: Addressing the Risks of Bigness in Generative AIJuly 7, 2023…Published: July 7, 2023

From a GDP perspective, this concentration can still look highly successful. Large AI firms may generate enormous productivity gains. Consumers may receive cheaper services. Entire sectors may become more efficient. But efficiency alone does not tell us whether citizens retain meaningful leverage over the systems they rely on.

Three forms of autonomy loss matter especially:

Economic dependency: workers become replaceable complements to AI systems controlled by a few firms.
Informational dependency: people increasingly rely on opaque recommendation and decision systems to navigate reality.
Institutional dependency: access to essential services becomes mediated through proprietary AI infrastructure.

In each case, output can rise while human control weakens.

Ownership concentration and dependency risks

The modern AI economy depends heavily on scarce resources: advanced chips, giant data centres, cloud infrastructure, proprietary training data, and elite research talent. That creates unusually high barriers to entry.

As a result, much of the frontier AI ecosystem is concentrated among a relatively small set of corporations and states. This does not automatically produce abuse. Large-scale coordination may genuinely accelerate scientific progress and reduce costs. But concentration changes the balance of power between institutions and individuals.

If a handful of firms control the systems that mediate communication, employment screening, medical triage, tutoring, finance, or software development, people may formally retain freedom while practically losing alternatives.

This is not entirely new. Industrial economies have long depended on large infrastructures such as railways, electricity grids, telecommunications networks, and banking systems. The difference is that advanced AI may increasingly mediate cognition itself: search, memory, judgement, persuasion, planning, and decision support.

That creates deeper forms of dependence than earlier infrastructure systems.

A worker whose schedule, productivity targets, performance reviews, route assignments, and future employability are continuously shaped by AI management tools may technically remain employed in a growing economy while losing practical discretion over daily life. OECD research on algorithmic management notes that these systems can improve efficiency and consistency, but also create significant concerns around worker surveillance, work intensity, and reduced control. [OECD]oecd.orgOECDUsing AI in the workplaceTo support the adoption of trustworthy AI in the workplace, this policy paper identifies the main risks that… [OECD]oecd.orgalgorithmic management in the workplace 287c13c4 enAlgorithmic management in the workplaceby A Milanez · 2025 · Cited by 48 — Algorithmic management – the use of software, which may includ…

The risk becomes larger if AI systems become difficult to contest. Human beings can negotiate with managers, explain context, or appeal to judgement. Automated systems often reduce these informal channels. Even when humans remain nominally “in the loop”, organisational incentives may encourage rubber-stamping algorithmic outputs.

The workplace as an early warning sign

The clearest present-day examples already exist in workplaces shaped by algorithmic management.

In warehouses, logistics firms, call centres, ride-hailing platforms, and delivery services, software systems increasingly allocate tasks, measure performance, predict behaviour, and trigger disciplinary actions. AI systems can monitor keystrokes, movement patterns, customer interactions, biometric signals, and productivity metrics at scales impossible for human supervisors alone.

Supporters argue that such systems improve efficiency, reduce waste, optimise staffing, and standardise decisions. Critics argue that they can produce environments of continuous surveillance and behavioural pressure.

Research literature on algorithmic management consistently identifies risks to worker autonomy, including reduced discretion, intensified monitoring, and diminished capacity to challenge decisions. FEPS 3PMC [AI Now Institute]ainowinstitute.orgalgorithmic managementRestraining Workplace…11 Apr 2023 — To meaningfully build worker power, we must create policies to regulate algorithmic management tha…

This matters because these systems can coexist with rising economic output. A logistics network managed by AI may move goods faster and cheaper than ever before. GDP records higher productivity. Investors benefit. Consumers receive lower prices. Yet workers inside the system may experience less agency over pace, scheduling, privacy, and negotiation than workers in a less efficient but more human-managed economy.

That tension illustrates the broader point behind critiques of GDP-centred AI optimism: aggregate output does not reveal how power is distributed inside productive systems.

Why convenience can hide coercion

Loss of autonomy rarely arrives as explicit authoritarianism. More often it appears as convenience.

AI assistants may handle scheduling, communication, shopping, navigation, financial planning, education, and healthcare recommendations so effectively that opting out becomes increasingly costly. Over time, the distinction between assistance and dependency can blur.

The problem is not simply that AI systems influence choices. Human institutions always influence behaviour. The issue is whether people retain:

meaningful alternatives,
visibility into how decisions are made,
the ability to contest outcomes,
ownership of their data and identity,
and genuine exit options.

If essential services become accessible only through AI-mediated systems, formal freedom may remain while practical autonomy shrinks.

A future in which citizens cannot realistically function without a small number of integrated AI platforms may still appear economically successful. Prices could fall. Services could improve. Scientific discovery could accelerate. Yet individuals may possess less ability to shape the systems governing their lives.

This is one reason many researchers distinguish between abundance and empowerment. Cheap access to goods and services does not automatically create a flourishing society if people lose influence over the institutions producing that abundance.

Lost autonomy illustration 2

AI systems can shape behaviour, not just predict it

Modern AI systems increasingly do more than classify or recommend. They optimise for behavioural outcomes.

Recommendation engines, advertising systems, engagement models, and personalised interfaces already shape what people see, buy, believe, and pay attention to. More advanced systems may become substantially better at persuasion and behavioural steering.

The EU AI Act reflects growing concern about manipulative or exploitative AI systems, including systems that distort behaviour or exploit vulnerabilities. [Artificial Intelligence Act]artificialintelligenceact.euArtificial Intelligence ActArticle 5: Prohibited AI PracticesThe EU AI Act prohibits certain uses of artificial intelligence (AI). These… [Reuters Academic work on the Act has argued that preserving informed decision-making requires clearer protections against covert manipulation and psy]reuters.comEffective fully on August 2, 2026, with some provisions starting February 2, 2025, the act bans certain abusive AI practices. Employers m…Published: August 2, 2026 chometric targeting. [arXiv]arxiv.orgarXivAI and the EU Digital Markets Act: Addressing the Risks of Bigness in Generative AIJuly 7, 2023…Published: July 7, 2023

This creates another way GDP can mislead.

An economy may become highly productive because AI systems become extraordinarily effective at directing consumption, attention, labour allocation, and user behaviour. But a society in which human choices are continuously shaped by opaque optimisation systems may not represent an expansion of freedom, even if material living standards improve.

The strongest versions of the AI bloom vision depend on the opposite outcome: AI increasing human judgement, creativity, and capability rather than replacing them with passive dependence.

Why bargaining power matters as much as income

Economic statistics often focus on wages, prices, or total output. But autonomy also depends on bargaining power.

Two societies with similar GDP per capita can feel radically different if one allows people to negotiate, organise, move between employers, and challenge institutions while the other leaves individuals dependent on systems they cannot influence.

AI could strengthen workers if productivity gains are widely shared, education becomes abundant, and people gain more flexibility and leverage. But AI could also weaken labour’s negotiating position if firms can automate tasks quickly while monitoring workers more intensively.

OECD analysis has warned that AI adoption may interact with existing inequalities and labour market asymmetries. [OECD]oecd.org287c13c4 enAlgorithmic management in the workplace6 Feb 2025 — Algorithmic management – the use of software, which may include artificial intelligen… Concerns include not only wage inequality, but also work intensity, surveillance, and asymmetrical control over data.

This distinction matters because people often tolerate difficult jobs when they possess meaningful exit options. Autonomy declines when workers cannot realistically refuse conditions imposed by increasingly automated systems.

An AI economy that delivers cheap goods but weakens labour mobility, privacy, or negotiating power may therefore increase consumption while reducing real freedom.

Measures that track freedom, security, and voice

If GDP alone misses autonomy, what should societies measure instead?

Several alternatives already exist, though none fully solves the problem.

Capability and human development measures

The capability approach associated with Amartya Sen and the UN Human Development framework focuses on what people are genuinely able to do and become rather than on output alone. In an AI context, this means asking whether people gain practical control over education, health, mobility, creativity, and civic participation.

A society where AI systems are broadly accessible, understandable, and contestable may expand capabilities even if GDP growth is modest. A highly productive but tightly controlled system may do the reverse.

Lost autonomy illustration 3

Labour quality and worker control

Measures of job quality increasingly matter in AI debates. Relevant indicators include:

control over scheduling,
workplace surveillance levels, [ainowinstitute.org]ainowinstitute.orgalgorithmic managementRestraining Workplace…11 Apr 2023 — To meaningfully build worker power, we must create policies to regulate algorithmic management tha…
ability to appeal automated decisions,
union participation,
flexibility,
psychological stress,
and protection from arbitrary dismissal.

These metrics often reveal dynamics that aggregate productivity statistics hide.

Market concentration and switching costs

Competition metrics also become important proxies for autonomy.

Questions include:

Can people easily change platforms?
Can businesses interoperate with dominant AI systems?
Are models open or closed?
Who owns the underlying data?
How dependent are governments and institutions on a few providers?

A highly concentrated AI ecosystem may create systemic dependency even if services are excellent.

Civic and democratic indicators

AI systems increasingly shape political information environments. Measures related to press freedom, institutional transparency, public trust, and democratic participation therefore become more relevant to assessing whether AI-enabled growth strengthens or weakens self-government.

The core issue is not whether AI exists inside institutions, but whether citizens retain meaningful oversight over those institutions.

The deeper AI bloom question

The debate over autonomy cuts directly into the larger AI bloom vision.

The optimistic case for advanced AI is not merely that machines will produce more wealth. It is that intelligence itself could become abundant: accelerating science, reducing scarcity, extending healthy life, and expanding what humanity can collectively achieve.

But if intelligence infrastructure becomes highly centralised and opaque, abundance alone may not secure human flourishing.

A civilisation where most people live materially comfortable lives while lacking bargaining power, privacy, meaningful political voice, or control over the systems guiding their behaviour would fall short of the stronger idea of human bloom. The long-term promise of AI depends not only on capability growth, but on whether that growth expands human agency rather than compressing it.

That is why debates over open models, worker protections, interoperability, democratic oversight, competition policy, and rights against automated decision-making are not peripheral regulatory details. They are part of the central question: whether AI-enabled abundance ultimately enlarges human freedom or merely manages it more efficiently.

Endnotes

Source: arxiv.org
Link: https://arxiv.org/abs/2308.02033
Source snippet
arXivAI and the EU Digital Markets Act: Addressing the Risks of Bigness in Generative AIJuly 7, 2023...

Published: July 7, 2023
Source: oecd.org
Link: https://www.oecd.org/en/publications/using-ai-in-the-workplace_73d417f9-en.html
Source snippet
OECDUsing AI in the workplaceTo support the adoption of trustworthy AI in the workplace, this policy paper identifies the main risks that...
Source: oecd.org
Title: algorithmic management in the workplace 287c13c4 en
Link: https://www.oecd.org/en/publications/algorithmic-management-in-the-workplace_287c13c4-en.html
Source snippet
Algorithmic management in the workplaceby A Milanez · 2025 · Cited by 48 — Algorithmic management – the use of software, which may includ...
Source: oecd.org
Title: 287c13c4 en
Link: https://www.oecd.org/content/dam/oecd/en/publications/reports/2025/02/algorithmic-management-in-the-workplace_3c84ed6d/287c13c4-en.pdf
Source snippet
Algorithmic management in the workplace6 Feb 2025 — Algorithmic management – the use of software, which may include artificial intelligen...
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC11672927/
Source snippet
PMCHow Do Algorithmic Management Practices Affect Workforce...by H Zayid · 2024 · Cited by 15 — Algorithmic management practices often r...
Source: oecd.org
Title: artificial intelligence job quality and inclusiveness a713d0ad
Link: https://www.oecd.org/en/publications/oecd-employment-outlook-2023_08785bba-en/full-report/artificial-intelligence-job-quality-and-inclusiveness_a713d0ad.html
Source snippet
Artificial intelligence, job quality and inclusiveness11 Jul 2023 — Chapter 7 discusses the risk that employers use AI-powered worker sur...
Source: reuters.com
Link: https://www.reuters.com/technology/artificial-intelligence/eu-lays-out-guidelines-misuse-ai-by-employers-websites-police-2025-02-04/
Source snippet
Effective fully on August 2, 2026, with some provisions starting February 2, 2025, the act bans certain abusive AI practices. Employers m...

Published: August 2, 2026
Source: arxiv.org
Title: arXiv Strengthening the EU AI Act: Defining Key Terms on AI Manipulation
Link: https://arxiv.org/abs/2308.16364
Source snippet
arXivStrengthening the EU AI Act: Defining Key Terms on AI ManipulationAugust 31, 2023...

Published: August 31, 2023
Source: oecd.org
Title: artificial intelligence and wage inequality bf98a45c en
Link: https://www.oecd.org/en/publications/artificial-intelligence-and-wage-inequality_bf98a45c-en.html
Source snippet
It uses a measure of occupational exposure to AI derived from that...
Source: oecd.org
Title: We work
Link: https://www.oecd.org/en.html
Source snippet
OECD: The Organisation for Economic Co-operation and...The OECD (Organisation for Economic Co-operation and Development) is a forum and...
Source: oecd.org
Link: https://www.oecd.org/en/publications/oecd-employment-outlook-2023_08785bba-en/full-report/ensuring-trustworthy-artificial-intelligence-in-the-workplace-countries-policy-action_c01b9e49.html
Source: oecd.ai
Title: The OECD Artificial Intelligence Policy Observatory
Link: https://oecd.ai/
Source snippet
It gives access to 900+ national AI policies and initiatives, live data about...
Source: oecd.ai
Link: https://oecd.ai/en/incidents/2021-04-02-6d38
Source snippet
ob displacement, meeting the criteria for a plausible future harm scenario.Read more...
Source: ainowinstitute.org
Title: algorithmic management
Link: https://ainowinstitute.org/publications/algorithmic-management
Source snippet
Restraining Workplace...11 Apr 2023 — To meaningfully build worker power, we must create policies to regulate algorithmic management tha...
Source: artificialintelligenceact.eu
Link: https://artificialintelligenceact.eu/article/5/
Source snippet
Artificial Intelligence ActArticle 5: Prohibited AI PracticesThe EU AI Act prohibits certain uses of artificial intelligence (AI). These...
Source: Wikipedia
Title: Artificial intelligence
Link: https://en.wikipedia.org/wiki/Artificial_intelligence
Source snippet
Artificial intelligenceArtificial intelligence (AI) is the capability of computational systems to perform tasks typically associated w...
Source: Wikipedia
Link: https://en.wikipedia.org/wiki/OECD
Source snippet
OECDIt is a forum whose member countries describe themselves as committed to democracy and the market economy, providing a platform to...
Source: artificialintelligenceact.eu
Link: https://artificialintelligenceact.eu/annex/3/
Source snippet
Annex III: High-Risk AI Systems Referred to in Article 6(2)This Annex lists use cases that would qualify an AI system as 'high-risk' acco...
Source: artificialintelligenceact.eu
Link: https://artificialintelligenceact.eu/article/26/
Source snippet
Before using a high-risk AI system, workers must be informed. If the system is...Read more...
Source: facebook.com
Link: https://www.facebook.com/theOECD/
Source snippet
OECD | ParisExplore our new newsletter for more on defence, the foundations for growth and competitiveness, our latest Energy Support Tra...
Source: britannica.com
Link: https://www.britannica.com/technology/artificial-intelligence
Source snippet
r-controlled robot to perform tasks commonly associated with intelligent beings.Read more...

Additional References

Source: quillbot.com
Link: https://quillbot.com/ai-chat
Source snippet
AI ChatUnlock your potential with QuillBot's free AI chat! Brainstorm, draft content, get instant research & overcome writer's block. Try...
Source: eei-institute.eu
Link: https://eei-institute.eu/wp-content/uploads/2025/11/Fact-sheet-1-AI-regulation-in-the-EU-implications-for-the-workplace-WEB-1.pdf
Source snippet
AI regulation in the EU: implications for the workplaceWith regard to high-risk AI systems, Article 26(7) of the AI Act stipulates that A...
Source: eversheds-sutherland.com
Link: https://www.eversheds-sutherland.com/de/slovakia/insights/eu-ai-act-prohibited-and-high-risk-systems-in-employment
Source snippet
EU AI Act: Prohibited and high-risk systems in employmentIn the employment context, high risk AI systems include those used for recruitme...
Source: feps-europe.eu
Link: https://feps-europe.eu/wp-content/uploads/2024/09/Algorithmic-management-in-traditional-workplaces.pdf
Source snippet
ALGORITHMIC MANAGEMENT IN THE WORKPLACEAlgorithmic Management in the Workplace. Worker privacy and surveillance... constant surveillance...
Source: ilo.org
Link: https://www.ilo.org/media/372856/download
Source snippet
The Algorithmic Management of workThis paper provides a conceptual framework for the emerging phenomenon of algorithmic management and ou...
Source: bloomberglaw.com
Link: https://www.bloomberglaw.com/external/document/X1VHEMB8000000/employment-overview-us-employers-and-eu-ai-act-faq
Source snippet
US Employers and EU AI Act, FAQEmployers should be aware that the EU AI Act imposes significant compliance obligations for high-risk AI s...
Source: europarl.europa.eu
Link: https://www.europarl.europa.eu/RegData/etudes/STUD/2025/774670/EPRS_STU%282025%29774670_EN.pdf
Source snippet
europa.euDigitalisation, artificial intelligence and algorithmic...This study investigates the integration and effects of artificial int...
Source: mckinsey.com
Link: https://www.mckinsey.com/capabilities/tech-and-ai/our-insights/superagency-in-the-workplace-empowering-people-to-unlock-ais-full-potential-at-work
Source snippet
AI in the workplace: A report for 202528 Jan 2025 — The book highlights how AI could enhance human agency and heighten our potential. It...
Source: lemonde.fr
Link: https://www.lemonde.fr/en/pixels/article/2025/02/02/artificial-intelligence-the-first-measures-of-the-european-ai-act-regulation-take-effect_6737691_13.html
Source snippet
These early measures focus on banning specific AI applications deemed unacceptable, such as social credit systems, predictive policing, e...
Source: linkedin.com
Title: rise ai powered employees boosting productivity jobs andre mxsce
Link: https://www.linkedin.com/pulse/rise-ai-powered-employees-boosting-productivity-jobs-andre-mxsce
Source snippet
The Rise of AI-Powered Employees: Boosting Productivity...A 2024 Stanford study of 5,179 knowledge workers using generative AI tools fou...