Why Experts Slow Down

Introduction

AI coding assistants are often described as early evidence for a future of “intelligence abundance”: software that can generate useful work on demand, compress years of learning, and let smaller teams achieve more. In many situations, that promise is real. Junior developers, prototyping teams, and engineers working on repetitive tasks often report substantial speed gains from tools such as GitHub Copilot, Cursor, Claude, and ChatGPT. [arXiv]arxiv.orgThis paper presents results from a controlled experiment with GitHub Copilot, an AI pair…Read more… [The GitHub Blog]github.blogThe GitHub Blogquantifying GitHub Copilot's impact on developer…Sep 7, 2022 — In our research, we saw that GitHub Copilot supports fas…

But experienced maintainers working inside mature software repositories can encounter the opposite effect. Instead of accelerating work, AI assistants may create extra review overhead, introduce subtle mistakes, and interrupt the mental models experts rely on to navigate large systems. A major 2025 study by the research organisation METR found that experienced open-source developers using frontier AI tools completed tasks about 19% more slowly than when working without them. [metr.org]metr.orgMeasuring the Impact of Early-2025 AI on Experienced…Jul 10, 2025 — Surprisingly, we find that when developers use AI tools, they take…

That result matters beyond programming. Software engineering is one of the few knowledge professions where productivity can be measured relatively directly. If AI struggles most in environments rich with tacit knowledge, historical context, and hidden constraints, that may reveal something important about the broader path toward AI-enabled abundance. Cheap code generation is not the same thing as reliable stewardship of complex systems.

What the METR trial found

The METR experiment became influential partly because it tested a setting very different from earlier AI coding benchmarks. Previous studies often used short, self-contained tasks: building a simple web server, solving programming exercises, or generating fresh code in relatively clean environments. In those cases, AI systems frequently improved completion speed. [arXiv]arxiv.orgThis paper presents results from a controlled experiment with GitHub Copilot, an AI pair…Read more…

METR instead examined real maintenance work on mature open-source repositories. Sixteen experienced developers worked on projects they already knew deeply, with an average of roughly five years of prior familiarity with their repositories. Tasks included bug fixes, feature additions, refactoring, and maintenance work typical of long-running software projects. [arXiv]arxiv.orgThis paper presents results from a controlled experiment with GitHub Copilot, an AI pair…Read more…

The headline finding surprised even the participants:

The slowdown did not come from total AI failure. The systems often produced plausible suggestions. The problem was that expert maintainers spent large amounts of time:

One widely discussed detail from the study was that fewer than half of AI-generated suggestions were accepted directly. [TechRadar]techradar.comTech Radar Using AI might actually slow down experienced devsConducted on 16 seasoned developers working on 246 tasks across familiar open-source projects, the study revealed that while developers i…

This created a paradoxical workflow. The AI reduced some low-level typing and searching, but increased cognitive overhead elsewhere. Developers became editors, validators, and supervisors of generated code rather than simply writers of code.

Why repository knowledge changes the result

The METR findings make more sense once software maintenance is viewed as a knowledge problem rather than a typing problem.

Experienced maintainers are rarely limited by how quickly they can produce syntax. Their bottleneck is understanding the invisible structure of a system: why earlier decisions were made, which conventions are sacred, where historical bugs emerged, and which parts of the codebase are fragile.

In mature repositories, much of the important knowledge is only partially written down.

Hidden conventions matter more than syntax

Large software projects accumulate years of implicit norms:

An AI assistant may generate code that looks technically correct while violating one of these unwritten expectations.

That creates a verification burden for expert maintainers. They cannot simply trust plausible output. They must inspect it carefully because they understand the cost of subtle mistakes in production systems.

The more experienced the developer, the larger this burden can become. A beginner may accept generated code because they lack the context to spot hidden problems. An expert sees the downstream risks immediately.

Repository scale overwhelms local prediction

Modern language models work by predicting likely continuations from available context. But mature repositories often contain far more information than can fit cleanly into an AI system’s active context window.

Developers therefore spend time manually supplying context, selecting files, explaining architecture, or correcting misunderstandings. GitHub itself increasingly markets “context-aware” workflows because repository knowledge is now recognised as a central limitation of coding assistants. GitHub [YouTube Even when retrieval systems improve]youtube.comYour codebase, your rules: Customizing Copilot with context…Join us to unlock the full potential of GitHub Copilot in VS Code into a d…, maintainers still face a deeper issue: some repository knowledge is social rather than textual.

For example:

These are difficult for present AI systems to infer reliably.

Maintenance work is dominated by edge cases

Prototype generation is comparatively forgiving. Maintenance is not.

Long-running software systems often exist precisely because they survived years of strange edge cases, operational failures, and user behaviour that forced gradual adaptation. Expert maintainers carry mental maps of those scars.

AI-generated code can therefore create a misleading impression of progress. The first draft appears complete quickly, but the real work begins afterward:

In these settings, rapid code generation may simply front-load future debugging work.

How perceived speed can hide measured slowdown

One of the most interesting parts of the METR study was psychological rather than technical. Developers believed they were moving faster even while objective timing showed slowdown. [arXiv]arxiv.orgThis paper presents results from a controlled experiment with GitHub Copilot, an AI pair…Read more…

That gap matters because much of the public discussion around AI productivity still depends heavily on self-reporting.

AI can reduce effort without reducing time

Several developers and commentators suggested that AI-assisted work felt easier or less mentally draining even when it took longer overall. [Reddit]reddit.comMETR finds that experienced open-source developersRedditMETR finds that experienced open-source developers…July 11, 2025 — Working with AI is slower but you spend less effort because e…Published: July 11, 2025

This distinction is important.

Writing boilerplate manually may feel tedious. Delegating it to an AI can improve the subjective experience of programming even if the developer later spends additional time reviewing and correcting outputs.

Humans are not naturally good at estimating total workflow costs when effort is redistributed across stages. Waiting for generations, reviewing outputs, and fixing subtle mistakes may feel lighter than producing code from scratch, even if the clock says otherwise.

That creates a risk for organisations measuring productivity through perception alone.

Fast feedback can create an illusion of progress

AI systems respond immediately with large volumes of plausible material. That creates a strong feeling of momentum.

But maintainers often discover problems later:

• during testing,
• code review,
• integration,
• or deployment.

The delay between generation and validation can distort intuition. Developers experience the quick initial response vividly while underestimating the cumulative cost of downstream corrections.

This is not unique to programming. Similar effects appear in other forms of knowledge work where AI systems accelerate draft production but shift more labour into verification and editing.

Experts may be especially vulnerable to overestimating gains

Counterintuitively, highly skilled developers may overestimate AI productivity gains precisely because they are already fast.

An expert maintainer can often evaluate and edit generated code quickly. That creates a sense of fluent collaboration. But if the expert could have implemented the correct solution directly in less time, the apparent speedup is illusory.

The AI becomes a detour rather than a shortcut.

Why beginners and experts see different outcomes

The slowdown effect does not mean AI coding tools are useless. It means productivity depends heavily on task type and user context.

For beginners, AI systems often function as tutors, documentation aids, and confidence boosters. They lower barriers to entry and help users perform tasks they previously could not complete at all. [arXiv]arxiv.orgThis paper presents results from a controlled experiment with GitHub Copilot, an AI pair…Read more… [The GitHub Blog]github.blogThe GitHub Blogquantifying GitHub Copilot's impact on developer…Sep 7, 2022 — In our research, we saw that GitHub Copilot supports fas…

For experts maintaining mature systems, the situation changes because:

The more specialised the task becomes, the less useful generic code prediction may be.

This distinction matters for broader debates about AI and economic abundance. Early AI success often appears strongest in domains with:

• standardised patterns,
• clear evaluation criteria,
• abundant training data,
• and low penalties for approximation.

But many high-value human activities depend on accumulated local knowledge, tacit coordination, and responsibility for long-term consequences.

Software maintenance is one of the clearest measurable examples of this tension.

The larger lesson for AI abundance

Within the broader vision of AI bloom, coding assistants are frequently treated as a preview of a future where intelligence becomes cheap, scalable, and widely available. There is truth in that vision. AI systems already help many people produce software who otherwise could not.

Yet the expert-maintainer slowdown highlights an important constraint: generating outputs is easier than integrating them into durable systems.

The distinction matters because advanced civilisation depends heavily on maintenance work:

• preserving infrastructure,
• updating institutions,
• managing technical debt,
• ensuring reliability,
• and carrying knowledge across generations.

These tasks are often invisible compared with dramatic demonstrations of creation or invention. But they are central to whether abundance becomes sustainable rather than chaotic.

The METR findings therefore do not simply challenge marketing claims about coding tools. They expose a broader issue in the path toward AI-enabled flourishing. Intelligence assistance may scale quickly in domains where correctness is loose, context is shallow, and failure costs are low. It may scale more slowly where human expertise depends on deep historical understanding, tacit coordination, and stewardship of complex systems over time. [metr.org]metr.orgMETRMETR conducts research and evaluations to improve public understanding of the capabilities and risks of frontier AI systems. Our rese… [arXiv]arxiv.orgThis paper presents results from a controlled experiment with GitHub Copilot, an AI pair…Read more…

That does not rule out future breakthroughs. AI systems may become far better at repository understanding, memory, long-horizon reasoning, and organisational context. Tooling could evolve from autocomplete systems into genuine engineering collaborators.

But the present evidence suggests an important caution for the larger AI bloom debate: abundant generated intelligence does not automatically eliminate the hard human work of judgement, maintenance, and responsibility.

Endnotes

1. Source: arxiv.org
   Link: https://arxiv.org/abs/2302.06590
   Source snippet

   This paper presents results from a controlled experiment with GitHub Copilot, an AI pair...Read more...

2. Source: github.blog
   Link: https://github.blog/news-insights/research/research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/
   Source snippet

   The GitHub Blogquantifying GitHub Copilot's impact on developer...Sep 7, 2022 — In our research, we saw that GitHub Copilot supports fas...

3. Source: arxiv.org
   Link: https://arxiv.org/abs/2406.17910

4. Source: metr.org
   Link: https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
   Source snippet

   Measuring the Impact of Early-2025 AI on Experienced...Jul 10, 2025 — Surprisingly, we find that when developers use AI tools, they take...

5. Source: arxiv.org
   Link: https://arxiv.org/abs/2507.09089
   Source snippet

   arXivMeasuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity...

6. Source: techradar.com
   Title: Tech Radar Using AI might actually slow down experienced devs
   Link: https://www.techradar.com/pro/using-ai-might-actually-slow-down-experienced-devs
   Source snippet

   Conducted on 16 seasoned developers working on 246 tasks across familiar open-source projects, the study revealed that while developers i...

7. Source: github.com
   Link: https://github.com/features/copilot
   Source snippet

   GitHubGitHub Copilot · Your AI pair programmerScale knowledge and keep teams consistent by creating a shared source of truth that include...

8. Source: youtube.com
   Link: https://www.youtube.com/watch?v=0jEzUhU8bLc
   Source snippet

   Your codebase, your rules: Customizing Copilot with context...Join us to unlock the full potential of GitHub Copilot in VS Code into a d...

9. Source: github.com
   Link: [https://github.com/orgs/community
   Source snippet

   But each developer's local editor context (open files, unsaved changes) is private. So teammates working on the same...Read more...

10. Source: reddit.com
    Title: METR finds that experienced open-source developers
    Link: https://www.reddit.com/r/slatestarcodex/comments/1lwrb09/metr_finds_that_experienced_opensource_developers/
    Source snippet

    RedditMETR finds that experienced open-source developers...July 11, 2025 — Working with AI is slower but you spend less effort because e...

    Published: July 11, 2025

11. Source: arxiv.org
    Title: arXiv Practices and Challenges of Using Git Hub Copilot: An Empirical Study
    Link: https://arxiv.org/abs/2303.08733

12. Source: arxiv.org
    Link: https://arxiv.org/abs/2509.20353

13. Source: metr.org
    Link: https://metr.org/
    Source snippet

    METRMETR conducts research and evaluations to improve public understanding of the capabilities and risks of frontier AI systems. Our rese...

14. Source: metr.org
    Link: https://metr.org/blog/2026-02-24-uplift-update/
    Source snippet

    aused a 20% slowdown in completing tasks among experienced open-source...Read more...

15. Source: github.com
    Title: Git Hub · Change is constant
    Link: https://github.com/
    Source snippet

    GitHub keeps you ahead. · GitHubFrom your first line of code to final deployment, GitHub provides AI and [automation]({{ 'ai-bloom-abun/ai-bloom-abun-98d3a6-ai-medicine-l-7c4177-clinical-seco-8e7b35-automation-bi-6899ab/' | rel...

16. Source: github.com
    Link: https://github.com/Metr

17. Source: docs.github.com
    Title: what is github copilot
    Link: https://docs.github.com/en/copilot/get-started/what-is-github-copilot
    Source snippet

    is GitHub Copilot?GitHub Copilot is an AI coding assistant that helps you write code faster and with less effort. Then, you can focus mor...

18. Source: github.com
    Link: https://github.com/teamcmcbot/github-copilot-certification
    Source snippet

    GitHub Copilot Certification Study GuideThis repository serves as a comprehensive study resource for preparing for the GitHub Copilot Cer...

19. Source: arxiv.org
    Link: https://arxiv.org/pdf/2509.20353

20. Source: arxiv.org
    Link: https://arxiv.org/html/2509.20353v2
    Source snippet

    We conducted a...Read more...

21. Source: github.blog
    Link: https://github.blog/

22. Source: github.blog
    Title: inside the research github copilot
    Link: https://github.blog/news-insights/policy-news-and-insights/inside-the-research-github-copilot/
    Source snippet

    Inside the research: How GitHub Copilot impacts the nature...20 Dec 2024 — An interview with economic researchers analyzing the causal e...

23. Source: reddit.com
    Link: https://www.reddit.com/r/ExperiencedDevs/comments/1lwk503/study_experienced_devs_think_they_are_24_faster/
    Source snippet

    Study: Experienced devs think they are 24% faster with AI...Surprisingly, we find that when developers use AI tools, they take 19% longe...

24. Source: reddit.com
    Link: https://www.reddit.com/r/LocalLLaMA/comments/1m0gvhm/study_finds_ai_tools_made_open_source_software/
    Source snippet

    AI on Experienced Open-.Read more...

25. Source: github.dev
    Title: Git Hub.dev Setting up your web editor.Read more
    Link: https://github.dev/
    Source snippet

    Setting up your web editor.Read more...

26. Source: youtube.com
    Title: Why AI Coding Tools Slow Down Experienced Developers
    Link: https://www.youtube.com/watch?v=n66V5L1L2p0
    Source snippet

    The METR Study: When AI Assistants Hinder Productivity...

27. Source: youtube.com
    Title: The METR Study: When AI Assistants Hinder Productivity
    Link: https://www.youtube.com/watch?v=F3a19Y_T1Yw
    Source snippet

    Software Maintenance and the Reality of AI Coding Tools...

28. Source: youtube.com
    Title: Software Maintenance and the Reality of AI Coding Tools
    Link: https://www.youtube.com/watch?v=Xh0Yp894P_w
    Source snippet

    Why Experts Struggle with AI Pair Programmers...

29. Source: youtube.com
    Title: Why Experts Struggle with AI Pair Programmers
    Link: https://www.youtube.com/watch?v=k9XzG0V4MhY
    Source snippet

    The Paradox of AI Productivity in Software Engineering...

30. Source: youtube.com
    Title: The Paradox of AI Productivity in Software Engineering
    Link: https://www.youtube.com/watch?v=aG4o-18p4uE

31. Source: Wikipedia
    Link: https://en.wikipedia.org/wiki/METR
    Source snippet

    METRModel Evaluation and Threat Research (METR) (MEE-tər), is a nonprofit research institute, based in Berkeley, California, that eval...

32. Source: edmondscommerce.co.uk
    Title: github copilot
    Link: https://edmondscommerce.co.uk/research/ai/github-copilot/
    Source snippet

    Productivity Research3 Dec 2025 — This analysis examines GitHub's official research on Copilot's productivity impact, focusing on control...

Additional References

1. Source: letsdatascience.com
   Link: https://letsdatascience.com/blog/developers-thought-ai-made-them-faster-the-data-said-otherwise
   Source snippet

   AI Coding Tools Made Developers 19% Slower: METR StudyA METR randomized controlled trial found AI coding tools made experienced developer...

2. Source: businessinsider.com
   Link: https://www.businessinsider.com/ai-coding-tools-may-decrease-productivity-experienced-software-engineers-study-2025-7
   Source snippet

   Conducted with 16 seasoned developers familiar with open-source projects they had worked on for years, the study randomly assigned partic...

3. Source: linkedin.com
   Link: https://www.linkedin.com/company/metr-evals
   Source snippet

   METRMETR is a research non-profit that develops evaluations to empirically test AI systems for capabilities that could threaten catastrop...

4. Source: medium.com
   Link: https://medium.com/%40svdoever/experiments-with-github-copilot-context-ca4bdcccc10e
   Source snippet

   Experiments with GitHub Copilot — contextEven given a large codebase the selection of specific files as context could help to write code...

5. Source: byteiota.com
   Link: https://byteiota.com/ai-coding-tools-19-slower-think-20-faster-metr-2026/
   Source snippet

   AI Coding Tools: 19% Slower, Think 20% Faster (METR...6 days ago — METR's February 24, 2026 research update reveals a striking productiv...

   Published: February 24, 2026

6. Source: arstechnica.com
   Title: study finds ai tools made open source software developers 19 percent slower
   Link: https://arstechnica.com/ai/2025/07/study-finds-ai-tools-made-open-source-software-developers-19-percent-slower/
   Source snippet

   AI tools made open source software 19 percent slowerJul 14, 2025 — A new randomized controlled trial has found that experienced open sour...

7. Source: hackernoon.com
   Title: github copilot adds persistent memory for repository level context
   Link: https://hackernoon.com/github-copilot-adds-persistent-memory-for-repository-level-context
   Source snippet

   GitHub Copilot Adds Persistent Memory for Repository-...2 Jan 2026 — GitHub has begun rolling out a new memory feature for Copilot, givi...

8. Source: actuia.com
   Title: a metr study reveals that ai slows down experienced developers
   Link: https://www.actuia.com/en/news/a-metr-study-reveals-that-ai-slows-down-experienced-developers/
   Source snippet

   A METR Study Reveals that AI Slows Down Experienced...Jul 16, 2025 — TLDR: A study by METR found that experienced developers were 19% s...

9. Source: forbes.com
   Title: ai safety group finds agentic tools slow down open source devs
   Link: https://www.forbes.com/sites/johnwerner/2025/09/28/ai-safety-group-finds-agentic-tools-slow-down-open-source-devs/
   Source snippet

   AI Safety Group Finds Agentic Tools Slow Down Open...Sep 28, 2025 — METR's randomized trial found AI tools slowed experienced open-sourc...

10. Source: infoworld.com
    Title: ai coding tools can slow down seasoned developers by 19
    Link: https://www.infoworld.com/article/4020931/ai-coding-tools-can-slow-down-seasoned-developers-by-19.html
    Source snippet

    %Jul 11, 2025 — Experienced developers can take 19% longer to complete tasks when using popular AI assistants like Cursor Pro and Claude...

Amazon book picks

Further Reading

Books and field guides related to Why Experts Slow Down. Use these as the next step if you want deeper reading beyond the article.

Book

Rethinking Productivity in Software Engineering

By Caitlin Sadowski, Thomas Zimmermann

First published 2019. Subjects: Software engineering.

See on Amazon

Book

AI Code Revolution with Vibe Coding

By Rajender Kumar

Build Software Through Conversation – The AI Coding Revolution Starts Here Transform from idea to working application in hours, not month...

See on Amazon

Book

Software Engineering Productivity

By C. Stevenson

Poor quality development and maintenance of software continues to cause costly errors and as the demand for new software increases so our...

See on Amazon

Book

Programming Collective Intelligence

By Toby Segaran

First published 2007. Subjects: Social aspects, Internet programming, Information technology, Social aspects of Information technology, W...

See on Amazon

Browse more on Amazon

As an Amazon Associate I earn from qualifying purchases.