When AI Tutors Work

Introduction

Early evidence suggests AI tutors can sometimes beat conventional classroom instruction on narrowly defined learning tasks. But the important detail is not simply that “AI works”. The strongest results come from systems designed around established teaching methods: guided questioning, active recall, step-by-step feedback, and adaptation to the learner’s pace. Open-ended chatbots that simply provide answers often perform far worse.

That distinction matters for the wider question of whether AI could make one-to-one learning affordable at civilisation scale. If advanced AI systems can reliably deliver parts of high-quality tutoring at very low cost, they could widen access to intellectual support far beyond elite schools and wealthy families. But the current evidence remains limited, concentrated in short-term studies, and highly dependent on careful instructional design rather than raw model capability alone.

What recent controlled trials found

The most widely discussed recent result came from a 2025 randomised controlled trial at Harvard involving university physics students. Researchers compared a custom AI tutor against active-learning classroom instruction, which is already considered substantially better than passive lecturing. The AI-supported students learned significantly more in less time, while also reporting higher engagement and motivation. The study’s authors stressed that the tutor was deliberately designed around established pedagogical practices rather than unrestricted chatbot conversation. [Nature]nature.comNatureAI tutoring outperforms in-class active learningby G Kestin · 2025 · Cited by 175 — We find that students learn significantly more…

That result attracted attention partly because the comparison was demanding. Active learning classrooms already use discussion, problem-solving, and participation instead of traditional lectures. The finding therefore was not merely that AI beat old-fashioned rote teaching. It suggested that carefully engineered AI tutoring might reproduce some benefits normally associated with personalised human instruction.

Other emerging studies point in the same direction, though with more mixed outcomes.

Research on Khan Academy’s GPT-4-based tutor Khanmigo found that students often valued its step-by-step guidance, conversational support, and personalised pacing. In some studies, students using it showed meaningful learning gains, while in others outcomes were similar to conventional methods despite positive learner experiences. [ResearchGate]researchgate.netResearchGate(PDF) Leveraging “Khanmigo” Generative AI-Powered Tool…The study's findings suggest that while GenAI-powered tutoring syst… [2Érudit]erudit.orgÉrudit RecordLeveraging “Khanmigo” Generative AI-Powered…by N Slijepcevic · 2025 · Cited by 1 — Qualitative findings indicated that students percei…

Large-scale observational evidence around Khan Academy usage also suggests that structured AI-assisted practice can correlate with improved academic performance, though such studies are weaker than randomised trials because motivated students may naturally use learning tools more often. [Khan Academy Blog]blog.khanacademy.orgKhan Academy BlogStudy in Top Journal Shows Khan Academy Learning Gains21 Jan 2026 — A peer-reviewed study finds Khan Academy leads to be…

A separate line of research on hybrid human-AI tutoring found that AI support systems helped less experienced human tutors perform more like stronger tutors. Carnegie Mellon and collaborators reported gains in proficiency and engagement in low-income school settings when AI systems augmented human tutoring rather than replacing it entirely. [arXiv]arxiv.orgarXivImproving Student Learning with Hybrid Human-AI Tutoring: A Three-Study Quasi-Experimental InvestigationDecember 18, 2023…Published: December 18, 2023

Some distance-learning research also suggests that AI tutoring systems can substantially reduce study time while maintaining learning outcomes. One 2024 university study reported that students using an AI teaching assistant completed learning tasks roughly 27% faster on average. [arXiv]arxiv.orgarXivImproving Student Learning with Hybrid Human-AI Tutoring: A Three-Study Quasi-Experimental InvestigationDecember 18, 2023…Published: December 18, 2023

Taken together, the evidence does not yet prove that AI tutoring broadly surpasses classroom teaching. But it does support a narrower claim: under controlled conditions, with carefully structured systems and specific academic material, AI tutoring can outperform at least some standard instructional formats.

Why structured tutoring beats open chatbot use

The biggest lesson from recent evidence is that pedagogy matters more than conversational fluency.

A raw large language model is often a poor teacher. Left unconstrained, it may:

Several studies now suggest that unrestricted chatbot use can actively harm learning. Researchers at the University of Pennsylvania and collaborators found that students using generative AI for mathematics exam preparation often performed worse despite appearing more productive during practice. Many students copied solutions instead of building understanding. [PMC]pmc.ncbi.nlm.nih.govPMCGenerative AI without guardrails can harm learningKhanmigo (23), a GPT-4 based tutoring application. Our findings support both the need for educators to find ways to safeguard student lea…

This helps explain why the strongest AI tutor systems increasingly resemble structured teaching environments rather than free-form chat interfaces.

The successful systems usually include features such as:

• Socratic questioning instead of direct answers
• mastery learning, where students must demonstrate understanding before advancing
• spaced repetition and retrieval practice
• deliberate misconception correction
• guided hints rather than solution dumping
• continuous adaptation to learner difficulty
• monitoring for confusion or disengagement

The Harvard physics tutor, for example, was built around educational research principles already used in high-performing active-learning classrooms. [Nature]nature.comNatureAI tutoring outperforms in-class active learningby G Kestin · 2025 · Cited by 175 — We find that students learn significantly more…

Khanmigo similarly attempts to behave more like a tutor than an answer engine. It often prompts students to explain reasoning or solve intermediate steps themselves. [Automated Education]automated.educationwhat research says about ai tutoring8 Nov 2024 — Khan Academy's Khanmigo is one of the most visible LLM-based tutors in schools. It uses a large language model constrained b…

Even OpenAI’s later “study mode” features reflect this shift toward constrained tutoring design. The system tries to guide learners through questions instead of immediately providing completed answers. [WIRED]wired.comChat GPT's Study Mode Is HereIt Won't Fix Education's AI ProblemsJuly 29, 2025 — OpenAI has introduced a new "study mode" for ChatGPT aimed at reducing students' depe…Published: July 29, 2025

This is important because the optimistic vision for AI-enabled education depends less on raw intelligence than on scalable instructional discipline. A superhuman conversational model that constantly shortcuts student thinking may produce weaker education than a narrower system carefully engineered to support learning.

The real comparison is not against great teachers

One common misunderstanding is that AI tutors are being evaluated against the very best human teachers working one-to-one with motivated students.

In reality, much of the comparison is against educational scarcity.

Many pupils today receive:

Under those conditions, a competent AI tutor available at any hour may outperform ordinary educational reality for many learners even if it remains inferior to excellent human teaching.

This matters especially in areas where tutoring is expensive or rare. Wealthy families already buy forms of one-to-one educational attention through private tutors, smaller classes, intensive coaching, and enrichment programmes. AI tutoring potentially lowers the cost of personalised cognitive support enough to reach much larger populations.

That possibility links the topic directly to the broader idea of AI-enabled abundance. If intelligence itself becomes cheaper to distribute, educational inequality could narrow rather than widen — though this outcome is far from guaranteed.

The same systems could also help adult retraining, language learning, disability support, and education in regions with teacher shortages. A learner who currently receives almost no tailored academic support may benefit enormously from even moderately capable AI guidance.

What the evidence still cannot prove

The strongest claims about AI tutoring remain far ahead of the evidence.

Most existing studies are:

Researchers still do not know whether current AI tutors reliably improve:

There are also unresolved concerns about dependency. Students may become reliant on constant AI guidance instead of learning persistence and self-direction. Some evidence already suggests that easy AI assistance can reduce productive struggle, which is often essential for durable learning. [PMC]pmc.ncbi.nlm.nih.govPMCGenerative AI without guardrails can harm learningKhanmigo (23), a GPT-4 based tutoring application. Our findings support both the need for educators to find ways to safeguard student lea… [WIRED Another major uncertainty is social development. Schools do far more than transfer information. They provide peer interaction]wired.comChat GPT's Study Mode Is HereIt Won't Fix Education's AI ProblemsJuly 29, 2025 — OpenAI has introduced a new "study mode" for ChatGPT aimed at reducing students' depe…Published: July 29, 2025, emotional development, institutional structure, and exposure to different personalities and viewpoints. Even highly effective AI tutoring does not automatically replace those functions.

There are practical risks too:

The political economy matters. AI tutoring could widen educational opportunity, but it could also deepen inequality if elite schools combine human mentorship with advanced AI while poorer systems receive only automated instruction.

Why this still matters for the long-term future

Even cautious findings matter because education compounds across generations.

If AI systems eventually provide affordable, high-quality intellectual support to billions of people, the long-term effects could extend far beyond better homework completion. Wider access to personalised learning could increase scientific participation, technical capability, literacy, and problem-solving across entire societies.

Historically, civilisations often wasted enormous human potential simply because many people never received sustained education or mentoring. AI tutoring raises the possibility that cognitive support itself could become far more abundant.

That does not mean classrooms disappear. The more plausible near-term future is hybrid education:

The strongest current evidence points toward this hybrid model rather than full automation. Studies increasingly suggest that AI works best when embedded inside carefully designed educational systems with human oversight and clear pedagogical goals. [heinz.cmu.edu]heinz.cmu.eduIn Study of Human-AI Tutoring with U.SSeventh Graders…Students tutored in the program outperformed students tutored by AI alone. The study was conducted by researchers at C…

For the broader AI bloom question, that may be the more important lesson. Human flourishing is unlikely to come from AI acting alone. It is more likely to emerge from systems that combine machine scale with human values, institutions, and educational wisdom accumulated over generations.

Endnotes

1. Source: nature.com
   Link: https://www.nature.com/articles/s41598-025-97652-6
   Source snippet

   NatureAI tutoring outperforms in-class active learningby G Kestin · 2025 · Cited by 175 — We find that students learn significantly more...

2. Source: researchgate.net
   Link: https://www.researchgate.net/publication/396808798_Leveraging_Khanmigo_Generative_AI-Powered_Tool_for_Personalized_Tutoring_to_Learn_Scientific_Concepts
   Source snippet

   ResearchGate(PDF) Leveraging “Khanmigo” Generative AI-Powered Tool...The study's findings suggest that while GenAI-powered tutoring syst...

3. Source: arxiv.org
   Link: https://arxiv.org/abs/2312.11274
   Source snippet

   arXivImproving Student Learning with Hybrid Human-AI Tutoring: A Three-Study Quasi-Experimental InvestigationDecember 18, 2023...

   Published: December 18, 2023

4. Source: arxiv.org
   Link: https://arxiv.org/abs/2403.14642
   Source snippet

   arXivRevolutionising Distance Learning: A Comparative Study of Learning Progress with AI-Driven TutoringFebruary 21, 2024...

   Published: February 21, 2024

5. Source: pmc.ncbi.nlm.nih.gov
   Title: PMCGenerative AI without guardrails can harm learning
   Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC12232635/
   Source snippet

   Khanmigo (23), a GPT-4 based tutoring application. Our findings support both the need for educators to find ways to safeguard student lea...

6. Source: news.harvard.edu
   Title: Gazette Professor tailored AI tutor to physics course
   Link: https://news.harvard.edu/gazette/story/2024/09/professor-tailored-ai-tutor-to-physics-course-engagement-doubled/
   Source snippet

   5, 2024 — A Harvard study examining learning outcomes for students in a large, popular physics course who worked with a...

7. Source: automated.education
   Title: what research says about ai tutoring
   Link: https://automated.education/en-gb/blog/2024/11/08/what-research-says-about-ai-tutoring/
   Source snippet

   8 Nov 2024 — Khan Academy's Khanmigo is one of the most visible LLM-based tutors in schools. It uses a large language model constrained b...

8. Source: wired.com
   Title: Chat GPT’s Study Mode Is Here
   Link: https://www.wired.com/story/chatgpt-study-mode
   Source snippet

   It Won't Fix Education's AI ProblemsJuly 29, 2025 — OpenAI has introduced a new "study mode" for ChatGPT aimed at reducing students' depe...

   Published: July 29, 2025

9. Source: heinz.cmu.edu
   Title: In Study of Human-AI Tutoring with U.S
   Link: https://www.heinz.cmu.edu/media/2025/September/in-study-of-human-ai-tutoring-with-us-seventh-graders-human-tutors-enhanced-the-benefits-of-ai-tutors
   Source snippet

   Seventh Graders...Students tutored in the program outperformed students tutored by AI alone. The study was conducted by researchers at C...

10. Source: researchgate.net
    Link: https://www.researchgate.net/publication/392839220_AI_tutoring_outperforms_in-class_active_learning_an_RCT_introducing_a_novel_research-based_design_in_an_authentic_educational_setting
    Source snippet

    Chan, Lo...Read more...

11. Source: researchgate.net
    Title: (PDF) AI Tutoring Outperforms Active Learning
    Link: https://www.researchgate.net/publication/380587627_AI_Tutoring_Outperforms_Active_Learning
    Source snippet

    May 14, 2024 — We find that students learn more than twice as much in less time when using an AI tutor, compared with the active learning...

    Published: May 14, 2024

12. Source: researchgate.net
    Link: https://www.researchgate.net/publication/393070023_Khanmigo_in_the_Virtual_Classroom_A_Strategic_Evaluation_through_SWOT_and_Acceptability_Analysis
    Source snippet

    (PDF) Khanmigo in the Virtual Classroom: A Strategic...23 Dec 2025 — Approximately 140 students were registered in the pertinent course...

13. Source: arxiv.org
    Link: https://arxiv.org/html/2503.02885v2
    Source snippet

    “Would You Want an AI Tutor?” Understanding Stakeholder...9 Jun 2025 — As a launch partner, Khan Academy had early access to the model...

14. Source: gse.harvard.edu
    Title: ai can add not just subtract learning
    Link: https://www.gse.harvard.edu/ideas/news/25/04/ai-can-add-not-just-subtract-learning
    Source snippet

    Can Add, Not Just Subtract, From Learning8 Apr 2025 — The key question is whether AI improves learning or developmental outcomes compared...

15. Source: erudit.org
    Title: Érudit Record
    Link: https://www.erudit.org/en/journals/jtl/2025-v19-n4-jtl010465/1122080ar/abstract/
    Source snippet

    Leveraging “Khanmigo” Generative AI-Powered...by N Slijepcevic · 2025 · Cited by 1 — Qualitative findings indicated that students percei...

16. Source: blog.khanacademy.org
    Link: https://blog.khanacademy.org/national-study-in-top-journal-finds-khan-academy-learning-gains-after-accounting-for-key-unmeasured-factors/
    Source snippet

    Khan Academy BlogStudy in Top Journal Shows Khan Academy Learning Gains21 Jan 2026 — A peer-reviewed study finds Khan Academy leads to be...

17. Source: blog.khanacademy.org
    Title: khan academy efficacy results november 2024
    Link: https://blog.khanacademy.org/khan-academy-efficacy-results-november-2024/
    Source snippet

    Khan Academy BlogKhan Academy Efficacy Results, November 20246 Dec 2024 — How student learning gains change year to year as students incr...

    Published: november 2024

18. Source: blog.khanacademy.org
    Title: Their secret? Khanmigo, an AI tutor and teaching assistant.Read more
    Link: https://blog.khanacademy.org/how-enid-high-school-transformed-their-math-classrooms-with-ai-a-case-study/
    Source snippet

    Enid High School Transformed Their Math...16 Jan 2025 — By embracing AI in their math classrooms, they've seen a remarkable increase in...

Additional References

1. Source: linkedin.com
   Link: https://www.linkedin.com/posts/yasindahi_a-randomized-controlled-trial-from-harvard-activity-7386771107363745792-ow3V
   Source snippet

   AI tutors outperform active learning in educationIt showed that a well-designed, human-built AI tutor helped students learn short lessons...

2. Source: axios.com
   Link: https://www.axios.com/local/san-francisco/2024/08/22/ai-tutor-bay-area-classrooms
   Source snippet

   The study, involving nearly 1,000 students from grades 9 to 11 over four 90-minute sessions, found that while genAI can help with practic...

3. Source: linkedin.com
   Link: https://www.linkedin.com/posts/joseantoniobowen_ai-tutoring-outperforms-in-class-active-learning-activity-7363250307910291456-i8yy

4. Source: hellopraxis.com
   Link: https://www.hellopraxis.com/en/praxisnotes/ai-tutors-outperform-traditional-teaching-methods-in-groundbreaking-harvard-study
   Source snippet

   AI Tutors Outperform Traditional Teaching Methods in...2 Dec 2024 — Students using AI tutors learned more than twice as much compared to...

5. Source: quantumzeitgeist.com
   Title: ai tutor improves physics education outcomes by 10 in randomized trial
   Link: https://quantumzeitgeist.com/ai-tutor-improves-physics-education-outcomes-by-10-in-randomized-trial/
   Source snippet

   AI Tutor Improves Physics Education Outcomes By 10%3 Apr 2025 — A randomized controlled trial with 165 students found that an AI Peer des...

6. Source: scale.stanford.edu
   Title: creating customisable freely accessible socratic ai physics tutor
   Link: https://scale.stanford.edu/ai/repository/creating-customisable-freely-accessible-socratic-ai-physics-tutor
   Source snippet

   a customisable freely-accessible Socratic AI physics...8 Jul 2025 — We demonstrate this methodology by designing a Socratic physics prob...

7. Source: rickhess99.medium.com
   Title: can an ai powered tutor produce meaningful results b67d7376cb51
   Link: https://rickhess99.medium.com/can-an-ai-powered-tutor-produce-meaningful-results-b67d7376cb51
   Source snippet

   an AI-Powered Tutor Produce Meaningful Results?We went from about 68,000 Khanmigo student and teacher users in our partner school distric...

8. Source: chalkbeat.org
   Title: The district approved a data-sharing agreement with the online
   Link: https://www.chalkbeat.org/newark/2024/05/13/artificial-intelligence-khanmigo-chatbot-tutor-pilot-testing-districtwide-expansion/
   Source snippet

   Newark Public Schools wants new AI tutor after pilot testingMay 13, 2024 — Newark intends to expand an AI tutoring program developed by K...

   Published: May 13, 2024

9. Source: joshthompson.co.uk
   Title: ai tutors harvard study doubled learning gains uk education
   Link: https://joshthompson.co.uk/ai/ai-tutors-harvard-study-doubled-learning-gains-uk-education/
   Source snippet

   AI Tutors vs Classrooms: Inside the Harvard Study That...Jan 11, 2026 — In the study (N=194), physics students using an AI tutor outperf...

10. Source: edweek.org
    Title: opinion can an ai powered tutor produce meaningful results
    Link: https://www.edweek.org/technology/opinion-can-an-ai-powered-tutor-produce-meaningful-results/2025/07
    Source snippet

    Education WeekCan an AI-Powered Tutor Produce Meaningful Results?29 Jul 2025 — Khanmigo has been described as a “personal tutor and teach...

Amazon book picks

Further Reading

Books and field guides related to When AI Tutors Work. Use these as the next step if you want deeper reading beyond the article.

Book

Intelligent Tutoring Systems

By Stefan Trausan-Matu, Kristy Boyer et al.

First published 2014. Subjects: Computer-assisted instruction, Artificial Intelligence (incl. Robotics), Multimedia systems, Computer sci...

See on Amazon

Book

Artificial Intelligence in Education

By Ulrich Hoppe, Maria Feliza Verdejo et al.

This work reports on research into intelligent systems, models, and architectures for educational computing applications. It covers a wid...

See on Amazon

Book

Advances in intelligent tutoring systems

By Roger Nkambou, Jacqueline Bourdeau et al.

First published 2010. Subjects: Educational applications, Artificial intelligence, Intelligent tutoring systems, Computer-assisted instru...

See on Amazon

Book

Intelligent Tutoring Systems

By Gilles Gauthier, Claude Frasson et al.

ITS 2000 is the fifth international conference on Intelligent Tutoring Systems. The preceding conferences were organized in Montreal in 1...

See on Amazon

Browse more on Amazon

As an Amazon Associate I earn from qualifying purchases.