Human Judgement

Introduction

AI tutors are becoming far more capable at explaining concepts, generating practice questions, and adapting lessons to individual students. In optimistic visions of AI-enabled education, every learner could eventually receive something closer to a private tutor: personalised support available at low cost and at global scale. That possibility matters to the wider idea of AI-driven human flourishing because education is one of the main bottlenecks on scientific progress, economic mobility, and the development of human potential.

Human Judgement illustration 1 But the strongest evidence so far does not show AI replacing human educational judgement. Instead, it shows that AI works best when teachers and tutors remain responsible for interpretation, trust, emotional awareness, and the final decision about what a student actually needs. Studies of systems such as Tutor CoPilot suggest that AI can improve tutoring outcomes, especially for inexperienced tutors, while still depending heavily on human judgement in the moment. [EdWorkingPapers]edworkingpapers.comEdWorkingPapersTutor CoPilot: A Human-AI Approach for Scaling Real-Time…November 17, 2025 — by S Loeb — We introduce Tutor CoPilot, a…Published: November 17, 2025 2arXiv

This matters because fluent conversation is not the same thing as educational wisdom. A chatbot may produce plausible tutoring suggestions, yet still misunderstand whether a student is anxious, disengaged, overwhelmed, embarrassed, bored, or quietly pretending to understand. The future of high-quality education may therefore depend less on removing humans from learning and more on combining machine-scale assistance with human judgement that remains difficult to automate.

When AI Suggestions Miss the Student’s Needs

Good tutoring involves far more than delivering correct information. Human tutors constantly make situational judgements that are hard to reduce to rules or prediction models.

A student who says “I get it” may actually be confused but embarrassed. Another may need reassurance rather than another explanation. Some students benefit from challenge and faster pacing; others shut down if pushed too quickly. Tutors often infer these differences from tone, hesitation, frustration, confidence, posture, humour, or prior interactions accumulated over weeks or months.

Current AI systems remain limited at interpreting this broader context reliably.

The Tutor CoPilot research is revealing partly because it openly treats AI as advisory rather than authoritative. The system suggested prompts and pedagogical strategies during live maths tutoring sessions, but tutors still had to decide whether the advice made sense for the learner in front of them. Researchers also noted that tutors identified weaknesses in the AI’s suggestions, including responses that were not appropriate for the student’s grade level. [EdWorkingPapers]edworkingpapers.comEdWorkingPapersTutor CoPilot: A Human-AI Approach for Scaling Real-Time…November 17, 2025 — by S Loeb — We introduce Tutor CoPilot, a…Published: November 17, 2025 2arXiv

That distinction matters. Educational quality often depends on recognising when a technically correct response is still the wrong intervention.

For example, an AI tutor may:

provide too much help too early, preventing productive struggle;
misread uncertainty as ignorance;

continue explaining when a student mainly needs encouragement;
mistake politeness for understanding;
generate answers that sound authoritative but reinforce misconceptions;
optimise for conversational smoothness rather than durable learning.

Researchers working on pedagogical safety increasingly warn about these risks. The SafeTutors benchmark argues that the danger is often not dramatic failure but “quiet erosion of learning” through oversharing answers, reinforcing misunderstandings, or weakening scaffolding that helps students think independently. [arXiv]arxiv.orgarXiv Tutor Co Pilot: A Human-AI Approach for Scaling Real-Time ExpertisearXiv Tutor Co Pilot: A Human-AI Approach for Scaling Real-Time Expertise

This is one reason human tutors still matter even if AI systems become much more knowledgeable. Education is not simply information transfer. It involves judgement about timing, motivation, identity, confidence, and cognitive readiness.

Why Emotion and Trust Remain Central to Learning

One of the biggest weaknesses in fully automated tutoring is emotional interpretation.

Learning is emotionally uneven. Students become discouraged, defensive, distracted, anxious, overconfident, impatient, or ashamed. In many cases, emotional state determines whether learning happens at all.

Human tutors often notice these shifts before students explicitly articulate them. A tutor may recognise that a child suddenly going silent is not merely “processing”, but withdrawing after repeated mistakes. A teacher may sense when humour is masking embarrassment. These social judgements rely on long experience with human behaviour and on forms of contextual understanding that current AI systems only approximate.

Recent research into “emotion-aware” AI tutors exists precisely because this problem remains unresolved. Researchers studying AI tutoring dialogues found that confusion, curiosity, frustration, and emotional fluctuation are common during learning interactions, and that these emotional states can strongly affect educational outcomes. [arXiv]arxiv.orgarXiv Tutor Co Pilot: A Human-AI Approach for Scaling Real-Time ExpertisearXiv Tutor Co Pilot: A Human-AI Approach for Scaling Real-Time Expertise

But recognising emotional patterns statistically is not the same as exercising human care.

[A human tutor can decide:]edworkingpapers.comEdWorkingPapersTutor CoPilot: A Human-AI Approach for Scaling Real-Time…November 17, 2025 — by S Loeb — We introduce Tutor CoPilot, a…Published: November 17, 2025

when to abandon the lesson plan;
when a student needs a confidence boost instead of correction;
when pressure from parents, exams, bullying, or exhaustion is affecting concentration;
when to stop pushing and rebuild trust first.

These judgements are partly relational. Students often work harder for adults they trust and feel understood by. Motivation frequently depends on the belief that another person genuinely cares whether they succeed.

That interpersonal trust becomes even more important for struggling learners. Students who have repeatedly failed may interpret mistakes as evidence about their intelligence or social worth. Human tutors can respond with empathy, humour, patience, and moral encouragement in ways that remain difficult for AI systems to replicate authentically.

This does not mean AI has no role in emotional support. AI systems may eventually become useful at detecting frustration patterns, recommending pacing changes, or helping tutors notice disengagement earlier. But even optimistic researchers generally frame these tools as support systems for human educators rather than replacements for them. [ScienceDirect]sciencedirect.comScienceDirectHuman-in-the-loop in artificial intelligence in educationby B Memarian · 2024 · Cited by 77 — Understanding humans in the lo… [UNESCO]unesco.orgUNESCOBeyond the loop: Reclaiming pedagogy in an AI age2025 · Cited by 7 — AI tutors, AI writing assistants, AI feedback systems promise…

Human Judgement illustration 2

The Difference Between Conversation and Judgement

Modern language models are persuasive partly because they simulate competent conversation extremely well. In educational settings, this can create a dangerous illusion: students may feel they understand material because the interaction feels smooth and supportive.

Yet educational understanding is deeper than conversational fluency.

The OECD has warned about “false mastery”, where AI-assisted performance masks weak comprehension and reduced independent reasoning. Students may produce polished answers while understanding less than they appear to understand. [The Australian]theaustralian.com.auThe Australian AI chatbots creating 'false mastery' in students, OECD warnsThe report highlights concerns that GenAI fosters a deceptive sense of mastery among students by generating high-quality outputs that mas…

Human tutors are often better at detecting this gap.

An experienced teacher may notice:

that a student can repeat a formula but cannot apply it;
that memorised wording is hiding conceptual confusion;
that fast answers are actually copied patterns;
that confidence is disconnected from competence.

This diagnostic judgement is central to good teaching. A tutor’s job is not merely to help students finish tasks, but to decide whether genuine learning has occurred.

That becomes especially important in an AI-rich world where students may increasingly rely on tools that generate convincing outputs automatically. If education becomes too focused on polished answers, schools risk rewarding the appearance of competence instead of understanding.

The irony is that more powerful AI may increase the value of certain human educational skills rather than eliminate them. As machine-generated content becomes easier to produce, human educators may become more important as judges of reasoning quality, intellectual honesty, curiosity, persistence, and independent thought.

Human-in-the-Loop Tutoring as a Safer Model

The strongest near-term case for AI in education is therefore augmentation rather than automation.

Human-in-the-loop tutoring systems treat AI as a tool that expands a tutor’s capabilities while preserving human responsibility for judgement and relationships. In practice, this can mean:

AI-generated explanations that tutors adapt in real time;
suggested prompts that help inexperienced tutors ask better questions;
automated detection of possible misconceptions;
lesson summaries and progress tracking;
translation and accessibility support;
personalised exercises generated at scale.

The Tutor CoPilot trial showed especially strong gains among weaker tutors, suggesting AI may help distribute expert practices more widely without eliminating the human role. [EdWorkingPapers]edworkingpapers.comEdWorkingPapersTutor CoPilot: A Human-AI Approach for Scaling Real-Time…November 17, 2025 — by S Loeb — We introduce Tutor CoPilot, a…Published: November 17, 2025 2arXiv

This matters for the broader AI bloom argument because educational inequality is partly a scarcity problem. High-quality tutoring is extremely effective, but expert human attention is expensive and limited. If AI systems can help ordinary tutors behave more like excellent tutors, access to high-quality learning could expand dramatically.

Yet the same evidence also suggests caution against simplistic replacement narratives.

Brookings and other education researchers repeatedly emphasise that generative AI tutoring systems introduce concerns around pedagogical judgement, accuracy, dependency, and oversight. [Brookings]brookings.eduBrookingsWhat the research shows about generative AI in tutoring | BrookingsMary Burns unpacks the evidence of generative AI in tutoring… Human educators remain necessary not only for safety, but for interpreting context that the system cannot fully model.

This hybrid approach may ultimately prove more realistic than visions of entirely autonomous AI teachers. Human tutors provide accountability, trust, moral guidance, social understanding, and contextual interpretation. AI systems provide scale, memory, rapid adaptation, and constant availability.

Together, they may achieve something neither could accomplish alone.

Human Judgement illustration 3

Why This Distinction Matters for Human Flourishing

The difference between “AI replacing teachers” and “AI amplifying teachers” is not just a technical detail. It changes the social meaning of educational abundance.

A fully automated model risks treating learning as a narrow optimisation problem: maximise test performance at minimal cost. A human-centred model instead treats education as part of developing capable, thoughtful, socially connected people.

That distinction becomes increasingly important if advanced AI dramatically accelerates science, economic production, and technological complexity over the coming decades. In a world transformed by rapid intelligence growth, societies may need better human judgement, adaptability, and moral reasoning more than ever.

Educational systems that preserve human mentorship while expanding access through AI could help more people participate meaningfully in that future rather than becoming passive consumers of machine-generated outputs.

The optimistic case for AI in education therefore does not depend on proving that machines can replace teachers entirely. It may depend on something subtler and more achievable: using AI to make excellent human guidance less scarce while preserving the forms of judgement that still belong most naturally to people.

Endnotes

Source: edworkingpapers.com
Link: https://edworkingpapers.com/sites/default/files/ai24_1054_v2.pdf
Source snippet
EdWorkingPapersTutor CoPilot: A Human-AI Approach for Scaling Real-Time...November 17, 2025 — by S Loeb — We introduce Tutor CoPilot, a...

Published: November 17, 2025
Source: arxiv.org
Title: arXiv Tutor Co Pilot: A Human-AI Approach for Scaling Real-Time Expertise
Link: https://arxiv.org/abs/2410.03017
Source: brookings.edu
Link: https://www.brookings.edu/articles/what-the-research-shows-about-generative-ai-in-tutoring/
Source snippet
BrookingsWhat the research shows about generative AI in tutoring | BrookingsMary Burns unpacks the evidence of generative AI in tutoring...
Source: arxiv.org
Link: https://arxiv.org/html/2410.03017v2
Source snippet
Tutor CoPilot: A Human-AI Approach for Scaling Real-Time...26 Jan 2025 — We introduce Tutor CoPilot, a novel Human-AI approach that leve...
Source: arxiv.org
Title: arXiv Safe Tutors: Benchmarking Pedagogical Safety in AI Tutoring Systems
Link: https://arxiv.org/abs/2603.17373
Source snippet
arXivSafeTutors: Benchmarking Pedagogical Safety in AI Tutoring SystemsMarch 18, 2026...

Published: March 18, 2026
Source: arxiv.org
Link: https://arxiv.org/abs/2510.13862
Source snippet
arXivEnsembling Large Language Models to Characterize Affective Dynamics in Student-AI Tutor DialoguesOctober 13, 2025...

Published: October 13, 2025
Source: sciencedirect.com
Link: https://www.sciencedirect.com/science/article/pii/S2949882124000136
Source snippet
ScienceDirectHuman-in-the-loop in artificial intelligence in educationby B Memarian · 2024 · Cited by 77 — Understanding humans in the lo...
Source: unesco.org
Link: https://www.unesco.org/en/articles/beyond-loop-reclaiming-pedagogy-ai-age
Source snippet
UNESCOBeyond the loop: Reclaiming pedagogy in an AI age2025 · Cited by 7 — AI tutors, AI writing assistants, AI feedback systems promise...
Source: brookings.edu
Link: https://www.brookings.edu/articles/making-ai-work-for-schools/
Source snippet
Making AI work for schoolsJuly 31, 2025 — Recent experimental evidence underscores the opportunities AI presents: A randomized control tr...

Published: July 31, 2025
Source: brookings.edu
Title: A I’s future for students is in our hands
Link: https://www.brookings.edu/articles/ais-future-for-students-is-in-our-hands/
Source snippet
AI's future for students is in our hands - Brookings InstitutionJanuary 14, 2026 — AI can empower student learning by providing access to...

Published: January 14, 2026
Source: arxiv.org
Link: https://arxiv.org/html/2512.23633v1
Source snippet
AI tutoring can safely and effectively support students - arXivNovember 25, 2025 — Chatgpt-generated help produces learning gains equival...

Published: November 25, 2025
Source: edworkingpapers.com
Title: [PDF] What Impacts Should We Expect from Tutoring at Scale?
Link: https://edworkingpapers.com/sites/default/files/ai24-1031.pdf
Source snippet
Recent studies suggest that computer-assisted learning programs paired with tutoring (Bhatt et al., 2024) or integrated into core academi...
Source: theaustralian.com.au
Title: The Australian AI chatbots creating ‘false mastery’ in students, OECD warns
Link: https://www.theaustralian.com.au/higher-education/student-reliance-on-ai-is-a-shortcut-that-masks-a-failure-to-learn-the-oecd-warns/news-story/868d0c5769c42446ba140807e8de8fd4
Source snippet
The report highlights concerns that GenAI fosters a deceptive sense of mastery among students by generating high-quality outputs that mas...

Additional References

Source: thirdspacelearning.com
Link: https://thirdspacelearning.com/blog/ai-tutors-need-human-expertise/
Source snippet
Human in the Loop: The Key to Making AI Tutoring WorkWe believe the only effective AI tutoring is human-guided – and how schools like you...
Source: povertyactionlab.org
Link: https://www.povertyactionlab.org/evaluation/human-ai-cooperation-improve-tutoring-united-states
Source snippet
Human-AI Cooperation to Improve Tutoring in the United...The researchers introduced Tutor CoPilot, an AI program designed to improve edu...
Source: linkedin.com
Link: https://www.linkedin.com/posts/independenthead_ai-tutoring-a-really-interesting-summary-activity-7282105411833409536-cqDh
Source snippet
Mark Steed's PostStanford's Tutor CoPilot study explored AI to enhance human tutoring with nearly 1,000 students and 900 tutors. Findings...
Source: linkedin.com
Link: https://www.linkedin.com/posts/learning-agency_ai-tutors-with-a-little-human-help-offer-activity-7407098522564444160-pKLk
Source: fordhaminstitute.org
Link: https://fordhaminstitute.org/national/commentary/illusion-learning-danger-artificial-intelligence-education
Source snippet
It doesn't simply retrieve information—it makes evaluative decisions about what's relevant, what's plausible, and how to...Read more...
Source: researchgate.net
Title: 384680722 Tutor CoPilot A Human AI Approach for Scaling Real Time Expertise
Link: https://www.researchgate.net/publication/384680722_Tutor_CoPilot_A_Human-AI_Approach_for_Scaling_Real-Time_Expertise
Source snippet
(PDF) Tutor CoPilot: A Human-AI Approach for Scaling...3 Oct 2024 — We introduce Tutor CoPilot, a novel Human-AI approach that leverages...
Source: x.com
Title: lies in the thoughtful collaboration of humans and AI in the learning process
Link: https://x.com/BrookingsGlobal/status/2024945912099881280
Source snippet
What the research shows about generative AI in tutoring | BrookingsFebruary 20, 2026 — "The educational success of tutoring platforms...

Published: February 20, 2026
Source: blog.definedlearning.com
Title: why the human in the loop model is key to ethical ai in k 12 education
Link: https://blog.definedlearning.com/why-the-human-in-the-loop-model-is-key-to-ethical-ai-in-k-12-education/
Source snippet
the Human-in-the-Loop Model is Key to Ethical AI in K...30 Nov 2025 — This article explains why true success depends not on the tools th...
Source: future-ed.org
Title: Research Notes: Two Emerging Strategies for Using AI in Tutoring
Link: https://www.future-ed.org/research-notes-two-emerging-strategies-for-using-ai-in-tutoring/
Source snippet
February 17, 2026 — Two new randomized controlled trials find that AI embedded in live, chat-based math tutoring can improve student acad...

Published: February 17, 2026
Source: medium.com
Title: The Quiet Math of Ed Tech — Can AI Tutors Really Teach?
Link: https://medium.com/%40adnanmasood/the-quiet-math-of-edtech-can-ai-tutors-really-teach-737195005abf
Source snippet
The Quiet Math of EdTech — Can AI Tutors Really Teach?September 8, 2025 — A Systematic Review of Adaptive Learning and AI‑Based Tutoring...

Published: September 8, 2025