Can students still solve it alone?

Introduction

The most important question about an AI tutor is not whether students can finish today’s worksheet. It is whether they can solve a related problem next week without the AI beside them.

That distinction sits at the centre of the debate over AI tutors and human flourishing. If advanced AI is eventually able to provide personalised education at global scale, the prize is not simply faster homework completion. The larger hope is cognitive empowerment: helping billions of people develop skills, knowledge and judgement that remain useful when technology is unavailable, wrong or absent. The challenge is that AI-assisted performance can look impressive even when genuine learning has not improved.

Researchers increasingly distinguish between performance during AI use and transfer of learning. Transfer means that knowledge gained in one context carries over into new situations: different questions, delayed tests, unfamiliar problems or independent work. An AI tutor that raises scores only while it is present may be functioning more like a calculator for thought than a teacher. The strongest evidence therefore comes from studies that remove the AI and ask a simple question: can the student still do it alone?[pnas.org]pnas.orgGenerative AI without guardrails can harm learningby H Bastani · 2025 · Cited by 198 — Our research examines the impact of generative AI…

Why completed homework is a weak signal

A completed assignment often tells educators less than it appears.

Traditional education research has long recognised that students can produce correct answers without developing durable understanding. Memorisation, copying, excessive hints or step-following can all create the appearance of mastery. Generative AI introduces a far more powerful version of the same problem because it can generate entire solution paths, explanations and worked examples on demand.

This creates what researchers sometimes call a measurement problem. If students practise mathematics with an AI system and their homework accuracy rises dramatically, there are at least two possible explanations:

Those possibilities can look identical in assignment data.

A large field experiment in Turkish high-school mathematics illustrates the issue. Students using GPT-4 completed substantially more practice problems correctly than students working alone. Yet when the AI was removed and students sat an independent exam, the pattern reversed. Students who had relied on unrestricted GPT-4 performed significantly worse than the control group. The researchers concluded that improved practice performance was not a reliable measure of actual learning.[pnas.org]pnas.orgGenerative AI without guardrails can harm learningby H Bastani · 2025 · Cited by 198 — Our research examines the impact of generative AI…[PMC]pmc.ncbi.nlm.nih.govPMCGenerative AI without guardrails can harm learningNIHby H Bastani · 2025 · Cited by 187 — These results suggest that while access to generative AI can improve performance, it can su…

This matters because educational technology often markets itself using engagement statistics, completion rates or immediate performance gains. Those metrics are easier to collect than independent learning outcomes. But if AI bloom is partly a story about making intelligence and expertise more widely available, then the relevant measure is whether human capability grows. A system that generates correct homework while weakening independent problem-solving would represent a very different future from one that genuinely expands human competence.

Independent post-tests are the real benchmark

The strongest transfer studies deliberately separate learning from assistance.

The most common method is an independent post-test. Students practise with whatever tool is being evaluated, but later complete new problems without access to the system. Researchers then compare performance against students who learned through other methods.

The high-school mathematics experiment led by Hamsa Bastani and colleagues used exactly this design. During practice sessions, unrestricted GPT-4 boosted performance dramatically. However, students in the unrestricted AI condition later scored about 17% worse on independent exams than students who had never received AI assistance. The authors argued that many students had learned to rely on the system rather than developing the underlying mathematical skills themselves.[pnas.org]pnas.orgGenerative AI without guardrails can harm learningby H Bastani · 2025 · Cited by 198 — Our research examines the impact of generative AI…[PMC]pmc.ncbi.nlm.nih.govPMCGenerative AI without guardrails can harm learningNIHby H Bastani · 2025 · Cited by 187 — These results suggest that while access to generative AI can improve performance, it can su…

The same study tested a different version called GPT Tutor. Instead of freely giving answers, it used guardrails intended to encourage reasoning, hints and stepwise progress. The negative transfer effect largely disappeared. Students still did not show large learning gains relative to the control group, but the damage seen in the unrestricted condition was substantially reduced.[SSRN]papers.ssrn.comGenerative AI Without Guardrails Can Harm Learningby H Bastani · 2024 · Cited by 365 — Without guardrails, students attempt to use GP… PubMed This finding is easy to miss but important. The study was not merely comparing AI against no AI. It was comparing different designs of AI ass[pubmed.ncbi.nlm.nih.gov]pubmed.ncbi.nlm.nih.govAI without guardrails can harm learningby H Bastani · 2025 · Cited by 90 — Without guardrails, students attempt to use GPT-4 as a "crutch… istance. The results suggest that transfer depends not only on model capability but on how the tutor structures the interaction.

For educational systems, this changes the engineering goal. Success is not simply maximising immediate student performance. It is designing interactions that produce better independent performance later.

Delayed tests reveal whether learning lasts

Immediate post-tests can still overestimate learning.

Students sometimes retain enough short-term familiarity to perform well shortly after studying even when deeper understanding remains weak. Educational researchers therefore often use delayed assessments given days or weeks later.

A randomised controlled trial examining ChatGPT as a study aid in higher education tested retention after a 45-day delay. Students who used ChatGPT while learning AI-related course material later scored significantly lower on the surprise retention test than students who used traditional study methods. The gap suggested that some of the apparent learning benefits observed during study did not transfer into durable knowledge. Researchers linked the effect to reduced cognitive effort during learning.[ScienceDirect]sciencedirect.comChatGPT as a cognitive crutch: Evidence from…by A Barcaui · 2025 · Cited by 39 — This study addresses this critical gap t…

The logic behind delayed testing is straightforward. Learning that survives time is usually more valuable than learning that survives only until tomorrow’s assignment deadline.

This principle becomes especially important in an AI-rich world. Future workers may have powerful systems available most of the time, but not necessarily in every circumstance. They may need to verify outputs, recognise errors, adapt knowledge to new contexts or continue functioning during outages and failures. Long-term retention remains economically and socially valuable even when AI assistance becomes abundant.

For that reason, researchers increasingly argue that educational AI should be evaluated using delayed transfer measures rather than only immediate performance metrics.[ScienceDirect]sciencedirect.comChatGPT as a cognitive crutch: Evidence from…by A Barcaui · 2025 · Cited by 39 — This study addresses this critical gap t…

What counts as genuine transfer?

A student does not need to solve the exact same question again to demonstrate learning.

Researchers usually look for several forms of transfer:

The further the transfer, the stronger the evidence that genuine understanding has developed.

For example, a student who learns algebra through an AI tutor might first be tested on similar equations without assistance. Later they might face word problems requiring the same underlying concepts but presented differently. Strong performance across both tasks provides much better evidence of learning than successful completion of AI-assisted homework.

This is one reason many researchers remain cautious about claims that AI tutoring has already solved education. Demonstrating transfer is harder than demonstrating task completion. It requires carefully designed assessments that separate what the student knows from what the system knows.

Warning signs of AI dependence

Several patterns repeatedly appear when AI support is helping performance more than learning.

Students ask for answers before attempting solutions

In the mathematics field experiment, researchers found many students using GPT-4 as a shortcut rather than a tutor. Instead of requesting conceptual help, they often sought direct solutions. When that strategy succeeded, later independent performance suffered.[SSRN]papers.ssrn.comGenerative AI Without Guardrails Can Harm Learningby H Bastani · 2024 · Cited by 365 — Without guardrails, students attempt to use GP…

The problem is not merely dishonesty. Even well-intentioned students often choose the path of least resistance when deadlines, grades and frustration are involved. An AI system that instantly reveals answers can therefore undermine the productive struggle that contributes to learning.

Confidence rises faster than competence

One risk of fluent AI explanations is that they create an illusion of understanding.

Students may feel they understand a concept because the explanation sounded clear, even if they could not reproduce the reasoning independently. Researchers studying AI-assisted learning have repeatedly highlighted this gap between perceived mastery and demonstrated mastery.[LinkedIn]linkedin.comWithout Guardrails, Generative AI Can Harm EducationKey Takeaways Students performed better in practice sessions with gen AI, but…

This makes independent testing especially important. Self-reports of confidence are often a poor substitute for transfer assessments.

Performance collapses when assistance disappears

The clearest warning sign is simple: scores fall sharply once AI access is removed.

This pattern appeared in the high-school mathematics study and in later work examining retention and independent problem-solving. Students often perform well during AI-supported practice but struggle when required to solve related problems alone.[PMC]pmc.ncbi.nlm.nih.govPMCGenerative AI without guardrails can harm learningNIHby H Bastani · 2025 · Cited by 187 — These results suggest that while access to generative AI can improve performance, it can su…[PsyPost -]psypost.orgPsy PostPsychology NewsUnrestricted generative AI harms high school math…Apr 21, 2026 — With AI access, students scored 48% higher o…

When this happens, the educational system may be measuring tool use rather than learning.

Why this matters for the larger AI bloom vision

The optimistic case for AI in education is unusually ambitious.

Advocates do not merely hope that students will finish assignments faster. They imagine a future where personalised instruction becomes available to anyone with an internet connection; where tutoring no longer depends on wealth; where people can continuously acquire new skills throughout life; and where educational opportunity expands on a civilisational scale.

That vision depends on transfer.

If AI tutors genuinely help people acquire knowledge that remains usable independently, then they could become one of the strongest mechanisms for spreading human capability. They could help learners in under-resourced schools, adults changing careers, people studying in remote regions and populations historically excluded from high-quality tutoring.

But if AI systems mainly increase assisted performance while leaving underlying skills unchanged, the long-term picture becomes much less transformative. Intelligence would be outsourced rather than expanded. Apparent gains in productivity could mask growing dependence on external systems.

This is why independent post-tests, delayed assessments and transfer measures matter so much. They are not merely technical details in education research. They are among the clearest ways to distinguish between two very different futures: one in which AI helps humans become more capable, and one in which it mainly becomes a cognitive crutch.

The real test of an AI tutor is therefore remarkably old-fashioned. After the lesson ends, after the hints disappear and after the chatbot window closes, can the learner still think through the problem alone? The answer to that question may determine whether AI education becomes a pathway to broader human flourishing or merely a more sophisticated form of homework completion.

Amazon book picks

Further Reading

Books and field guides related to Can students still solve it alone?. Use these as the next step if you want deeper reading beyond the article.

Book

Make It Stick

By Peter C. Brown, Henry L. Roediger III et al.

Directly addresses whether learners can perform later without support.

See on Amazon

Book

Why Don't Students Like School?

By Daniel T. Willingham

Explains the cognitive basis of independent understanding.

See on Amazon

Book

How Learning Happens

By Paul A. Kirschner, Carl Hendrick

Explains transfer, practice and memory mechanisms.

See on Amazon

Book

Visible Learning for Teachers

By John Hattie

Helps frame how to measure learning beyond completed tasks.

See on Amazon

Browse more on Amazon:Make It StickWhy Don't Students Like School?How Learning Happens

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Example marketplace items related to this page. Use the search link to explore similar finds on eBay.

Shop location

UsingUSA USA USAUKCanadaAustraliaIreland

Example eBay listing

Drawing Robot For Kids With 100 Cards Automatic Painting Robot Education Toy UK

SearcheBay.co.uk: educational robot

Browse similar oneBay.co.uk

Example eBay listing

Kids Drawing Robot Educational Drawing Machine Voice Interaction Learning Toy

SearcheBay.co.uk: educational robot

Browse similar oneBay.co.uk

Example eBay listing

TTS Blue-Bot Educational Bluetooth Robot + USB Cable - Fully Tested

SearcheBay.co.uk: educational robot

Browse similar oneBay.co.uk

Example eBay listing

RED5 14 In 1 Educational Solar Robot Kit STEM Science DIY Build Toy New Open Box

SearcheBay.co.uk: educational robot

Browse similar oneBay.co.uk

Browse more oneBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: pnas.org
Link:https://www.pnas.org/doi/10.1073/pnas.2422633122

2. Source: pmc.ncbi.nlm.nih.gov
Title: PMCGenerative AI without guardrails can harm learning
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC12232635/

3. Source: papers.ssrn.com
Link:https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4895486

4. Source: sciencedirect.com
Link:https://www.sciencedirect.com/science/article/pii/S2590291125010186

5. Source: linkedin.com
Link:https://www.linkedin.com/pulse/without-guardrails-generative-ai-can-harm-education-wkife

6. Source: psypost.org
Title: Psy Post
Link:https://www.psypost.org/unrestricted-generative-ai-harms-high-school-math-learning-by-acting-as-a-crutch/

7. Source: linkedin.com
Link:https://www.linkedin.com/posts/hamsa-bastani-4a346955_generative-ai-without-guardrails-can-harm-activity-7343667696540033025-a1Ms

8. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/40560616/

Additional References

9. Source: researchgate.net
Link:https://www.researchgate.net/publication/398113409_ChatGPT_as_a_cognitive_crutch_Evidence_from_a_randomized_controlled_trial_on_knowledge_retention

Published: May 2026

10. Source: thirdspacelearning.com
Link:https://thirdspacelearning.com/blog/intelligent-tutoring-systems/

11. Source: nxgl.ai
Link:https://nxgl.ai/post/mastery-and-scale-in-education-ai-tutors-as-an-integrated-part-of-an-online-learning-experience

12. Source: facebook.com
Link:https://www.facebook.com/groups/703007927897194/posts/1052128856318431/

13. Source: evelynlearning.com
Link:https://www.evelynlearning.com/blog/the-socratic-method-meets-machine-learning-how-ai-tutoring-tools-are-teaching-students-to-think-not-just-answer

14. Source: chibe.upenn.edu
Link:https://chibe.upenn.edu/publications/generative-ai-without-guardrails-can-harm-learning-evidence-from-high-school-mathematics/

15. Source: researchgate.net
Title: 390576465 AI Tutors in Higher Education Comparing Expectations to Evidence
Link:https://www.researchgate.net/publication/390576465_AI_Tutors_in_Higher_Education_Comparing_Expectations_to_Evidence

16. Source: researchgate.net
Link:https://www.researchgate.net/publication/394273654_GPT-4_as_a_Homework_Tutor_Can_Improve_Student_Engagement_and_Learning_Outcomes

17. Source: youtube.com
Link:https://www.youtube.com/watch?v=cxozGuNOVv8

18. Source: hamsabastani.github.io
Link:https://hamsabastani.github.io/education_llm.pdf