AI tutors: what the 2024–2026 randomized trials show

The randomized trials of AI tutors so far split cleanly. Tutors built around a fixed task, with the solution written in advance, one step at a time and a human nearby, produced real gains (0.31 SD in Nigeria; a larger effect in a two-lesson Harvard study). An open chat window added to existing practice did not: in a two-year trial in 18 schools, Khanmigo's AI layer added no measurable benefit. Peabody (peabody.kids), free through the end of 2026, has no chat tutor; its one short typed chat, for 3rd grade and up, asks more than it tells. Here are the studies, what they share, and their limits.

The positive trials

Harvard, physics, 2025. Gregory Kestin and colleagues randomized 194 undergraduates between an AI tutor and an active-learning class for two lessons, then swapped. The median post-test score was 4.5 with the AI against 3.5 in class, a regression effect size of 0.63 that the authors think is an underestimate because many students hit the ceiling, and the AI group took a median 49 minutes against 60. The design detail is the point: the tutor was GPT-4 with expert-written step-by-step solutions in its prompt, and the platform, not the prompt, enforced one step at a time. The paper is open in Scientific Reports.

Nigeria, English, 2025. A World Bank team led by Martín De Simone ran a randomized trial of six weeks of after-school sessions in which senior secondary students used Microsoft Copilot (GPT-4) with a teacher in the room. Scores rose by 0.31 SD overall and 0.23 SD in English, more for girls and for stronger students, and the team estimated the gain at 1.5 to 2 years of typical schooling. The World Bank blog summarises it. For many students it was their first time using a computer, which is part of why the size should be read carefully.

Tutor CoPilot, 2024. Rose Wang, Dora Demszky, Susanna Loeb and colleagues put the AI in a different seat: it suggested next moves to human tutors working with K-12 students, who could use them or not. Students of tutors with the tool reached mastery on exit tickets 66% of the time against 62%, and the gain was 9 points for students of the lower-rated tutors. It cost about $20 per tutor per year. The working paper is on EdWorkingPapers. Here the AI helped the adult, not the child.

The trials that warn

Turkey, high school math, 2024. Hamsa Bastani and colleagues gave about 1,000 students GPT-4 during practice, in two versions. The plain chat version raised practice scores and then left students 17% worse on a later exam taken without help: they had let it do the work. A version with guardrails that gave hints and not answers mostly removed the harm. The paper is on the lead author's site.

Khanmigo, middle school, 2026. Philip Oreopoulos and Hanna Low ran a two-year cluster randomized trial in 18 middle schools. Khan Academy practice with the Khanmigo chat tutor on top raised remedial math by 0.06 to 0.08 SD a year, and the AI layer added no measurable benefit over the practice alone. Chalkbeat's report on the trial noted that students messaged the tutor in only 17% of sessions. The working paper is on EdWorkingPapers. The finding is not that Khan Academy does nothing; it is that the chat window was not where the learning came from.

Benchmarks are not trials

Several models are now tuned for teaching. Google's LearnLM, for example, was preferred by expert raters over GPT-4o by 31% on seven pedagogy benchmarks, as its 2024 report describes. That is a preference rating of transcripts, not a measure of what a student learned. Several apps for young children also publish parent surveys or internal fluency data; the research summary behind this post found no peer-reviewed controlled study for them as of October 2026.

What the positive studies share

Read across the rows and the pattern from older research returns. Kurt VanLehn's 2011 review found that tutoring software which checks each step came close to human tutors, while software that only marks the final answer did much less. The chat interface is not the active ingredient. Structure is.

How to read effect sizes here

Matthew Kraft's 2020 guidelines are the right scale: most education trials land below 0.10 SD, and 0.20 on a standardized test is large for anything delivered at scale. By that scale the Nigeria result is big, the Khanmigo package result is ordinary, and the Harvard result is large but measured on two lessons with a ceiling. None of these trials ran with children under ten.

What Peabody does instead

Peabody has no chat tutor; its one short typed chat, for 3rd grade and up, asks more than it tells. The design follows the pattern the positive trials share: the task is bounded and the answer is known in advance.

• Practice is code. Every question in the games and practice rounds comes from the practice brain, which is ordinary code. It decides the level, the next question and when something comes back. No AI decides a child's level.
• Three boxes, the answer known. Every answer is one of three big boxes, so there is nothing to type and nothing for a model to mark.
• AI writes; it does not tutor. A language model (Claude, from Anthropic, by default) writes stories at the child's measured reading level and screens them for safety. The one place a child's own typed words go to a model is Peabody Studio, where a child describes a game and the model writes it; even there, the game's questions come from the practice brain. Each call is budgeted and logged against the child.
• The grown-up is the tutor. The grown-up view shows which words are sticking and which facts are slow, because the best-supported "AI tutor" result so far is the one that made the human better.

Every way Peabody uses AI is written out on how Peabody uses AI. More on the reasoning is in what an AI tutor for kids can and can't do and on the science page. Start free or read a story from the library.

What this does not show

• Short exposures. Two lessons, six weeks: none of the positive trials ran a school year.
• Older students. The trials above are undergraduates, secondary and middle school students. Evidence for AI tutoring of five- to eight-year-olds is thin.
• Authors built the tools. The Harvard team built their platform; the CoPilot team built theirs. That is normal and is a reason to wait for replication.
• Peabody is not in any of these trials. It has not been tested in a trial, and nothing here is a claim about what it does to a child's scores.

Learn more elsewhere

• Stanford SCALE: the evidence base on AI in K-12: a 2026 review: of more than 800 papers, about 20 were strong causal studies, and hint-giving tutors did better than answer-giving chatbots.
• OECD Digital Education Outlook 2026: doing a task with AI isn't learning it; gains often vanish when the AI is taken away.
• Hechinger Report on the Khanmigo trial: Jill Barshay's plain-English column on the study, and on AI-and-learning research every week.
• Ethan Mollick: against "brain damage": a balanced walk through the studies on when AI helps learning and when it replaces it.
• Education Next: two-sigma tutoring, fiction and fact: why Bloom's famous tutoring number is overstated, and what real tutoring programs get.

For about seventy more, checked and sorted by who they're for, see where to learn about AI.

Questions parents ask

Do AI tutors work?

Some do, in trials, when the task is bounded, the solution is written in advance and the tutor goes one step at a time. An open chat tutor added to practice did not add measurable learning in the largest school trial so far.

Can an AI tutor hurt learning?

It can if it does the work. In a 2024 trial in Turkey, students who used plain GPT-4 during practice scored 17% worse on a later exam without it. A hint-only version mostly removed that harm.

Is there good evidence for AI tutors for young children?

Not yet. The randomized trials so far are with undergraduates and secondary and middle school students. For children under ten, structured practice with a grown-up nearby is the better-supported choice.

Why doesn't Peabody have a chat tutor?

Because the trials point to structure, not chat, as what helps, and because an open chat with a young child can go anywhere. Peabody's questions are code with known answers, and its AI writes and screens stories and, in the Studio, writes the games kids describe.