Bloom's 2 sigma problem: can AI tutoring close the gap?

What did Bloom’s 2 sigma paper actually find?

In the June–July 1984 issue of Educational Researcher, Benjamin Bloom, then at the University of Chicago and Northwestern University, published “The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring.” The headline result did not come from a large trial. It came from dissertations by two of his doctoral students, Joanne Anania and Joseph Burke, which compared three ways of teaching the same material:

• Conventional: about 30 students per teacher, with periodic tests used for grades.
• Mastery learning: the same class size and usually the same teacher, plus formative tests used for feedback, corrective work for students who had not yet mastered the unit, and a parallel test to check that they now had.
• Tutoring: a good tutor for each student (or for two or three at once), followed by the same cycle of formative tests, feedback and correctives.

Students were randomly assigned to the three conditions. Bloom reports that the study was replicated with four samples of students in grades 4, 5 and 8, in two subjects (probability and cartography), with instruction limited to 11 class periods over three weeks.

The results were striking. Measured in standard deviations (sigma) of the conventional class, the average tutored student scored about 2 sigma higher: above 98% of the control class. The average mastery-learning student scored about 1 sigma higher: above 84%. Around 90% of tutored students and 70% of mastery-learning students reached the level that only the top 20% of the conventional class reached. Time on task moved the same way, from about 65% in conventional classes to 75% under mastery learning and over 90% under tutoring.

The “problem” in the title was economic. Bloom called one-to-one tutoring “too costly for most societies to bear on a large scale” and asked researchers to find group methods that could match it. Most of the paper is about that search: pairing mastery learning with other changes, such as better explanations or more student participation, to see whether two changes together could approach 2 sigma.

What does “2 sigma” mean in percentiles?

A sigma is one standard deviation of the comparison group’s scores. If scores are roughly normal, a student at the average sits at the 50th percentile, one standard deviation above sits at about the 84th, and two above sits at about the 98th. That is the arithmetic behind Bloom’s two claims.

In classroom terms: in a conventional class of 30, a 2 sigma gain means the typical tutored student would outscore about 29 of them. Smaller effects still matter. A gain of 0.3 SD moves an average student from the 50th to about the 62nd percentile; 0.8 SD moves them to about the 79th. Keep those conversions in mind for the numbers that follow.

Why did later research shrink the 2 sigma effect?

Bloom’s number has become shorthand for what tutoring can do, but it does not describe what tutoring typically does. Four things explain the gap.

The tests were narrow. Anania and Burke chose probability and cartography because the topics were unfamiliar to the students, and the post-test covered exactly what the three-week unit taught. In a 2024 essay in Education Next, Paul von Hippel points to a 1982 meta-analysis by Cohen, Kulik and Kulik that found tutoring effects averaged 0.84 SD on narrow tests written by the researchers, but only 0.27 SD on broader standardized tests.

Half the effect was mastery learning. The mastery-learning classes, with no tutor, gained about 1 SD on their own (von Hippel puts it at about 1.1). The tutoring condition included the same formative tests and correctives. So 2 sigma is the result of tutoring plus regular checks and corrections, not of a tutor alone.

Reviews of many studies found smaller effects. Kurt VanLehn’s 2011 review in Educational Psychologist compared human tutoring, computer tutoring and no tutoring. It noted the common belief that adult human tutors produce an effect of about 2.0, and found d = 0.79 instead. Intelligent tutoring systems, most of which work step by step, came in at 0.76, “nearly as effective as human tutoring.” A 2024 meta-analysis of randomized preK–12 tutoring experiments by Nickow, Oreopoulos and Quan in the American Educational Research Journal found a pooled effect of 0.288 SD. Effects were largest for programs that used teachers or paraprofessionals as tutors, ran in earlier grades, met at least three days a week and took place during the school day.

Scale adds friction. High-dosage tutoring works at scale, but at a different size. Two randomized trials of Saga Education’s math tutoring for secondary students in Chicago, published by Guryan and colleagues in the American Economic Review in 2023, found gains of 0.18 to 0.40 SD in math test scores that persisted into later years. The program used paraprofessional tutors to hold down cost.

What have AI tutoring trials found so far?

Generative AI revived the 2 sigma question, because an AI tutor costs a fraction of a human one. Three studies published in 2025 are worth knowing in detail, because they point in different directions.

The Harvard result is the one most often quoted as “AI reaches 2 sigma.” It does not quite say that. The comparison was a strong active-learning class, not a lecture; it covered two lessons, surface tension and fluid flow; and the post-test came right after each lesson. The authors also note that the tutor’s accuracy relied on pre-written answers. It is still the best evidence so far that a carefully structured AI tutor can teach a lesson well, in less time.

The Nigeria pilot matters because it ran in real schools as an after-school program. The World Bank team equates the gain to 1.5 to 2 years of “business-as-usual” schooling and reports that students who attended more sessions gained more. Teachers were in the room, opening each session with a suggested prompt and guiding students as they worked.

The Turkish study is the warning. Students with open GPT-4 access used it as a crutch: they did better while it was there and worse once it was gone. The version that gave teacher-designed hints instead of answers largely avoided the drop. Same model, opposite effect on learning. The difference was design.

What made tutoring work in the first place?

Strip Bloom’s tutoring condition and the newer studies down to what the tutor actually did, and the same mechanisms keep appearing. None of them is “a smarter explanation.”

1. Frequent checks. Formative tests after each unit in Bloom’s studies; in one-to-one teaching, the check can come after every step. The teacher finds out what the student understood instead of assuming it. Our guide to checking for understanding covers twelve ways to do this online.
2. Correctives, then a second check. Mastery learning did not just return a score. Students who had not mastered the unit did corrective work and then took a parallel test.
3. Mastery before moving on. The next unit built on a foundation the student had shown, not one the schedule assumed.
4. The student does the thinking. Time on task passed 90% under tutoring. In the Turkish study, the tutor that withheld answers protected learning; the one that handed them out did not.
5. Accurate, timely feedback at the student’s pace. Kestin and colleagues list scaffolding, managing cognitive load, accurate feedback delivered promptly and self-pacing among the practices their AI tutor was designed around. Their students’ time on the lesson varied widely and did not predict their post-test score.

What does this mean for how an AI tutor should be designed?

The evidence adds up to a fairly concrete specification. An AI tutor meant to capture some of the tutoring effect should do six things:

1. 1

   Teach from a plan

   A sequence of concepts taken from the course, taught in order. A blank chat box assumes the student already knows what to ask.
2. 2

   Check after each concept

   A question whose answer shows understanding, scored against a rubric. “Does that make sense?” is not a check.
3. 3

   Hint before telling

   Give the smallest nudge that lets the student find the answer. Handing over answers is the GPT Base condition.
4. 4

   Hold the line on mastery, with a cap

   Don’t advance on a wrong answer, but cap the retries and flag the student for a human rather than looping forever.
5. 5

   Revisit what was weak

   Come back to shaky concepts before the topic closes and at the start of the next session, instead of trusting one right answer.
6. 6

   Stay anchored to vetted material

   The Harvard tutor’s accuracy relied on solutions written by instructors. Grounding the tutor in reviewed content does the same job.

A general chatbot does the opposite by default: it waits to be asked, answers in full and never learns whether the student understood. That is not a flaw in the model, it is a mismatch of job. We compare the two approaches on Tutezy vs ChatGPT, and cover how grounding keeps a tutor faithful to your material in keeping an AI tutor grounded.

Is the 2 sigma problem still a cost problem?

Bloom framed his problem in terms of cost, so it is worth doing the sum. In the Chicago trials reported by Guryan and colleagues, Saga’s program gave students 50 minutes of tutoring every school day, two students per tutor, and cost $3,200 to $4,800 per student per year during the study. The authors compare that with Chicago Public Schools’ spending of about $17,000 per pupil. That is cheap for human tutoring, and still a large new line in any school or course budget.

Per-minute AI pricing changes the arithmetic. At $0.03 a learner-minute (Tutezy’s rate; other tools charge per seat or per student, compared in how much an AI tutor costs), the same 50 minutes a day is $1.50. Over an assumed 180-day school year, that is $270 per student. The catch is that the price is the only part of the comparison anyone knows. None of the studies above tested an AI tutor at that dose for a full year, so nobody should promise Saga’s 0.18 to 0.40 SD from one, and the Turkish study shows how easily an AI tutor can produce a loss. The budget side of Bloom’s problem has become easy. The design and evidence sides have not.

How should you read an AI tutoring claim?

Any vendor, including us, will be tempted to quote Bloom. Before you believe a sigma number attached to a product, ask these questions:

• What was the test? A test written for the unit just taught will show a much bigger effect than a standardized test. Cohen, Kulik and Kulik’s 0.84 vs 0.27 is the size of that difference.
• How long was the study? Two lessons, three weeks and a school year are different claims.
• What was the comparison? Nothing, a lecture, or a strong active-learning class?
• Was learning measured without the AI? Scores while the tool is available can rise even when learning falls.
• Who were the learners, and who ran the study? Results from motivated volunteers or a vendor’s own analysis need independent replication.

For a buyer, the practical version is simpler: pilot on one chapter, give a short quiz a week later without the tutor, and compare it with a group that studied the usual way. Our buyer’s checklist for AI tutoring software walks through setting that up.