How do we improve schools so that kids learn more?
It’s a straightforward question — deceptively so, because improving schools, measuring whether you have improved schools, and writing down a recipe for school improvements that can be successfully used elsewhere is an astonishingly hard problem.
Philanthropists in the US have spent hundreds of millions of dollars promoting massive overhauls of the education system, often without improving test scores even a bit. Investing in “small schools” didn’t produce the hoped-for gains; neither did pushes to video and evaluating teachers or personalized learning programs. Educators, meanwhile, have protested that test scores don’t even measure the thing we really care about, which is whether students are successfully learning.
Internationally, the situation is even worse. In many parts of the world, lots of kids don’t even get the chance to attend school — and the ones who do often attend profoundly inadequate schools. By age 10, 90 percent of children in low-income countries still cannot read with comprehension. Some schools combat extraordinary rates of teacher absenteeism. Kids can go years barely learning anything because of an inadequate educational environment.
Lots of things have been tried to improve education in the developing world. Nonprofits, governments, and researchers have bought students uniforms, paid their scholarships, bought them menstrual products, built more schools, purchased textbooks and technology for schools, paid to reduce class sizes, trained and mentored teachers, and much more. Research suggests that many of these interventions make a difference. But figuring out which makes the biggest difference — and which are cost-effective enough to be worth investing in — is extraordinarily hard.
That’s the subject tackled by a new working paper from the World Bank Group, “How to Improve Education Outcomes Most Efficiently? A Comparison of 150 Interventions Using the New Learning-Adjusted Years of Schooling Metric.” The paper lays out a simple problem: We have an enormous wealth of research studying the effects of school interventions, but not much chance to compare the programs and figure out which one works best.
When researchers compared more than 150 interventions in developing countries, one trend stands out: Investing in teachers is strikingly cost-effective. It’s popular to help students in poor countries by building schools, buying them school uniforms or textbooks, or paying for scholarships. But, the researchers find, popular interventions like these do not compare favorably to ones that focus on teaching. If we want to improve schools, the most cost-effective approach is getting teachers professional development and training.
Why it’s hard to compare educational programs
One paper finds that paying for scholarships enables girls to stay in school three years longer. Another finds that building a new school improves attendance by 20 percent. Another finds that students offered a new curriculum are more fluent readers.
Policymakers trying to improve schools in their country can afford to make one of these changes, but not all of them. Which should they make?
The current research literature, the World Bank paper argues, is ill-equipped to answer this question for us. That’s because research papers all measure their outcomes in different ways.
Often, research papers describe the impact of their program in terms of an improvement of either the increase in years students spend in school or improved performance on local test scores. But we care about how many years students spend in school primarily as a proxy for how they learn.
It’s not easy to compare interventions even when they report how much students have learned. Many papers report how much students improved on a local test, one that isn’t necessarily comparable across subjects, age groups, or countries. “When we compare studies using standard deviations as our metric,” the World Bank paper argues, “we impose the assumption that the difference in learning levels between the median and 66th-percentile student in a fourth-grade math class in Kenya is equivalent to the difference in learning levels between the median and 66th-percentile student in a twelfth-grade history class in Peru.”
That assumption is obviously mistaken — some countries will have far more variance in student test scores than others — and relying on it will make it harder to figure out which educational interventions work best.
So should we just declare that these outcomes cannot be compared to each other at all? Doing that means that policymakers with finite resources have no way to decide which interventions are best.
Enter the quality-adjusted year of learning. The idea is to look at how much a student learns in a year of education at a high-quality educational program (such as Singapore’s schools). Then, other interventions can be benchmarked by how many months or years a student would need to spend in a high-quality educational environment to learn the same amount.
“Learning-adjusted years of schooling translates gains into a format that is more accessible to policymakers,” David Evans, one of the co-authors of the paper, told me.
This simple concept hides a lot of complications, of course. In applying the quality-adjusted learning framework to the 150 interventions the paper reviewed, the authors had to make lots of assumptions about how more conventional reported results translate to quality-adjusted years of schooling. But if the framework is widely adopted, then papers will more directly report the information needed to calculate quality-adjusted years of schooling, making it easier for papers to compare results and learn from each other.
Nor does it solve all the problems with education intervention research, which will likely continue to be plagued by difficulty in getting similar results from interventions in different contexts.
But if widely adopted, a metric like learning-adjusted years of schooling solves one of the major barriers to researchers learning from each other’s results — and it identifies some particularly promising areas where education interventions should get a closer look.
Which education interventions work best?
Some educational interventions really stand out in the World Bank team’s analysis. “Some of the most cost-effective interventions can deliver the equivalent of three years of high-quality education (i.e., three years of education in a high-performing country such as Singapore) for as little as $100 per child,” the paper finds. Others achieve effectively nothing, with no measurable gains in learning-adjusted years of schooling.
The list of the most cost-effective is a bit surprising. The paper finds that the interventions that get the most done per dollar are “information campaigns” to inform families about reasons their kids should go to school, “interventions to target teaching instruction by level” — that is, teaching kids based on what skills they have already mastered and which skills they’re working on — and “structured lesson plans with linked student materials, teacher professional development, and monitoring.”
To unpack all that a bit more: If students are in a class where they are being taught material that is too easy or too hard, they will not learn. Encouraging schools to place students based on what skills they have, then, is a highly cost-effective way to improve performance.
If teachers do not know how to teach, students won’t learn. So lesson plans, training for teachers, and monitoring to determine which teachers are doing a good job is highly cost-effective.
Finally, explaining to parents and communities why education is important shows promise as a highly cost-effective way to increase the benefits of education (at least sometimes).
“Highly cost-effective” can mean that an intervention works exceptionally well and is moderately expensive, or that it only works a little bit but is very cheap. Information campaigns are only moderately effective, but they are very inexpensive compared to lesson plans, teacher training and monitoring, and curriculum development work; as a result, they end up highly cost-effective.
“Light touch” campaigns to tell people about the benefits of educating their kids “actually can deliver pretty sizable gains,” Evans told me, though “there’s more variation across those interventions than across some of the pedagogical interventions.” Information campaigns seem to only work sometimes, and we don’t know why they sometimes work and sometimes don’t; improving teaching quality, by contrast, seems to always work.
Why is investing in teachers so effective? The World Bank paper largely doesn’t speculate on this, but the papers it cites on curriculum development interventions explain. In Kenya, for example, one educational intervention that looks fairly cost-effective is the Tusome national literacy program. It includes national expectations for student performance, classroom visits, and feedback to teachers.
That might all sound very basic. But in many cases, it reflects a significant change in how schools operated. “For years, if not decades, schools and teachers have operated in virtual isolation from the education system … accepting the prevailing norm that they are not actually expected to perform their job up to any reasonable standard (such as, most fundamentally, showing up regularly to teach),” a 2018 paper on Tusome’s effects argues. With no one checking whether teachers showed up or did their jobs, many of them didn’t. A nationwide system for observing teachers and tracking results could be highly effective just by making teachers aware that they are in a professional role where they are expected to get results.
Education interventions that are ineffective might not always stay that way
Notably, some of the high-profile educational interventions the general public knows the most about don’t make the list. Providing textbooks, for example, is not found in their analysis to have any benefits. Building schools is not a very cost-effective way to improve education, and scholarships for students to attend aren’t, either. Efforts to reduce class sizes, provide school uniforms, or give schools technology (without additional reforms) have no effect at all.
It is dispiriting to see such poor results for interventions that seem, intuitively, like they ought to make a big difference. Many nonprofits advertise to donors their work on building schools and getting them needed school supplies. But rather than discouraging, these reports should mostly be humbling. Trying to understand how to educate students halfway around the world is hard. Just because something sounds like a good idea doesn’t mean it will actually make a difference.
Of course, the ineffectiveness of those other interventions is contingent on the conditions surrounding them. Keeping kids in school for more years may do a great deal for them if they have good instruction but might be worthless if quality instruction is not available.
In poor countries, many students attend school for years while still not mastering basic concepts. Giving textbooks to students who can’t read won’t do anything, but once we’ve increased the share of children who can read, that will change the impact of giving out textbooks.
Keeping kids in school for longer will likely be much more effective once those schools are effective at teaching, so implementing the most promising reforms identified might open the door for other reforms, making them cost-effective when they aren’t right now.
At the same time, we should keep in mind that many education interventions, even ones that seem like a good idea and have preliminary evidence in their favor, likely won’t work out. In the US, philanthropists have tried again and again to reform education — and they’ve often made things worse.
Education is complicated, requiring buy-in from teachers, parents, students, and local communities. Lots of interventions that seem promising won’t work — and that’s okay. The important thing is that we measure results and report them consistently, so researchers can learn from research elsewhere and we can focus our efforts where they’ll matter most for kids.
Sign up for the Future Perfect newsletter. Twice a week, you’ll get a roundup of ideas and solutions for tackling our biggest challenges: improving public health, decreasing human and animal suffering, easing catastrophic risks, and — to put it simply — getting better at doing good.
