New Study Shows Negative Effect for Vouchers. We’ve Got Some Explaining To Do.
After an unbroken streak of gold-standard, random assignment studies finding either positive or neutral results for school voucher programs, a new paper published by NBER finds large, negative results for the Louisiana Scholarship Program.
My friends Adam Peshek, Matt Ladner, Jason Bedrick, Lindsey Burke, and Jonathan Butcher have written what I think are fair explanations of the findings. Based on survey research and buttressed by the enrollment patterns of schools participating in the program, it appears that the requirements that the program placed on schools kept good schools from participating. This drove students into lower-quality schools and, not surprisingly, worse outcomes. Yet another reason to remember that program design matters.
Let me add two points:
First, we should be Bayesians. To borrow from the branch of statistics, when trying to understand a phenomenon we should make assumptions about how it works, test them, update our assumptions based on the results of our tests, test them, update again, and so on, in a slow march toward the truth. Study after study has supported the belief that private school choice programs benefit the students who participate (across a number of indicators). This study should decrease our confidence, but—especially given the issues that Peshek and others raise with the fundamental design of the program—it should not decrease it a great deal. That said, there is clearly a lot going on here, and we need to keep digging and updating what we know.
Second, and more importantly, if you live by the sword, you die by the sword. For years now, advocates (present company included) have used state math and reading test scores as the primary means to argue that school choice “works.” In addition to probably not capturing everything that we want out of schools, we should also take into account that it appears that more and more families are opting into private schooling to get away from schools that they think are obsessed with standardized testing . We should not be surprised when we look at standardized test scores from private schools and see that these students are scoring lower. In fact, we should probably expect it. But, if we’re going to support our arguments for choice with test scores (using them to show either shortcomings in public schools or the benefits of choice), we have hitched our wagon to them and can’t be surprised if people attack vouchers when poor test score results come out. Similarly, advocates (present company included) have treated voucher programs as interchangeable when talking about their effects, even though they differ in meaningful ways. If we’ve talked about their benefits without taking program design into account, we can’t be surprised if people attack their shortcomings without doing it either.
I hope this study causes a course correction in the school choice community on several fronts (understanding the costs of regulation, how we think about test scores, the fact that all voucher programs are not created equal). It should be a wakeup call, not a death knell.
