I Don’t Think the Data Say What You Think They Say
Much to the consternation of public education officials, the State Board of Education voted to remove the commissioner of education, Margie Vandeven, from her position last Friday. The vote came after a failed attempt to remove the commissioner just a couple of weeks ago. The five members of the board who voted to oust Vandeven were all appointed by Gov. Eric Greitens. This post, however, is not about the maneuvering of the governor to assemble a board that would take this action. This post is about bad evidence.
When it became clear that the governor was attempting to remove Vandeven, public educators rallied to her cause. Some defended Vandeven as a competent and well-regarded public servant. Others, however, sought to attack charter schools. They presumed that the whole ordeal wasn’t about Vandeven, but about putting someone in place who would work to expand school choice in Missouri. Several public school administrators tweeted the below graphic. One wrote, “More tax dollars could soon be siphoned to these ineffective schools if @GovGreitensMO gets his way.”
The claim that this chart shows anything meaningful about charter school effectiveness would be amusing if it wasn’t being vigorously advanced by numerous individuals who should know better. Take a look to see what I mean. The table shows a comparison of public school districts performance on the Annual Performance Review with individual charter schools. There are two huge problems with this. First, we are comparing whole school districts with individual schools. Second, it compares districts with schools that serve entirely different types of students. Charter schools in Missouri are only in Saint Louis and Kansas City. It is hardly fair to compare their performance to the Rockwood or Nixa School Districts that serve much more affluent students.
A better comparison would be to compare charter school performance to the performance of individual schools in Saint Louis and Kansas City. I’ve done that below. As expected, the comparison is much more favorable for charter schools. In total, 58% of Missouri charter schools scored above 70% on the APR. Meanwhile, just 45% of public schools in Saint Louis and Kansas City scored in that range. Interestingly, 13 of the 16 traditional public schools in the top category were magnet schools, as were 6 of the 11 in the next category. These are hardly traditional schools. Neither traditional schools nor charter schools are allowed to have admissions requirements, but magnets can. Some magnet schools require students to take admissions tests and score at a certain level before they can be admitted.
Given this more accurate context, charter schools appear to be outperforming their traditional public school counterparts. Yet, even this comparison tells us little about the effectiveness of the schools themselves. For that, we’d need a more sophisticated analysis. Fortunately for us, the Center for Research on Educational Outcomes (CREDO) at Stanford University conducted such a comparison. Using a matching design, in which they compare charter students to similar students in district schools, they found Missouri’s charter schools outperformed the district schools in reading and math.
It is understandable for public school officials to defend Margie Vandeven or even for them to oppose charter schools. However, it is not acceptable for them to build their case on misleading data. Let’s have a productive dialogue about public education and school choice in Missouri, and let’s have that conversation by looking at what the evidence actually says.
