Do Charters Make the Grade?

Over the weekend, the Columbia Missourian ran an interesting article that questioned the effectiveness of Missouri’s charter schools. Using MAP scores as a benchmark, the article suggests that the lower average MAP scores of charter school students as opposed to traditional public school students show that charter schools are doing worse in educating children. Problems abound in this analysis.

The article makes an oft-repeated mistake in education: viewing education as static, as happening in a vacuum. The static view leads one to analyze student scores as isolated events — as snapshots — and ignore the underlying trend. Only in taking a longer view of the data can we begin to see the degree of value added at each stage. Yes, the average test scores of students in St. Louis charter schools are lower than the statewide average of students in traditional public schools. But, no, it does not follow that those charter schools are worse. For starters, that claim is based on an apples-to-oranges comparison between St. Louis city charter schools and statewide public schools. Instead, it should be based on the relative performance of St. Louis charter schools as opposed to St. Louis traditional public schools — a gap which, it should be noted, has also been established and will be outlined in more detail in a forthcoming Show-Me Institute study. But charter critics still have to contend with the value-added component, something the article does not do.

So, the article fails to reject the claim that charter schools are effective. Can we actively prove the claim? As Show-Me Institute policy analyst Dave Roland writes, in the comments section of the article:

A simple comparison of MAP scores doesn’t even come close to telling the real story. None of these comparisons take account of “selection bias” in making charter and non-charter comparisons. A recent study by the Center for Research on Outcomes (CREDO) at Stanford University examined the effectiveness of charter schools in 16 states, including Missouri. The Stanford study took careful account of the prior test scores of charter school students, and every charter school student was paired with one or more similar students (aligned by race, sex, grade, poverty status, English language learner status, special education, and prior test score by subject) in schools that acted as feeders for the charter school. The study revealed that Missouri’s charter schools attracted students who were underperforming in their traditional public schools, and that this state’s charter school students were realizing larger academic gains than their counterparts in traditional public schools. The difference was small, but statistically significant.