Using Questionable Data To Back City Planning
A recent report from the Pew Charitable Trust called for increased affordable housing in cities across America. As the Atlantic reports, the authors suggested:
Solutions like requiring developers to include affordable housing units in new projects and developing metropolitan-wide transportation are politically unpopular. But they are a necessary part of any effort to restore economic mobility and the American Dream.
While everyone supports economic mobility and the American Dream, there is not adequate evidence to back the authors’ call for more regulation and transportation subsidies.
According to the report, “Mobility and the Metropolis: How Communities Factor Into Economic Mobility,” income segregation in a city leads to low economic mobility. This is important for Missouri cities such as Saint Louis, where income segregation is high and income mobility is low. The authors of the study would have Saint Louis expand transit and entice developers to build mixed-income housing. However, the track record of Saint Louis-area transit-oriented development has been less than ideal, as exemplified by a subsidized apartment complex near the Richmond Heights Metrolink station. A closer look at statistical models of the report calls their entire conclusion into question.
Without getting into too much detail, the problem with the report is that their model only finds an effect from income segregation when they don’t include other relevant explanatory factors. For instance, analyze New York City, which has among the highest income segregation (A) and lowest economic mobility (B) ratings. The report would have you believe that A contributes to/causes B, so improving A improves B. But what if (C), the dominance of financial services or legal professions, causes both A and B in New York City? Then fixing A will have no practical impact on B. In the complicated areas of income segregation, we must account for many possible factors like C.
However, the report promotes a simplified model that only shows a relationship between income segregation and economic mobility without including enough alternative variables. The report did not even mention that their model demographic variables did not show that income segregation was associated with economic immobility. Despite this shortcoming, both the report and media coverage have used the report to make irresponsibly expansive policy recommendations.
This over-hyping of statistical data can be all too common in the social sciences, so Missouri policymakers and citizens have to be vigilant. It is easy to twist data to justify government intervention if no one challenges the strength of the result.
