New Research Shows Traditional Education Rankings Are Misleading
Traditional education rankings rankings fail to provide “apples to apples" comparisons among states. By treating states as though they had identical students, they ignore the substantial variation present in student populations across states. Conventional rankings also include inappropriate or irrelevant data to the educational performance of schools, such as raw spending per pupil, graduation rates, and pre-K enrollment.
To better measure educational outcomes, Stan J. Liebowitz, Cato adjunct scholar and Ashbel Smith Professor of Economics at the University of Texas at Dallas (UTD) with Matthew L. Kelly, a graduate student at UTD, compare state test scores for each of three subjects (math, reading, and science), four major ethnic groups (whites, blacks, Hispanics, and Asian/Pacific Islanders) and two grades (fourth and eighth), for a total of 24 potential observations in each state and the District of Columbia. They give each of the 24 tests equal weight and base their ranking on the average of the test scores.
After adjusting for the heterogeneity of students, states in New England and the Upper Midwest who typically perform favorably fall in the rankings, whereas many states in the South and Southwest score much higher than they do in conventional reports.
The authors also produce rankings that, unlike most conventional reports, consider states’ cost-effectiveness of education spending. Florida, Texas, and Virginia are the most efficient in terms of quality achieved per cost of living-adjusted dollar spent. Conversely, West Virginia, Alabama, and Maine are the least efficient. Some states, such as Massachusetts and New Jersey, do an excellent job educating students but also spend quite lavishly and thus fall considerably when spending efficiency is considered.
While the authors observe a positive relationship between spending and achievement using nominal dollars, it disappears when state-level cost of living adjustments are made. This does not necessarily imply that spending overall has no effect on outcome, but merely that most states have reached a sufficient level of spending such that additional spending does not appear to be related to achievement as measured by these test scores.
The authors also briefly examine additional factors that affect student performance. They find states with stronger unions tend to get worse academic outcomes. Unions are negatively related to student performance, presumably through opposing the removal of underperforming teachers, opposing merit-based pay, or because of union work rules. Additionally, the authors’ results indicate that having a greater share of students in charter schools is positively related to student achievement.
Although this study constitutes a significant improvement on leading state education rankings, it retains some limitations. There exists substantial variation in education quality within states and disagreement about desired educational outcomes. However, state-level rankings do provide an intuitively pleasing basis for lawmakers and interested citizens to compare state education policies. The authors’ main goal is to provide rankings that more accurately reflect the learning that is taking place by focusing only on academic achievement and disaggregating scores, rather than scoring inputs and state-wide test scores.
