Executive Summary
In response to the Kansas Supreme Court’s recent ruling in the Gannon V case, the Kansas Legislature recently contracted with a vendor to analyze the “cost” of educating public school students in grades K-12. Specifically, the vendor was asked to “estimate the minimum spending required to produce a given outcome within a given educational environment.” The vendor, WestEd, used a “cost function” approach in order to estimate the costs of providing students in each public school in Kansas with an adequate education.
There are three primary reasons why supposed “cost” functions do not provide valid and reliable estimates of the minimum “cost” of achieving a given outcome:
1. Researchers do not have access to data on all external factors that impact the cost of educating students.
2. Researchers do not have access to all input prices; or, even accurate measures of the input prices they include in their modeling.
3. Researchers cannot fully control for the presence of inefficiency by public schools. Purging inefficiency from spending data is “the key step in converting a spending function to a ‘cost’ function.”
While the WestEd research team endeavored to collect good data and control for only some possible inefficiency, these three concerns apply to the WestEd model.
There is a further concern with the WestEd model. Gronberg, Jansen, and Taylor (2011: page 25) wrote, “For cost function analysis, best practice requires researchers to adopt appropriate modeling and estimation strategies and to check carefully for robustness and reliability of results” (emphasis added). The third author listed who wrote those words was the lead author—Dr. Lori Taylor—of the WestEd report. A careful check for robustness and reliability of the WestEd cost function estimates was not performed. The following best practices should be performed to “check carefully for robustness and reliability of results”:
Best Practice #1: Report 95 percent confidence intervals for estimates of “minimum” costs.
Given the large degree of estimation involved with the WestEd cost model—and in any cost model done by anyone else—estimates are created with error. Given that the estimates are created with error, it is best practice to report 95 percent confidence intervals—upper and lower bounds of the outcome of interest such that one can be 95 percent confident that the true value lies in that range.
Best Practice #2: Check whether the parameter estimates of the WestEd cost model predict past changes in Kansas student outcomes—given Kansas’ changes in spending per-student.
If the parameter estimates from the WestEd cost function are correct, then they should do a reasonable job of predicting past historical patterns in Kansas state test scores, graduation rates, and spending per-student.
Best Practice #3: See if the WestEd cost function estimates predict historical changes in Kansas’ National Assessment of Education Progress (NAEP) scores.
The parameter estimates from the WestEd cost study can be easily used to see if they predict the historical relationship between Kansas public school NAEP test scores and per-student spending. There is evidence that NAEP scores are associated with future outcomes that are valuable to students, which are contemplated by the Kansas Rose Standards. In fact, the Kansas State Department of Education (KSDE) has said the NAEP “is considered the ‘gold standard’ of assessments.”
Best Practice #4: Report the sensitivity of the minimum “cost” estimates to changes in specifications, variables included, and measurement of variables.
There are myriad reasonable ways to model the costs of educating students. WestEd should report results from several alternative specifications—to see if their primary results are robust to different specifications.
Best Practice #5: Estimate a production function to see if the estimates from the production function produce the same relationships between spending and outcomes as the WestEd cost function estimates.
According to microeconomic theory, a cost function contains “essentially the same information” as a production function. That is, a cost function indicates the minimum level of cost needed to produce a given level of output, while a production function indicates the maximum output possible for a given level of expenditure.
These five best practices are widely implemented in academic research on the economics of education; and, are not controversial. That said, including these best practices in the WestEd cost function analysis runs the risk of showing that the WestEd cost model does not predict the recent history of the relationship between spending and achievement in Kansas public schools. Further, the current amounts spent on Kansas public schools may already be well within the 95 percent confidence interval of spending levels that would achieve standards deemed adequate.
Perhaps these risks are best shown with some contextual and historical evidence on the relationship between public school spending and student outcomes. For example, at best Kansas public schools do not outperform Florida public schools on the NAEP, but Kansas public schools spend 25 percent more per-student than Florida public schools, bearing in mind cost of living differences. New York public schools do not outperform either Kansas or Florida, and New York spends more than double what is spent per-student in Florida.
2017 is the most recent NAEP data available and 2015 is the most recent comparable data available on public school spending by state.
Given that Florida public schools are achieving at levels at least as good as Kansas at a 25 percent lower cost, one must conclude that the minimum level of spending “required” to achieve at current levels in Kansas is significantly less than what is spent now. Given recent history in Kansas, as this report details below, it is not reasonable to conclude that giving the Kansas public school system—as currently constituted—a large boost in spending will significantly improve student outcomes.
As shown in the body of this report, Kansas has increased real (inflation-adjusted) spending per-student rapidly this century—perhaps largely via mandates from the Kansas State Supreme Court. These increases in spending did not translate into significant gains in student outcomes on the NAEP—a battery of tests for which there is evidence that they predict valuable future outcomes for students, and the KSDE has said “is considered the ‘gold standard’ of assessments.”
If the Kansas State Supreme Court insists on further spending increases—into the current education system in Kansas—recent history and comparisons among Kansas, Florida, and New York suggest this policy will be a costly failure. In contrast, the estimates from the WestEd cost model promise significant improvements in student outcomes would result from large increases in per-student spending.
So, on what should policymakers rely—(a) estimates from a cost model or (b) recent Kansas history and interstate comparisons of the relationship between spending and valuable student outcomes? If recent and large increases in spending have not increased measurable student outcomes by very much in Kansas, and if another state (Florida) has a higher cost of living, perhaps more student disadvantage, significantly lower spending, and better student outcomes, then that is strong evidence that the minimum cost to achieve the current level of achievement in Kansas is significantly lower than what it is spending now.
If history and interstate comparisons are not persuasive—that is, if you believe that “this time will be different,” implementing each of the five best practices recommended here into the WestEd cost function approach will either validate the cost model estimates (by showing a tight 95 percent confidence interval around minimum cost estimates, by accurately predicting past relationships between spending and student outcomes, etc.), or these five best practices will show that the cost model estimates are not valid or reliable predictors of any past relationships between spending and outcomes and will find that current spending levels are well within 95 percent confidence intervals of spending levels deemed adequate.
Given the vast sums of taxpayer funds at stake, the Kansas Governor, Legislature, and State Supreme Court should insist that these five best practices be implemented into the WestEd cost function approach—to discover the truth about the relationship between spending and valuable student outcomes.
