Kansas Policy Institute commissioned a review of the recent WestEd education cost study that was undertaken for the Kansas legislature at the behest of the Kansas Supreme Court. Dr. Benjamin Scafidi of Kennesaw State University has provided a response (here) critical of WestEd’s use of a “cost function” approach to determine education funding in Kansas. But more precisely, Dr. Scafidi is not so critical of the WestEd study itself as he is of using a cost function approach to determine education spending in the first place.
Click here to read Scafidi’s full paper.
Scafidi identifies three primary reasons why cost functions do not serve the purpose of providing reliable minimum cost estimates for achieving student outcomes. The first has to do with external factors that impact costs, the second is the absence of all or accurate measures of input prices, and lastly the inability to control for the presence of inefficiency by public schools.
Dr. Scafidi summarizes the inherent limitations in employing a cost function approach to determine suitable education costs:
My point is that given the lack of available and accurate data and given the tenuous relationship between spending and outcomes in the conventional public school system over the past 5 to 8 decades, education spending decisions should not be made on the basis of estimates from any cost function approach. (p. 7 – emphasis added)
Emphasizing those innate issues, Scafidi compared Kansas NAEP results and spending patterns to Florida and New York. All three states had similar 2017 NAEP results with much different per-pupil spending. As the accompanying chart shows, Florida spends just over $10,000 per pupil which is about 25% less per pupil than Kansas. New York’s per-pupil cost in excess of $21,000 is nearly double that of Kansas. Clearly, factors beyond the realm of a simple cost function are responsible for these differences.
Florida is of particular note. Not only does Florida have about the same 2017 NAEP results as Kansas with lower per-pupil spending in a state with a higher cost of living, Florida made more progress on NAEP than any other state in the nation. Figure 1 reveals the stark contrasts among Florida, Kansas and New York in terms of increased real spending and changes in grade 4 NAEP reading scores from 1998 to 2015. Florida’s real spending increased at one-tenth the rate of Kansas, but Florida’s change in reading scores outpaced Kansas 15 fold. Unlike Kansas, Florida has embraced school choice, with more than 1.7 million students choosing a school or program that best fits their individual needs.
He also points out that temporal constraints placed on the WestEd study is a factor in the Kansas study. Given the extremely tight time-frame established by the Court, the analysis provided by WestEd precluded the estimation model from performing five “best practices” (identified by the lead WestEd author herself in a previous study) to check for the “robustness and reliability of results.”
President Reagan was once asked that while he was actor how concerned the producers were with the quality of his movies. He responded by saying something like “they didn’t want them done well, they wanted them done Thursday.”
The Court’s insistence on having a study available with such haste is their version of getting it “done Thursday,” and serves as a reminder of the disconnect that exists with the Kansas Supreme Court regarding the legislative process. Scafidi recognizes the WestEd study is the product of an unrealistic court/legislative time table. As such, it precludes the WestEd study from being subjected to a best practices analysis despite that “each of these five best practices are doable with existing data—if time were permitted by the several branches of Kansas state government.” (emphasis not added)
Three of those best practices are worthy of discussion here.
-Best Practice #2. Check whether the parameter estimates of the WestEd cost model predict past changes in Kansas student outcomes—given Kansas’ changes in spending per-student. It makes intuitive sense that a cost function model be subjected to the test of whether it would be consistent with historic changes. If it couldn’t be shown to have worked in the past, how could it be expected to work in the future?
-Best Practice #3. See if the WestEd cost function estimates predict historical changes in Kansas public school NAEP scores. Since NAEP is the “gold standard” in standardized testing, again, it only makes sense that the robustness of the model be tested against past experience in NAEP. Obviously, given the NAEP experiences presented above, increased spending has not led to gains in student outcomes.
-Best Practice #4. Report the sensitivity of the minimum “cost” estimates to changes in specifications, variables included, and measurement of variables. As the Supreme Court itself declared during Gannon, there are literally hundreds of ways education funding could pass constitutional muster. Scafidi rightly claims “there are myriad reasonable ways to model the costs of educating students. WestEd should report results from several alternative specifications—to see if their primary results are robust to different specifications.”
Scafidi provides a contrast between the promises of the WestED model and the recent historical experiences of Kansas, Florida, and New York in terms of spending and outcomes. He poses the question: Should policymakers rely on (a) estimates from a cost model, or (b) recent multi-state experiences to establish comparisons between spending and outcomes? He offers the bold statement that the results in Florida is “strong evidence that the minimum cost to achieve the current level of achievement in Kansas is significantly lower than what it is spending now.” (emphasis added)
It is vital that one does not lose site of the fact that this is not merely an academic exercise. Now that the Court has taken oral arguments regarding the constitutionality of the new school finance law and a Gannon VI opinion will be handed down shortly, the future of Kansas public school students and hundreds of millions of taxpayer dollars hang in the balance.
