An analysis by the attorneys who represent Schools For Fair Funding (SFFF) spent taxpayer dollars in an attempt to refute the findings of KPI’s At-Risk Funding: Increased Money Fails to Increase Achievement. SFFF claims to show that, in their words: “the achievement gap was substantially reduced during the time period of increased resources brought on by the Montoy moneys.” (emphasis not added) Specifically, SFFF includes the lowest state assessment category of passing – “Meets Standard” (which SFFF equates with “proficiency”) to make their case. We chose not to use that category in our analysis.
Here’s why.
The “Meets Standard” category may have met the subjective threshold for passing the state assessment, but it did not meet the standard for being college or career ready. According to the U.S. Department of Education’s National Center for Education Statistics, Kansas had among the lowest performance standards in the nation, and the definition of the “Meets Standard” category was lower than the “Basic” category associated with the National Assessment of Educational Progress (NAEP – more on that assessment below). Moving students up to the “Meets Standard” level, which SFFF considers a success, is like improving a grade from an F to a D. That’s an improvement of some degree but certainly not the degree needed to get kids ready for college and career.
We chose to take the top two performance categories, “Exceeds Standard” and “Exemplary,” and define them as “full comprehension” (words that KSDE used as part of their definition of “Exceeds Standard” for reading). These two categories are much more aligned with being college and career ready. SFFF implies the term “full comprehension” was a subjective term created by KPI and incorrectly stipulates the term is equivalent to getting an A+ on a test.
The chart below has the adopted “cut scores” utilized by KSDE for all performance categories for each grade in both math and reading. The lowest score to earn an “Exceeds Standard” ranking is hardly the same as getting an A+ on the test. A 77 was good enough for “Exceeds Standard” in 7th grade reading. A high school student who got a 68 on the assessment would have been considered one who exceeds the standard. Those are scores that are traditionally considered Cs and Ds.
Not only does the SFFF analysis have definitional issues, there are two serious flaws in the way the results are graphically presented.
- The SFFF graphs went back to 2004, but choosing that year gives distorted results because the definition of the categories changed. In fact, the definitions of the standards were lowered. In other words, “Meets Standard” in 2004 didn’t mean the same thing as it did in 2006 and after.
- SFFF double-counted students. Their graphs (shown immediately below) put students in the categories of “Free/Reduced Lunch” and “All Students” (directly from the KSDE Report Cards). However, the “All Students” category also includes the students in “Free/Reduced Lunch” category.
Starting in 2006, the year in which both Montoy money began and the standards definitions changed, the graphic representation for those who scored in the categories of “Meets Standard” (Proficient), “Exceeds Standard” and “Exemplary” looks like these two graphs.
Separating the low-income students from the not-low-income students dramatically changes the gap figures reported by SFFF. They reported 2006 gaps for reading and math as 12.6% (actually percentage points) and 12.1%, respectively. In reality, those gaps were 19.8 for reading and 19.3 for math. Their claim of those gaps improving by 2013 is vastly overstated. The 8.5 point gap in reading is really 17.0 and the 10.2 gap in math is actually 20.4.
Even choosing to use the proficient category as a “success,” the gap in reading only improved 2.8 percentage points, while the achievement gap in math increased by 1.1 percentage points.
A final point of contention is that SFFF chose only to look at state assessment results. A look at the NAEP outcomes reveals that achievement gaps are not being narrowed. The table below shows the achievement gaps in both 4th and 8th grade reading and math both before and after the injection of the Montoy money.
As the data clearly shows, even after a 679% increase in at-risk dollars between 2003 and 2015, the achievement gap actually increased in every category!
But the most disappointing of the SFFF analysis is not the technical issues we have brought to light. What’s most troubling is that the attorneys for SFFF seem satisfied – nay, proud – of these results. Seriously, is getting the low-income students to the lowest level of proficiency the goal of education? Given the focus on preparing students for college and careers, are these results acceptable? SFFF reports that getting students to the “proficient” level is an accomplishment. Do we not care if the low-income students perform at a level that will prepare them for college or career? Those student who scored in the “proficient” level can hardly be considered college and career ready.
If one didn’t know better, one might think SFFF is more interested in protecting the image of the schools than the performance of the students, especially those most at-risk.
