In a just released on-line report, the Foundation for Excellence in Education documents the gap in proficiency rates between the results from the National Assessment of Education Progress (NAEP) and those reported by each state pursuant to the requirements of No Child Left Behind (NCLB).
The study compares the rates of proficiency students achieved in fourth and eighth grade reading and math in both tests for every state (the two sets of grades and subject matters the two assessments have in common). For those not familiar with NAEP, it is the largest nationally representative and continuing assessment of what America’s students know and can do in various subject areas. NAEP is administered by the U.S. Department of Education and is the only test the federal government administers in all states. It is given every two years (it was given earlier this year) to a sample of fourth, eighth and twelfth graders in subjects that include math and reading.
Their methodology was a simple, yet powerful one. They took the four subgroups common to both assessments and subtracted the NAEP proficiency rate in each state from their corresponding state assessment proficiency rate, as defined by each state individually. The result is what they termed the “proficiency gap.”
The findings for Kansas are nothing short of startling.
The following table shows how large the Kansas proficiency gap is, and how we rank among the 50 states.
For example, only 38% of Kansas fourth graders tested proficient in NAEP reading, but 85% were considered proficient in the most recent (2013) state assessment. Therefore, a proficiency gap of 47 percentage points exists between the two. Ranked 39th in the nation, only 11 states had larger gaps. Kansas fared worse in eighth grade reading, with a proficiency gap of 50 percentage points, giving us the sixth largest gap nationally.
And if that isn’t serious enough, the gaps get bigger when controlling for demographic groups. The proficiency gaps for each demographic subgroup (disabled, low income, Black and Hispanic) except White are actually larger. The full report on Kansas can be accessed here.
What does this mean and why is it important?
Clearly there is a sizable chasm between the norm-referenced NAEP test and the state-designed and administered NCLB state assessment. Does this mean the state assessment is too easy? Does it mean the threshold for being considered proficient is too low as KPI has shown in this paper? Is it a combination of those two? Or is it something different? Without knowing for sure, the answer is probably an amalgamation of factors. Only one thing is certain, there are no consequences for schools for NAEP performance, good or bad, but low NCLB scores had the potential for triggering remedial action by the feds.
In my teaching career I had one experience with NAEP. Several years ago, my school’s fourth grade was selected for NAEP and I helped proctor the test. Although I have no recollection of the test itself, the school’s principal made it clear that NAEP was not a priority and the assessment was given in a perfunctory manner. That cursory disregard was in stark contrast to how seriously schools approach state assessments. Kids are readied for those tests with practice sessions, strategy sessions, pep rallies, rewards to students (although rewards were supposed to be forbidden, schools did it anyway) and other tactics all aimed at improving student achievement.
It is my opinion that if schools in general take the same casual approach to NAEP but a highly involved approach to state assessments, some of the proficiency gap is attributable to the consequences that flow from the test results.
It is important because Kansans deserve realistic assessment results of student performance, not artificially inflated ones based on unattainable federal requirements like the 100% proficiency rate target of No Child Left Behind. But there is encouraging news on this front. As of this writing, NCLB is being rewritten in Congress, which reportedly will ease, if not eliminate, the federal proficiency requirements.
This is also a very timely issue. Kansas has transitioned to a new state assessment, one based in Kansas College and Career Ready standards, aka Common Core. The state is in the process of developing new performance categories based on the results of the 2015 state assessment (the 2014 results were corrupted and ultimately discarded due to computer hacker issues). This year was also a NAEP year, so there will be an indication of whether the new state assessment categories reduce what has been called the “honesty gap.”
We will provide an update when the 2015 results are available.
