At the July 2022 State Board of Education (SBOE) meeting board members confessed confusion, frustration and ignorance regarding the process of setting and understanding state assessment cut scores. This was manifested during a presentation by Dr. Neal Kingston of KU, the lead person hired by KSDE for state assessment work. Dr. Kingston was giving the board an overview of 10th grade math cut scores for approval, which is scheduled to take place at the next SBOE meeting on August 9. What began as a straightforward explanation from Dr. Kingston and KSDE Assistant Director Assessments Beth Fultz on the process of setting cut scores turned into an admission on the part of several board members of not understanding neither the process for determining nor the meanings of what the four performance levels and the scores that define those levels. (The one-hour presentation/discussion can be viewed here.)
This article is the first of two parts regarding what transpired at the July 12 meeting. The second article will describe how too much attention is put on addressing the entire process of state assessment scoring while no attention is given to accountability including addressing income-based achievement gaps observed from these and several other tests.
Board president Jim Porter set the stage by making two blatantly false statements in response to Dr. Kingston’s presentation, showing his ignorance of the entire state assessment process. According to Porter, “we set the highest cut scores in the nation…(and) we are beaten up across the street (meaning the state legislature) because of that.”
He went on quite the ramble regarding state assessment scores. “It concerns me that we are perceived as not being successful by those people who want to criticize public education because our test scores are in fact the highest in the nation. I’m sort of proud that they are, but I’m also wary that there are those who use that against us. I don’t even know what I’m asking.”
Lack of coherence aside, it is patently untrue to say that Kansas has either the highest cut scores in the nation or Kansas has the highest test scores in the nation when it comes to state assessments. Those are false statements because there is no such thing as either one of those, and Porter should know that. There is not a highest (or lowest) set of cut scores or test scores because each state has a different test and sets their own cut scores. It is impossible to compare even two states, like comparing Kansas to Missouri, let alone the entire country because of those simple realities.
Board member Ann Mah displayed her lack of understanding of the process when she asked Dr. Kingston if the panel of educators who were in charge of making the cut scores had a goal of making “the standards the highest in the nation again.” Standards and cut scores are two entirely different steps in the process. Standards are the specific subject items that are to be taught to at each grade level. Cut scores are the dividers between performance levels.
Betty Arnold expressed frustration in not understanding the steps in the entirety of the process. “When you see this much confusion…there are a lot of things we should be understanding that we don’t.” Arnold went on to state “I don’t like the idea that we say, ‘Ok, our standards are higher’, that sounds like a typical response when you’re not where you want to be in comparison to the rest of the nation.” She clearly makes her case, but it’s clear she, along with other board members, fails to appreciate the fact that state assessments are not comparable across the states. At least she had the integrity to openly admit she doesn’t understand.
Jim McNiece simply stated: “It’s all Greek to me.” His frustration is that when it comes to state assessment scores, the board is not “speaking the language the people in our state understand, and that includes the education community. They cannot explain” what test scores mean.
His and other board members’ exasperation concerns the four performance levels the board adopted after the feds replaced No Child Left Behind (NCLB) with the Every Students Succeed Act (ESSA) during Obama’s second term in office. The board chose to have only four levels of performance instead of the five that were used under NCLB.
The change in number of performance levels was addressed by Assistant Director Fultz. She claimed the new test (ESSA) “needed to be different from the old test…so that trend data had to start all over…that was the decision at the time of everyone to make it difficult to go from five (levels) to four. I will say, politically, at the time, the assessments weren’t looked at as stringently as they are today…I don’t know what the right number (of performance levels) is.” That is a very, very revealing statement. It is clear that the board at that time, under the lead of Commissioner Watson, wanted to create a distance between the ESSA testing and NCLB testing and changing the number of levels was a sure-fire way to do just that. Furthermore, I don’t understand how Fultz can believe state assessments are now looked at “stringently” today. More on that later.
The confusion created by switching to four performance levels led to KPI creating a system of A-F grading of schools across the state. KPI has translated those four levels into an A-F system, one that actually does speak to the people in the community, an easy-to-understand grading system. Board member McNiece should be accessing A-F grading when speaking to the people in his community. It’s readily available to all at the main KPI website.
The entirety of the confusion, frustration and ignorance on the part of the SBOE begs a single question: Why doesn’t the board have a complete understanding of the state assessment scoring process, or at least a working knowledge, one they can use when communicating with their constituents, or when working to hold schools accountable for the learning of their students?
I put the blame squarely on the shoulders of Commissioner Randy Watson. Several board members were not on the SBOE when the new process was adopted and operationalized back in 2015. Watson has been in his post since 2015. Obviously, he has not considered it a priority that the SBOE members understand the workings of the state assessments. In fact, since he was appointed, he has summarily downplayed state assessments as something that is required, but not to be taken seriously. Instead, he has led the board on quixotic missions like Kansans Can and a school redesign project that is supposed to analogous to a moon landing.
Just look at the way he presents state assessment results to the board each year, or doesn’t as was the case in 2021. Last year Watson failed to mention state assessment scores in his annual student achievement update in the October SBOE meeting. Instead, he makes innocuous statements like ‘we’re not where we want to be’ followed by excuses like ‘we’re getting more hard to educate students’. Then ultimately he confuses the situation by stating, as he did last year, that despite the low scores Kansas has the highest standards in the nation. That must be why board members seem to keep going to the “highest standards in the nation” well when defending low student achievement.
This statement from Betty Arnold sums everything up perfectly: “It is imperative we as a board have a complete understanding of what we are doing.” I couldn’t agree more. And it is a dereliction of duty on the part of Commissioner Randy Watson that they do not.
Next up: Too much attention on addressing process, no attention on addressing achievement gaps.
