Hope this isn't injuriuous to their self-esteem; Minnesota's youngsters aren't quite as good as they thought:
Because of "an error in judgment," the Department of Education inflated the percentage of elementary school students who passed the 2003 Minnesota Comprehensive Assessments in math and reading, Education Commissioner Cheri Pierson Yecke said Monday.
The percentage of students labeled "proficient" or better is really 3 percent to 6 percent lower, meaning that dozens more Minnesota schools probably would have been deemed underperforming under the federal No Child Left Behind law, state officials said.
Although Yecke said she was first alerted to the mistake in November and immediately began the process to correct it, the error apparently was brought to the attention of the state's testing director months earlier -- and before the state released the inaccurate results.
Mmm, not good. Errors can happen to anyone; psychometricians are people, too. That doesn't justify the deliberate release of bad data, although the issue here isn't with the tests themselves, but with how the standards were set:
There is no problem with the actual tests...[and] the students' raw scores are accurate. But the "cut score" -- the number of questions students must get right to pass -- was set too low. The reason: A committee of teachers formed last May to try to align the test results with proposed new standards in reading and math lowered the bar too far, Yecke and Olson said.
That committee never should have been called, Yecke said, because the tests didn't cover material in the new standards. When committee members saw how low the scores would be under the new standards, Olson said, they lowered the "passing" definition. The result was dramatically higher proficiency marks in 2003 than what kids had scored in 2002.
Okay, that's a pretty bad mistake. Why on earth would a standard-setting group have been assembled to judge test scores based on standards that weren't congruent with the tests? Yecke is right to say that group should never have been assembled, and their standard should not have been the one used.
The real scores, arrived at by using the previous standards, showed improvement -- just not the knock-your-socks-off type of improvement. For instance, the number of third-graders scoring proficient in math rose from 65 percent to 72 percent -- but not to 75 percent as was reported last July.
Yecke said that as many as 50 additional schools would have joined the list of 143 schools that were deemed "not making adequate yearly progress" if the correct scores were used. The mistake didn't put any schools on the list, Yecke, Olson and Davison said.
So the schools that would have otherwise been deemed inadequate receive a "Get Off the List Free" card this year. And the revised lower proficiency rates will be used for comparison next year, which means that any future improvements will help schools even more. Also, a completely new test will be developed to cover the new standards, so that the old test will not have to be aligned to the new standards.
Minnesota's DOE is reacting to this appropriately, but this is a pretty big horse to let out of the barn. The standard-setting protocol should have been one of the more rigidly-defined and QC'ed part of the process, and the news that the now-former state testing director Reg Allen released the scores after the problem was discovered should give further pause:
Although Yecke said she was first alerted to the mistake in November and immediately began the process to correct it, the error apparently was brought to the attention of the state's testing director months earlier -- and before the state released the inaccurate results.
Reg Allen, who resigned from the Education Department two weeks ago, made the decision to release the inflated scores despite having the accurate results in hand, said Mark Davison, head of the University of Minnesota's Office of Educational Accountability. But, Davison said, he couldn't persuade Allen to release the accurate results.
"He said he was trying to adjust for a transition to the new reading and math standards," Davison said of Allen's argument at the time...
"We had a disagreement over how we ought to do this," Davison said. "I mean, I viewed it as his call. After he made the call, he put together a document describing what the process had been. I signed off on that."
He added: "The commissioner also saw that. Whether the commissioner actually realized whether this process would have yielded scores that different from prior years? Probably not."
The press release is here. The language is exquisitely euphemistic - the tests are fine, but "changes do need to be made in how the scores are interpreted." I'll say.
Posted by kswygert at March 10, 2004 11:36 AM