People who criticize exit exams on the basis of the potential for error unfortunately have a point:
A testing company faces a fine after it mistakenly failed hundreds of students on Ohio's new graduation test, state education officials said Monday.Measurement Inc. graded 1,599 tests and failed 890 students after accidentally converting raw test data to passing and failing grades, the state Education Department said. The error was made on tests given last summer to students entering their junior and senior years, as well as students who were in 12th grade last year but haven't graduated.
Whether the test was the only thing keeping any students from graduating -- and whether anyone might have wrongly been sent back to school this fall -- wasn't immediately clear.
The scores have since been corrected; the state and the company planned to notify 272 school districts this week whose students were affected. The corrected test scores still were not enough to pass for 543 students.
So let's do the math here. 1599 students across two grades. Most exit exams, though given to 11th- and 12th-graders, are set at a level below that. Thus, one would hope that the pass rate on such an exam would be around 70-80%, at least.
Measurement Inc comes up with a pass rate of only 45%. Even given how much we bemoan the current state of public school affairs, that's astonishingly low. The true rate - if we assume the corrected scores to reflect reality - is more like 66%, which is not great, but it does include a variety of school districts and seniors who hadn't managed to graduate yet.
Someone should have caught this - specifically, Measurement Inc. should have. From the description given here, it sounds like raw scores were directly scaled onto the final score scale, and the equating was left out. That's a pretty damn big part to leave out. What's more, the horrendous 55% fail rate for a group of juniors and seniors should have been a major red flag. There's no way that equating was done properly if that were the case, not unless the state of Ohio had recently changed to using much harder test items, or just happened to not require students to actually show up for class last year.
I would not care to defend exit exams to someone who's been disadvantaged by this. When we attempt to convince the public that these tests can be advantageous in pointing out the importance of basic skills and the need for schools to teach them, we have to be willing to guarantee that we, the psychometricians, are not going to screw up on our basic equating and scoring skills. Errors like this are perhaps not unexpected as we gear up for The 21st Century aka The Century of Testing Anyone Who Moves, but errors like this are still unacceptable.
Posted by kswygert at December 13, 2005 05:09 PM