June 24, 2003
Test design blunder negates NY Regents scores
N2P Reader and humble genius Bas Braams has sent a couple of emails my way regarding the mathematics portion of the NY Regents Exam. For those of you unfamiliar with the exam, it's produced by the New York State Education Department to assess if students have met the New York State Learning Standards, and it's a requirement for graduation in New York State. The administrator's manual for the exam is here.
The first email that Bas sent covers the negative press the June 2003 exam had been receiving, and he concluded that perhaps the exam was okay, although the difficulty level may have been inconsistent with past exams.
However, he followed this up with an email in which he decided, after further inspection, that it appeared the January and June 2003 exams differed greatly in difficulty, and that some items were badly worded as well. The open-response section of the June exam appeared to him to be more difficult than the corresponding section in the January exam. Now, even when exams are well-constructed, some difficulty fluctuations may appear, but if care is taken when assembling the test forms, the fluctuations will be small, and can be corrected through equating (this is done on the SAT).
Now, though, it's a moot point, because schools have been given the option of discarding the math portion of the exam. The test construction method must have failed at some point, because the statistics show the June exam was in fact far more difficult than the January form:
A high failure rate on the Regents exam had called into question the fairness of the test and imperiled the right of thousands of seniors to graduate from high school this week. Commissioner Richard Mills also ruled that the planned August administration of the math test would be suspended to give education officials more time to review the June test results and why so many students failed...
Last Tuesday, tens of thousands of high school students across the state took the test, known as the Math A Regents examination. The state immediately ordered a speedy review of the results because of initial reports by school districts and parents of unusually low passing rates...
Speediness now doesn't quite compensate for incompentency earlier. Incidents like these bother me a great deal, not only because I feel for the kids who had to take an exam that is essentially going to be thrown out, but because this also adds fuel to the fire for those who advocate removing high-stakes tests altogether.
