"Irregularities," indeed.
In a rare move, the State Personnel Board threw out a promotional exam for one of the highest ranks at the California Highway Patrol, ruling that the oral test for deputy chief was so riddled with irregularities that there was no way to tell which of the 17 candidates should have passed. The board's recent audit of the June 2004 exam found that the CHP could not produce the notes taken by two panel members conducting the exam, including former Commissioner Dwight O. "Spike" Helmick, contrary to personnel guidelines.The exam panel had three members. One, who wasn't named, challenged some competitors' responses, prompted others and tossed the exam materials across a desk in a way that one applicant considered hostile, the audit said.
Completed last month, the report found that the exam panel showed possible bias against one applicant, Hubert A. Acevedo.
I'm always amused (in an annoyed sort of way) when testing critics insist that the innocent little objective multiple-choice item is biased and discriminatory, while "performance-based" exams that use "alternate" methods of assessment must naturally be more fair and accurate. This theory, of course, is assuming that the people involved in rating the examinees bring no biases whatsoever to the table, and that the lack of standardization involved in something like this has no effect on examinee performance. As demonstrated here, the beauty of multiple-choice items that are vetted through many developers and pre-tested on many examinees is that those items can be stripped of bias as much as possible before going live. In an oral exam where an examiner does something inappropriate, those safeguards for the examinee just aren't there.
(Hat tip: Darren.)
Posted by kswygert at September 29, 2005 06:00 PM