March 17, 2004

The desperate need for psychometricians

"Hire Ed," an article in the March 2004 issue of Washington Monthly, does a great job of presenting both the pros and cons of NCLB in an unbiased manner. What's more, this is the first article I've seen that emphasizes both the importance of psychometricians and the scarcity of them:

You might expect...that California would have been in a good position to handle a key NCLB provision, that each state rank the proficiency of each of its schools. Instead, when that provision kicked in last year, California stumbled..."It was chaos," says Bill Padia, who heads the [DOE's] policy and evaluation division.

The problem, it turned out, was that the law required the policy and evaluation division to make the calculations much faster than it had ever done before, using an assessment formula many times more complex than it was used to...And because the NCLB law has begun to create intense demand for a limited pool of experienced and knowledgeable testing experts, some of Padia's best people had been poached by testing companies and affluent school systems that could offer higher salaries. Once two-thirds of his 31 staffers had held doctorates, but by the time of the NCLB debacle last summer, only two did, one of whom was Padia himself...

At the end of the article, in the midst of the suggestions for helping NCLB work comes this stunning paragraph:

It is almost impossible to exaggerate just how unprepared these departments are for the task, or how vital the federal government's role in preparing them will be. To take just one example, each state will need teams of specially trained statisticians to oversee the development and administration of state tests. This is crucial not just to improve the very low quality of many tests currently in use, but also to avoid the kind of errors that have befallen California and other states in the last six months. Right now, however, the nation's education schools produce just 36 graduates with these skills each year. These testing experts are the equivalent of Arabic-speaking U.S. soldiers and spies in Iraq: We simply don't have enough of them, and the lack of such talent is costing us dearly. Washington needs to mount a crash effort to create that talent.

Emphasis mine. I've been saying this for a while; psychometric organizations have been addressing it as well. Efforts have been made to recruit more people to the field of psychometrics, but it's slow going. We desperately need more people in the field, and we need ones who are willing to put up with the (relatively) high stress and low pay that accompanies jobs in state-level education departments.

The paragraph above is stunning in part because the media description of tests tend to be so thoughtlessly and thoroughly negative that readers could be forgiven for deciding that psychometricians are hateful, bigoted people who deliberately create baffling, biased items that are guaranteed to be too hard for minority students. And who would want to be one of those? In order to see a rise in the number of psychometricians, there needs to be a change in their public image; this article is a start.

Posted by kswygert at March 17, 2004 06:02 PM