May 21, 2004

A biased article about "biased" items

This Times-Record (AR) article is about cultural biases on tests, but no evidence is provided to support the charges:

In remarks in Van Buren last week, President Bush said he is committed to narrowing an “achievement gap” demonstrated in standardized test scores among the nation’s students. To some extent, the gap is the legacy of decades of segregation, some area educators said...

Jim Hattabaugh, Mansfield superintendent, said his experience as a guidance counselor and administrator has shown him that most standardized tests have cultural and racial biases. Minorities tend to perform worse than whites on the test because the tests are authored by whites, he said.

And does he have proof of that? Does he know for a fact that each and every single item writer is white? Does he have any knowledge about the extensive item bias review that all standardized test items undergo? Is he aware that item writers, test developers, and psychometricians come in all sizes, shapes, sexes, and ethnicities? Does he have any evidence whatsoever to prove the causal relationship between the skin color of the item writer and performance of examinees of different colors on the exam?

I doubt it. He's saying this because he knows the reporter will not challenge him to provide proof.

“If you look at how an inner-city student is raised and what they’re exposed to, you ask those questions on a test and they don’t have a clue,” he said.

And why is the only legitimate explanation for that cultural bias? Couldn't it also be because those inner-city schools are not teaching children to read, they're failing to introducing them to concepts outside their narrow environment, they're not expanding their vocabularies, and they're teaching these kids that you have to "be white" to do well on tests?

“There could be some possibility of a cultural bias,” said Lavaca Superintendent Harvie Nichols. “I know, for example, when I look at textbooks, it’s difficult to understand, even with a college degree, what the textbook wants that child to do.”

What the heck does this mean? It could be that it's a bad textbook. It could be that Nichols college degree wasn't too useful. But why should we assume this anecdote proves that textbooks are culturally biased? Textbooks undergo even more review for bias than do test items. If Nichols can't make heads or tails of the textbook, then he should order new ones, instead of insisting that cultural bias must be to blame.

This is an example of shoddy reporting. The claims of cultural bias against tests in general - including the outrageous and inaccurate charge that only white people write test items - go unchallenged by the reporter, as does the unspoken assumption that inner-city students should never be expected to understand material that is not specific to their very narrow sphere of experience.

Posted by kswygert at May 21, 2004 01:47 PM
Sitemeter