You know, I'm still feeling under the weather, with low energy and even lower mood. But Devoted Reader Bas Braams knows what will get me up and shaking my fist at the world - SAT criticism that includes obsolete items. This LA Times article drags up, yet again, the infamous but long-gone "regatta" item as part of current criticism against the test:
...the National Center for Fair & Open Testing...contends that the SAT is biased against lower-income students and those for whom English is a second language. As evidence, center public education director Robert Schaeffer cited several analogy questions from over the years, including this one, since deleted:
RUNNER: MARATHON ::
A) envoy: embassy
B) martyr: massacre
C) oarsman: regatta
D) referee: tournament
E) horse: stable
The answer was C.
"That's incredibly culturally centered," Schaeffer said. "You don't see a regatta in center-city L.A., you don't see it in Appalachia, you don't see it in New Mexico."
I've emphasized the phrase "over the years" above, and for good reason. It's an attempt to drag up old news and disguise it as current criticism. This item hasn't been on the SAT for at least 13 years, perhaps longer. For at least that same amount of time, SAT items have undergone review for differential item functioning in order to root out these types of items - something the writer doesn't mention here.
According to Schaeffer's reasoning, because a few SAT items were once perhaps biased, we should assume that most of them still are. What 's more, no evidence is given here to show that at the time the item was written and used, it was in fact biased. If the word "regatta" was once on vocabulary lists, then it could have been unbiased, and there's no shame in the College Board deleting items once they're obsolete.
What Schaeffer is really saying is that now it isn't politically correct to insist that disadvantaged kids learn the word "regatta," and it's odd to see that taken seriously as test criticism.
Schaeffer also doesn't seem to understand the very basic purpose of the SAT, which is that it measures English comprehension. Therefore, kids who don't know English well should do worse on the SAT than kids who do know English. Yet, in his mind, this is "bias" against non-native English speakers, rather than evidence that the SAT is a valid measure of how well a kid knows English. Funny how test critics use the word "bias" so loosely, isn't it? ETS and the College Board should put the word "bias" in an analogy item so that the situation will come full circle.
Sadly, this article is somewhat commendable because writer Paul Pringle at least hints that the offending item is no longer in use. Some writers don't even go that far, as can be seen in this article; an unsuspecting reader might believe that this item was still in use.
(By the way, there is a regatta in New Mexico. Just so you know.)
I'm also not impressed by this tired argument:
...the SAT is often the target of complaints that material like the analogies can be professionally coached. Many say that gives an edge to students whose parents can afford tutors and prep courses.
"As soon as you start exposing some kids to the methods behind the questions, there start to be unfair advantages," said Larry Berger...He told of sleuthing for patterns to answers to boost his chances of guessing right on questions that flummoxed him.
The analogies, he said, are spiked with "distracters" — words designed to trip up students. An example: A question begins with the prompt "COVEN: WITCHES." The possible answers include "amulet: vampires," but the correct choice is "choir: singers." "The student is distracted by the superficial relationship between witches and vampires," Berger said.
What? You mean the SAT is a test for which one can actually prepare? Horrors! I eagerly await an alternative test from these critics - one on which preparation affords no advantage, so that a kid who practices under time limits and familiarizes himself with the tasks will do the same as a kid of similar aptitude who is completely unprepared for the occasion. After all, we wouldn't want to reward extra effort, would we?
Oh, and that "superficial distractor"? That's, um, the point. An amulet is not composed of vampires. The occult reference in this response makes it an attractive distractor, but a kid who understands the relationship won't pick the wrong option. I suppose Berger's ideal test item would be one in which all but the key response are clearly wrong. So - the test should not reward practice, and all correct responses should be very obvious. Have I got all this straight?
And isn't it interesting how, when private companies that are completely unrelated to the testing companies charge an arm and a leg for tutoring, it's the testing companies that suffer the criticism, as though they're the ones gouging customers? And this is despite the fact that the test prep companies have yet to present solid data attesting to the efficacy of their methods. Reporters almost never pick up on this misdirection of bile.
Amazingly, this article even treats us to the sight of Princeton Review assistant VP Jeff Rubenstein professing his "gall" at the sight of certain SAT analogy items - but the staff writer doesn't point out that Rubenstein isn't galled enough to refrain from helping run a company that charges students hundreds of dollars to learn how to answer these items.
If the items really galled him, don't you think he'd work for peanuts - even free?
(Joanne Jacobs also covered this article in a much more succinct fashion than I did. She always has better titles, too - "The regatta is over." Heh.)
Posted by kswygert at July 28, 2003 06:00 PM