A little bird forwarded me a truly unbelievable letter from one Rebecca McBride DiLiddo, published in the Chronicle of Higher Education. The link is here, if you've a subscription; if not, just read on to enjoy the experience of witnessing something quite boneheaded:
How to Make the SAT 'Truly Colorblind'
To the Editor:
I agree with Jay Rosner that the SAT score gap between whites and minorities reflects bias in the test ("Researchers Charge Racial Bias on the SAT," October 10). As a scientist, I find it intuitively obvious that the SAT must have some inherent bias. The scores simply do not do what they are said to do: accurately predict performance in college for all groups. ...
That's funny, I thought scientists relied on data, not intuition. What on earth makes it "intuitively obvious" that the SAT must have "inherent" bias, and why should we take the word of a scientist who presents no data to back her claim?
What's more, I've already taken apart Mr. Rosner's ridiculous claims that only items which favor white test-takers make it onto the SAT. I won't repeat those arguments here; I'll just note that Ms. DiLiddo's complaint that the SAT doesn't "accurately predict performance in college for all groups" demonstrates as profound a lack of understanding of testing and predictive validity as Mr. Rosner showed for test bias and construction.
How do I know this? The key is Ms. DiLiddo's use of the word "accurate" in this context. Those who understand testing do not talk of an absolute "accuracy;" they speak of validity, which takes several forms. I assume by "accuracy" Ms. DiLiddo means predictive validity, which is never perfect or absolute with this type of measurement in the real world. One measure of predictive validity for the SAT is its correlation with variables such as first-year college GPA; any positive relationship between the two gives you an idea of how much of GPA variability coincides with SAT score variability, and the square of the correlation is the percent of variability in GPA explained by the SAT score. This relationship does indeed vary for different subgroups and different colleges; unfortunately for Ms. DiLiddo, that variability does nothing to prove her point.
For example, if we want to talk data, what we find is that the SAT tends to overpredict non-Asian minority college performance; removing the SAT in those cases could result in fewer such students being admitted. High school GPA also tends to follow the same patterns for ethnic groups as the SAT, so removing the test and just using grades probably wouldn't change anything. I suppose the theory is that grades are "inherently" biased, too?
It's not that there aren't valid criticisms of the SAT to be made. At least one article suggests that the SAT II is more useful in prediction of college grades for University of California freshmen than the SAT I (but yet another article suggests that the SAT and SAT II produce freshman classes of similar performance and quality). Has Ms. DiLiddo ever seen any real data showing the predictive validity of the SAT and SAT II? Or is she content with her "intuition" about the test's "accuracy"?
Jay Rosner gives us an easy way to fix the problem. Simply use the information that the Educational Testing Service already has to develop a truly colorblind test. Use a mix of questions that includes a certain number on which whites do well and an equal number on which minorities do well, and questions on which each subset does poorly. Each group would be equally advantaged and disadvantaged.
Unfortunately for Ms. DiLiddo, this paragraph demonstrates that she knows very little about test bias and measurement error. Tests are not constructed so that there are equal numbers of items that disfavor each group, nor should they be. Items that are biased in this way are measuring something other than the intended construct, and these types of items are not, and never should be, included on tests such as the SAT. Ms. DiLiddo's definition of "colorblind" here is completely perverted; a test in which each item measures some extraneous construct that is correlated with skin color is as far from "colorblind" a test as one can get.
We all know why that won't happen. The SAT is not about creating a level
playing field. It is about maintaining the advantage of an empowered class. To maintain such an advantage in an age of affirmative action, it is essential to find a way to make it look like all groups receive equal treatment while allowing the empowered class to maintain its edge. ...
Yep, that's why the Asians score so well on the math section (and have a higher mean on the Verbal section than women overall). It's because we are determined to make them "the empowered class" in America. That's how those Asians maintain their "edge" in these days when colleges such as the University of Michigan award more points to "under-represented minorities" than to applicants with perfect SAT scores (at least, Michigan used to do this; now they'll have to stop, or just be more secretive). Yep, this is all about keeping those Asians on top.
It is particularly important to expose discrimination in cases where it hides behind the facade of equal treatment. It is time for the ETS to admit the bias of the SAT, fix it, and allow academia to get down to the task of fulfilling its promise to provide equal opportunity to all students.
This statement is as uninformed and unenlightened as it is insulting to the dozens of psychometricians, test developers, and researchers who work for the College Board (not ETS) and who ensure that biased items do not go onto the SAT.
Rebecca McBride DiLiddo
Interim Associate Vice President of Institutional Research and Assessment
Fitchburg State College
Fitchburg, Mass.
Hmm, it's Associate Vice President, is it? Let's see, that leaves us with only three options:
(1) Ms. DiLiddo doesn't have a college degree or attended a college that didn't require a standardized test, thus avoiding the demonic, ruling-class-empowering minions of the College Board,
(2) Ms. DiLiddo did just fine on her SATs, thus benefiting from the "biased" system that she now despises, or
(3) Ms. DiLiddo did poorly on her SATs - but that didn't stop her from becoming "empowered" enough to be Interim Associate VP, did it?
Actually, (1) can be disproven because Ms. DiLiddo received her PhD only two years ago, which is surprisingly recent for someone in her position (and this means she probably did well on the evil GRE as well as the evil SAT).
Looking further through Google, we see that Ms. DiLiddo (or someone with her exact same name and affiliation) is listed as a member of a "think" tank, despite having shown here evidence of rather unenlightened scientific thinking. Her name is also associated (under a different college affiliation) with a few scientific works...on botany and plant biology. Yes, that's exactly the field in which I'd expect to find an expert in the "inherent" biases of standardized educational testing; one who can see right through the devious conspiracies of test developers and who knows that all tests should be constructed so that each test item contains biases based on skin color.
If this is too much sarcasm for you, I apologize. It's just truly mind-boggling (and blood-pressure-increasing) to see this sort of completely uninformed material written by those who should know better, and published by those who should know better (unless the Chronicle figured, as would I, that Ms. DiLiddo's own statements are the most damning thing one can print about her).
Posted by kswygert at November 17, 2003 08:58 PM