SAT redesign and California's desperation
Mark Goldblatt reports on the changing SAT in the Jewish World Review. CNN had their version of the same story back in March. Mark's version is much more interesting and informative, because he goes into detail about criticisms of the test and alleged reasons for the change, whereas CNN glosses over the controversy and presents only one side of the story by including a tired, obligatory quote from FairTest at the end.
The JWR article mentions the three charges most commonly leveled against the current SAT, and I'd like to supplement the response.
"1) The exam is culturally biased against minorities...[Mark] except that Asian students consistently outscore white students."
Once again, those who don't agree with the test results feel free to use the psychometrically-precise term "bias" to mean whatever they want it to mean (I'm speaking here of the criticisms, not Mark's reply). Let me explain this one more time - group mean differences on a test is not proof that a test is biased. It is only proof that one group, for whatever reason, is performing worse on average than other groups. See my post on 5/20 for a discussion of reasons.
"2) The availability of SAT preparatory courses skews the scores of students from families affluent enough to afford them...[Mark] except that, according to the latest evidence, the average gains of students who take a prep course are 6-12 points on the verbal section and 13-26 points on the math section — out of a possible 800 — and that comparable gains can be achieved simply by taking the test twice."
I wish I had the citation for that evidence, because it does agree with the data that the testing organizations collect, rather than the data the test preparation companies trumpet. I'm a capitalist, so I don't begrudge the test prep companies their desire to make a buck. However, I'm in awe of those who have the nerve to take overinflated test prep score-gain claims at face value and then present that as "legitimate" evidence that the tests, and not the test prep processes, are somehow biased.
" 3) The exam measures only the ability to take the exam and doesn't accurately forecast future success in higher education...[Mark] Now in a limited sense, this is true. Survey after survey has shown only a slight correlation between SAT scores and college grade-point averages or graduation rates. There is, however, a vast logical leap from acknowledging that the SAT does not predict grades or graduation rates to concluding that it predicts nothing except the ability to take the exam."
Thank you, Mark Goldblatt, for directly stating what so many test critics willingly miss, which is that just because a test is not perfect does not mean it is not useful. We have a saying in psychometrics - "All models are wrong, but some are useful", which translates here to, "All tests are imperfect, but some are useful". Mark even goes on to explain the restriction-of-range phenomenon with correlations (although he doesn't use that exact term), which is another thing I rarely see in all the testing debate in the media. Great job.