The Godless Capitalist strikes again
Major dude Godless Capitalist fact-checked Mark Goldblatt at JWR and sent me the results. Charitable as always, GC prefaced his email to me by saying, "I'm surprised you didn't mention [Goldblatt's error]", which is a nice way of saying, "Hey, take a closer look at the data next time before you praise someone's work." A good thing to remember.
However, I'm going to play Devil's Advocate again and suggest that Mark's main point was correct, although some of his statements were incorrect. Here's the relevant part of Mark's argument:
"Survey after survey has shown only a slight correlation between SAT scores and college grade-point averages or graduation rates.[...] The predictive factor of the SAT is very strong at the extreme ends of the scoring spectrum. So, for example, a student who scores a combined 750 is far likelier to flunk out of Dartmouth or Stanford — or, for that matter, any accredited liberal arts college — than a student who scores a combined 1500. Now imagine a school that attracted equal numbers of 1500 and 750 scorers. Do you think there would be a noticeable correlation between SAT results and grades? Or between SAT results and graduation rates?[...]In the real world, of course, there is no such school — since elite universities utilize standardized tests to screen out most applicants who score below, say, 1300[...] It's this very screening process, however, that undermines the SAT's ability to predict grades and graduation rates since it ensures a relative homogeneity among students at any given college. Once the pool of students is narrowed to those who scored between, say, 1100 and 1300, then variables such as home environment, discipline, and maturity — which the SAT cannot measure — tend to override the statistically minor deviation between, say, a 1130 student and a 1170 student."
Now, take another look at the numbers that GC cites. Those numbers are indeed higher than Mark seems to be giving the SAT credit for. It's entirely possible that Mark didn't take a look at the College Board data, or that he did, but didn't consider a correlation of (for example) 0.52 to be high, when in fact it's huge (from a social science perspective). So GC was correct to call him on that.
However, I really think Mark was trying to make a different point, which is that these correlations are not the same from school to school, and that the more restricted the range of SAT scores for admitted students, the smaller the correlation. After all, the correlations listed by the College Board are weighted averages across schools, and no measure of variability is given. Mark's point was that some critics of the SAT may think they've proven their point by referring to this school or that school that shows a low correlation between SAT and GPA, when the cause might be the restricted range of SAT scores. However, he was remiss in not getting the facts straight from the horse's mouth and reporting the College Board numbers, which also refute the critics.