Discussions of test bias
John Rosenberg of Discriminations has pointed me towards an interesting blog, the Kitchen Cabinet, and, specifically, a post about LSAT scores for various ethnic groups enrolled in Michigan's law school (scroll down to "Affirmative Action Numbers" if this link doesn't work). The blogger, Kate Malcolm, notes that out of 4,461 law school applicants who obtained a score of 165 or better on the LSAT, and who had a GPA of greater than 3.5, only 29 were black and 114 were Hispanic. While I find nothing wrong with her posting the numbers, I do take issue with how she has chosen to question the numbers.
She writes:
Is there a bias in the LSAT? What I would first like to know is how many black students with GPAs over 3.5 are scoring below 165 on the LSAT as compared to the overall number of law applicants with those numbers.
Nothing wrong at all with asking that question - but as I've pointed out before, that has absolutely NOTHING to do with test bias. Group mean differences, or differing score distributions among different ethnic groups, is NOT evidence of test bias. If fewer black students make LSAT scores above 165 than do members of other ethnic groups, this is NOT evidence of test bias.
Large-scale standardized admissions tests are, in general, not biased, because there are processes in place which remove items that show differential functioning. Differential item functioning, or DIF, is how psychometricians define bias, and DIF is defined as items that perform differently for examinees in different groups who are at the same level of overall ability. That last point is key, and if you want to read more about it, click here, or here, or here.
All large-scale admissions testing programs, including LSAC, review items for DIF, and items which show DIF are not administered (there are other ways to assess bias, and rest assured the testing companies produce research showing that these other methods also indicate no bias). The LSAT is not biased - but there are differences in the score distributions, meaning that certain subgroups of minority examinees do not, as a whole, do as well on the exam as the subgroup of white examinees.
Now, because the exam is used for admission to law school, the exam does have differential impact on those minority groups - but that is not bias, and that does not mean that the group differences are not "real". The reasons behind these differences are hotly debated, but the fact remains that these groups differences do exist, and they exist on an exam that measures the same constructs (analytical reasoning, reading comprehension, and logical reasoning) for each examinee, regardless of race.
Next point:
Is there a bias in GPAs? This would be a far more outrageous and endemic problem. Useful to this inquiry would be how many black students are scoring above 165 on the LSAT, but have a GPA below 3.5, as compared to the overall number of law applicants with those numbers. Richard Atkinson, the President of the U California system, sheds some potential light on this: The most recent study found that 30 percent of Asian American students in California and 13 percent of white students met UC eligibility requirements; the figure was a disheartening 4 percent for Latinos and 3 percent for African Americans.
Again, I don't think she's using the term "bias" correctly. Is she asking whether minority subgroups are less likely to earn high GPAs? Once again, group differences don't mean bias. For one thing, if there were a substantial number of minority students with LSAT scores above 165 but below 3.5, this simply identifies such students as "under-achievers." That's the term testing companies use for students who have the skills (as evidenced by the test scores), but who don't demonstrate those skills day-to-day by working hard in school and raising their GPA. It's a label, not an insult; I was one myself when I was in high school.
This sort of discrepancy could also be seen as evidence that minority kids who are smart are nonetheless stuck in boring schools or schools which do not adequately reward their achievements. I think that the theory of bias - that teachers are systematically reluctant to give high grades to minority students who do well - is far down the list in terms of possibilities.
Finally:
...are there simply fewer black students with high numbers applying to law school? If so, why? A starting point here would be how many black students with GPAs over 3.5 do not apply to law school…
This doesn't seem to have anything to do with her initial point. The problem is not that high scoring blacks do not apply to law school. The problem, according to LSAC's own data, is that relatively few black students are meeting the criteria of high-performing students. I agree with Kate entirely when she says, "29 black law applicants with high numbers tells me only one thing: something is wrong with the system." However, I feel that affirmative action does not fix "the system", which is why I do not support AA. The reforms need to begin at the college, high-school, or even kindergarden level, so that at-risk minority students can become better prepared for academic work.