I've posted before about what a bad idea it is to tie IQ scores to death penalties. Not only is it absurd to rely on one test score in making a life-or-death decision, but it's hard to imagine that there won't be cheating by both inmate and psychologist:
Are the inmates in the 20 states that currently do not have laws against executing the mentally retarded going to clamor for retests, or will their current IQ scores stand? The potential for abuse is astounding here, and I wouldn't want to be the clinician in charge of testing individuals, knowing that a difference of a few points is indeed a matter of life and death.
IQ scores are not absolute, they're not error-free, and they're not invariant within examinees. It certainly would be easy to fake a low score if the alternative is the gas chamber; a judgment of mental retardation based on such data would be fraught with error. What's more, an inmate could genuinely get smarter over time if the prison had a helpful education program - or if his lawyer helped him learn.( For some criminals of deprived backgrounds, prison is the most instructive and structured environment they've ever known.) If that were to happen, which IQ score should be used when assigning punishment? Should an inmate essentially be punished for improving his mind in prison?
It's not a hypothetical question any more, folks:
Three years ago, in the case of a Virginia man named Daryl R. Atkins, the United States Supreme Court ruled that it was unconstitutional to execute the mentally retarded. But Mr. Atkins's recent test scores could eliminate him from that group. His scores have shot up, a defense expert said, thanks to the mental workout his participation in years of litigation gave him. The Supreme Court, which did not decide whether Mr. Atkins was retarded, noted that he scored 59 on an I.Q. test in 1998. The cutoff for retardation in Virginia is 70.
A defense expert who retested Mr. Atkins last year found that his I.Q. was 74. In court here on Thursday, prosecutors said their expert's latest test yielded 76.
Those scores really don't mean anything unless we're given some idea of the variability of those scores in the population and the standard error of measurement (or SEM). How many SEMs away from 70 is 74? How many SEMs separate 59 and 70? What's the reliability of this exam? Heck, the NYT doesn't even find it necessary to identify which IQ test is being used here, although I'd assume it's the WAIS-R (Wechsler Adult Intelligence Scale - Revised) or the WAIS-III. The WAIS-III shows a reliability of .96 and an SEM of 2.3; the WAIS-R, a reliability of .97.
(Update: Yes, it was the WAIS-III. Via Chris, who has more, and who catches an error in the NYTimes article.)
The SEM - which is the standard deviation times the square root of one minus the reliability, if you're interested in knowing - can be understood as follows: If an examinee were to take the same IQ test repeatedly, with no change in the level of intelligence, it is possible that some of the resulting observed scores would be higher or lower than the score that precisely reflects the examinee's actual intelligence level (i.e., the true score). The difference between an examinee's true score and his highest or lowest hypothetical score is the SEM. The bigger the SEM, the more the observed scores can vary from the true score, and the less reliable the exam (i.e., the more random error there is in the scores). Thus, the SEM tells us how precise an estimate the IQ observed score is for what we think of as a "true" IQ score (and all psychological test scores are exactly that - estimates).
Atkins' jump of 59 to 74 is far outside the range of what is to be expected by chance (assuming he took the WAIS-III). In a bell-shaped curve, there's a 99% probability that the true score lies within 3 SEMs on either side of the observed score. A score of 74 is over 6.5 SEMs away from 59, which suggests the new score really is due to an increase in intelligence, and not an indication of mere measurement error.
Why am I telling you all this? Because the NYT didn't. Sure, I know that reporters rarely delve into the mysteries of reliability within a news article, but how are readers supposed to understand a topic that revolves around test scores unless they are exposed to the statistical bits and pieces that clarify the results?
Mr. Atkins, a slight, balding 27-year-old in an orange jumpsuit, sat slumped with his chin on his hand as lawyers argued about whether his intelligence was low enough to spare him from execution. In 1996, he and another man abducted Eric Nesbitt, 21, an airman from Langley Air Force Base, forced him to withdraw money from an A.T.M. and then shot him eight times, killing him. He will be one of the first death row inmates to have a jury trial on the question of whether he is retarded. The jury's decision will determine whether his life will be spared...
Prosecutors say that Mr. Atkins has never been retarded and that the recent tests confirm it. "I don't see how a 76 is exculpatory and evidence of mental retardation," Eileen M. Addison, the commonwealth's attorney here, said in court on Thursday. "It needs to be under 70."
Ms. Addison has said that Mr. Atkins's crime also proves that he is not retarded. In an interview last year, she said that his ability to load and work a gun, to recognize an A.T.M. card, to direct Mr. Nesbitt to withdraw money and to identify a remote area for the killing all proved that Mr. Atkins is not retarded.
"I don't believe the truly mentally retarded commit these kinds of crimes," she said last year. She did not respond to recent messages seeking comment.
There's what everyone perceives as the meat of the matter, of course - even if the tests say he is retarded, does it matter, when he's demonstrated both the willingness and the ability to use a gun on an innocent person?
This is the first time I've seen an explicit statement court mental retardation judgments (although they differ from state to state):
... Seven states have passed new laws, according to the Death Penalty Information Center. They have adopted essentially the same definition of mental retardation, requiring defendants to prove three things: that their I.Q. is below 70 or 75, that they lack fundamental social and practical skills, and that both conditions existed before they turned 18.
Mr. Atkins was never tested as a youth, and so the jury will have to consider how to look back using his test scores as a young adult.
Nice of them to specify that the mental retardation has to be demonstrated in adolescence, but anyone not tested during that time gets to take the test as an adult and will be judged on that. Atkins may be at a disadvantage for not having youthful IQ scores. Where are all those testing opponents who are normally featured in the papers, insisting that tests are biased against minorities and indicative of nothing? Where are the critics who usually insist that people cannot be judged by one test score alone? Lord knows, they'd actually be of some use here.
Jurors in Mr. Atkins's case, which will be tried this spring or summer, will probably hear from mental health experts, teachers, family members, classmates and, perhaps, victims of some of the 16 other felonies that Mr. Atkins committed when he was 18 in what Dr. Nelson called a four-month crime spree. He dropped out of high school that year, his third attempt to pass the tenth grade.
Not much there to indicate that Atkins is very smart. There's also little to indicate that he should ever again be released into society.
There is apparently research to suggest that the mentally retarded may be more likely to confess to crimes they did not commit. To me, that suggests making life-in-prison and death penalties contingent on more than confessions, not making them contingent on high IQ scores. Despite the high reliabilities of IQ tests such as the WAIS-III, I still oppose the idea of setting an IQ score standard as a "Get Out of The Chair Free" card. The test was never intended to be used in such situations, and the possibility of cheating, abuse - and thorny questions such as what to do when IQ changes - arise.
Update: A roundup of opinions on Atkins, mostly the original court decision; I'll update with more recent posts as I find them.
Jenny D says this is good news for education, if not for Atkins.
Chris Correa: "...measurement error is a real issue and there is a problem with high-stakes testing in the courtroom."
ThreeDogBlog discusses the aspect of letting the jury decide on Atkins' intelligence level.
Boots And Sabers think we'll soon see an outbreak of mental retardation among defendants.
Gut Rumbles wonders when dumb stopped being a correlate of violent crime and started being a defense.