More California testing follies
A startling figure from CA's state high-school exit exam results - more than 90% of disabled students did not pass the exam. That's the big, attention-grabbing headline in the SF Gate, which gives plenty of attention to trigger-happy lawyer Sid Wolinsky, who's suing California on behalf of all 173,470 of its disabled high-school students.
You see, beginning in 2004, California's student will have to pass an exit exam in order to receive a diploma. They get three chances per year, meaning 12 chances in all, and despite the fact that California's schools extend to the 12th grade, the exit exam is pitched at the 10th-grade level. But, California has "nearly 580,000 students in all grades have a variety of impairments, from learning difficulties such as dyslexia to physical disabilities such as blindness", and I bet Sid would like to file a lawsuit on behalf of all of them. Despite that, while I realize that all those categories listed in the article qualify as disabilities, they certainly aren't equal with respect to the abilities required to pass a 10th-grade-level exam, and they certainly don't preclude the possibility that a student can have a disability and lack the cognitive ability to pass the exam.
Sigh. This is indeed a mess, but it isn't the fault of the exams, nor even necessarily of the state of California. The Americans with Disabilities Act (ADA) requirements specify that test developers remove any and every obstacle that a disability might pose to a test-taker, but the exam itself must test the same constructs for disabled and non-disabled test-takers alike. When it comes to physical disabilities, such as blindness, it's obvious that it would be blatantly unfair not to offer a blind student a test in Braille, or on audio-cassette, or with a personalized reader. Such disabilities pose mainly logistical problems for test developers and administrators, and once the test is modified to be fully accessible to a blind student, it's easy enough to support the decision to pass or fail the student on purely cognitive grounds.
It's the cognitive disabilities, and lawsuit-happy organizations such as Disability Rights Inc., who are causing most of the problems for test developers. How on earth do you decide what cognitive obstacles to remove from a test that is meant to be a measure of cognitive ability? As Rae Belisle, the counsel to the state Board of Education, puts it:
Belisle says that even though 90 percent of disabled students who took the exit exam in March did not pass it, no one actually "failed" it.It is true that some students have cognitive impairments so severe "that they won't get a diploma," she said. "That's the reality of it." But included among the tens of thousands of disabled students who did not pass the test are some -- she did not know how many -- who answered enough questions correctly to be eligible for a waiver from their local school board, allowing them to graduate.
These students did not actually pass the test because they took it using a "modification," Belisle said. A modification may be a device, such as a calculator, or even a human, such as a reader for a blind student, without which a student would be unable to take the test. Some students may require a calculator because their own brain cannot perform the calculation process. Experts are in wide agreement that such difficulties are organic and not a measure of intelligence or laziness.
Nevertheless, "using a modification changes what the exam is measuring," Belisle said, so that if a student uses a calculator or reader, the test is not measuring how well the student can read or do math. By contrast, an "accommodation," such as extra time or special lighting, does not change what skills are measured, she said.
Interesting. This is the first time I've seen a distinction made between accommodations that do not change the construct, which is what is required of test developers by the ADA, and accommodations that do change the construct, which is not required of test developers and which don't make a lot of sense to boot. I disagree with Belisle about the timing issue, because changing the time limits may very well affect the test construct, but it's possible that in this particular case, the timing won't make much difference.
Anyway, before you FairTesters out there compose your angry emails in response to my criticism of the disabled and their advocates, let me make a few comments:
1. According to the SFGate article, 90% of the disabled students did not pass the test. For some percentage of them, this was an accurate representation of their scholastic abilities - presumably, a percentage roughly equal to the non-disabled students who did not pass the test. When we compare those two numbers, it's really only the difference that should concern us.
I looked on the CA Dept. of Education website. The only press release for today that I could find doesn't say anything at all about disabled test-takers. It does separate out the "Special Education" students, and while 87% of those students failed, 48% of students who aren't considered Special Education failed. This, to me, indicates a miserable state of affairs for all students, not just the disabled ones, if in fact I'm suppose to equate "disabled" with "Special Education" and I'm not sure that I am.
What's more, one reason that there were so many non-passing is that students who took the test with special modifications for a cognitive disability, such as the use of a calculator on a math exam, are automatically judged as non-passing, and must pass a certain number of items and obtain a waiver. Sid considers the "test not valid" label given to the modified exams, and the waivers, to be indicators of "second-class status", rather than a vain attempt by California to preserve some sort of integrity for the test as it stands.
2. The reason I refer to organizations such as Disability Rights, Inc., as "lawsuit-happy" is because they seem unable to concede point 1, above. Not every student with a learning disorder flunks a test because the test is unfair. Somehow, the burden of proof for acheivement and ability seems to have shifted from the test-taker to the test developer. We're put in the position of trying to prove that someone with a cognitive disability really and truly doesn't have the cognitive skills that the test is trying to measure. We're put in the position of drawing the line between "cognitive disability" and lacking in cognitive function. And thanks to such lawsuits, it has apparently become politically incorrect to suggest, via test scores or otherwise, that someone with a cognitive disability might not deserve a high school diploma, because they lack the cognitive skills in addition to the disability. I don't mind a requirement that test developers try to remove obstacles for cognitive disabilities; I do mind the recent assertions that test developers are pushing "second-class status" on everyone with a cognitive disability who flunks a cognitive-skills-based exam.
3. Rae Belisle hit the nail on the head, psychometrically. A test is only valid if it measures what we intend it to measure. If we modify it beyond all recognition, then it no longer measures what it's supposed to measure, and there's no point in administering it. I am not in favor of high school exit exams, but if a state insists on having them, every test-taker should be held to the same standards. Otherwise, you might as well follow Nova Scotia's example and issue "special" diplomas to students who can't read (via Joanne Jacobs). (David Janes cites this as evidence in the Most Moronic Province Contest).
The point of high school exit exams is so that schools can claim that every student who receives a diploma - not every "temporarily abled" student, but every student - was exposed to a certain curriculum and performed up to a certain set of standards for that curriculum. If as Sid Wolinsky claims, Special Education students are not being taught the material on the exam, or if the exam is going to modified for a subpopulation of the students, then the exam results are invalid, and the exam shouldn't be administered.
4. If students with cognitive disabilities receive special treatment, students without disabilities will have incentives to fake such disabilities. Students who would not dream of faking blindness or motor disability may not have a problem with getting a learning disability diagnosis from a doctor, if it's going to give them a free pass through an exam. There are rumors, and anecdotal evidence, to suggest that the sharpest rise in cognitive-disability diagnoses is among the wealthy, white, upper-middle class kids. Ten percent of the California kids were enrolled in Special Education. Will that number rise in the future?
UPDATE: An Interested Reader wrote in to say, "I assume you saw that headline in the NYTimes last Thursday about rich kids paying to get learning disability diagnoses". Um, er, uh - of course I did! Yeah, just thought I'd give the story a little time to breathe before blogging about it. Yeah, that's it.
No, that's not it. I've been swamped, and it's been difficult to post something everyday. Many thanks to the reader who notified me of this story, because there's no excuse for me to have missed it. Unfortunately, I can't link to it because it's now a payment-required article. What's happening is exactly what I predicted would happen once the College Board announced that it would stop flagging SAT scores given under extra time. And the NYTimes coverage is just what I would have predicted - according to the article, the fact that rich kids are willing to pay people to essentially help them cheat is just more proof that these tests are inherently biased against poor students and further "privilege the privileged". Um, no. The College Board caved under pressure from disability advocates and the politically-correct, with the predictable result that learning disability diagnoses have become more prized. I don't see it as an indictment of the test so much as it's an indictment of the College Board's decision, and I think the politically-correct, anti-test philosophy pushed by FairTest and the NYTimes make it easier for diagnosis-fakers to justify their actions.
How can the NYTimes run a story on the flagging of accommodated test scores that doesn't contain one single quote about the psychometric reasons for flagging, a story that suggests that test developers are evil for making any distinction between accommodated and non-accommodated test scores, and then turn around and run this recent story, which bemoans the booming business in faking disabilities? Hello, doesn't the Times see that one necessarily follows from the other? If it's "wrong" to make a distinction between disabled students and non-disabled students, all students will have the incentive to fake a disability. This is the bed you helped make, NYTimes and FairTest. The money angle is the least important angle to this story (probably why the Times took it), because the educrats will make sure that fake diagnoses are available to all as soon as possible.