Number 2 Pencil

A provocative answer to an accommodations question

Education Next asks the question, what was the result of ETS dropping the "flags" for accommodated SATS?

When the College Board announced, in the summer of 2002, that it would stop “flagging” the test scores of students who were given special accommodations for the SAT, the gold standard exam for college admission, disability advocates were thrilled. “A triumphant day for millions of people with dyslexia and other disabilities,” exclaimed Thomas Viall, the executive director of the International Dyslexia Association. “With the ‘scarlet letter’ gone, people with disabilities are given the chance to succeed, based on their abilities”...

Indeed, the scarlet letter disappeared in October of 2003, but not everyone was so sanguine about the possible consequences. Miriam Freedman, an attorney specializing in issues of testing, standards, and students with disabilities, expressed the concern of many academics and practitioners (“Disabling the SAT,” Education Next, Fall 2003) that the deflagging decision would drive requests for special accommodations skyward as more students saw an opportunity to secure an advantage without anyone knowing it.

Ms. Freeman was not the only one predicting disaster - but did that disaster come true?

Surprisingly, now that detailed 2004 SAT results have become available, it appears that growth in special accommodations is not the real problem. As Figure 1 shows, dropping the flag did not accelerate the already steep climb in the numbers of test-takers given special accommodation—it appears to have reversed it. What happened?

As it turns out, the College Board, worried about the rush to accommodation, tightened the criteria high-school counselors and other professionals were to apply when granting waivers...

Ah. So the flags were dropped - but the College Board made it much tougher to get the accommodations. Did this change stop the expected surge of non-disabled students seeking fake diagnoses - or is it now the case that only the more well-off students can afford all the doctor visits that are required?

As the criteria for special accommodations permission has tightened, there are clear signs that the social composition of those given such permission has been altered. By abolishing any stigma that might come with a flagged test, while tightening access to special accommodations, the College Board has given new opportunities to the strategic, while leaving behind the less savvy and less financially well-endowed.

There is no statistical smoking gun indicating that the sophisticated are being given special treatment. But the test-score results presented in Figures 2 and 3 should give College Board officials cause for concern.

The test score results show that scores increased for students granted accommodations, and only for those students. What's more, College Board data indicates that examinees who are awarded special accommodations are 84% white, as opposed to 68% for standard examinees. If one wanted to make the argument (as Education Next apparently does) that the wealthier students are now taking advantage of the no-flagging rule to get accommodated tests, these data could certainly be used to support that argument.

What I said in 2003 seems like it applies now, even more so:

Just think about those "high-powered" parents who are so eager to have their kids labeled as disabled. These parents have $3000 to spend, and yet they can't manage to find tutors for their kids so that they can learn to take the SAT under normal timing conditions? Am I alone in finding it odd that in one generation, we have swung from the label of "disabled" being a stigma to that same label being something that is seen as desirable, and sought after? Do these parents really have so little respect for the test that they're willing to essentially help their kids cheat? Or are they so desperate to think of their kids as "special" that any means of setting them apart from the general population will do?

Performance is the key

Devoted Reader Shakir noticed this Dynamist.com article on accommodations in admissions testing for professional schools:

Over the past decade students with learning disabilities have gotten used to having extra time on tests and, in some cases, separate rooms to reduce distraction. In many cases that makes sense. Giving a dyslexic third grader extra time on a standardized test makes it more likely that his answers will show what he knows rather than how fast he reads.

But a sensible accommodation for little kids can create a misleading double standard for adults. How much you know isn't the only thing that matters in school--especially when you're training for a demanding professional job. What patient wants a genius doctor who can't focus in a distracting environment, reads so slowly that she can't keep up with medical journals or tends to misspell drug names on prescriptions?

Read it all, for it is interesting. The point below is particularly appropriate to the argument - yet it's not one you'll often see explicitly stated:

That argument [that tests only measure how good a test-taker you are] denies the fundamental reality of professional schools. No matter how theoretical their classes, these programs aren't about learning for learning's sake. They're trade schools that prepare and certify people for demanding jobs. In those jobs, performance--not intelligence or knowledge--is what matters.

High-stakes malingering

The rumor mill at UCLA is working overtime, and it's all about students faking disabilities to boost their test scores:

It's a clandestine rumor that is whispered in the stacks of Powell Library or passed around the table at a study session in Covel Commons.

Some students seem to think that the greatest shortcut on the path to graduate school is to fake a disability in order to obtain accommodations when taking standardized tests...

Very sad, but probably true (reporter Daniel Miller's hypothetical examples are pretty funny, though). Students probably do think that faking a disability is perfectly okay as long as one gets a boost from it - but that isn't as easy a task as it sounds:

...after talking with several psychologists and analysts who administer psychological evaluations that determine a patient's disability and the need for accommodations, it has become apparent that the notion that it's easy to lie about a disability to gain accommodations – and that this is a prevalent practice – is a myth.

"I would feel cheated out of all my hard work if I knew about people who lied to get extra time," said Ellie Altshuler, a fourth-year political science student who took the test Saturday. On Sunday, between sips of a celebratory mimosa, Altshuler also said that she did not know of anyone who got undeserved accommodations for the exam. LSAT takers needn't worry about malingerers distorting the test's curve because the scores of students who take the test with accommodations are not considered in the curve for the general exam.

Nice to see a student reporter who's done his research - he had to have contacted LSAC to get the information about accommodated test takers not being used in the final equating of scores. And I'm very glad to see someone put the word out that most testing companies have a series of very rigorous hurdles that one must pass to obtain an accommodated test. LSAC's in particular are quite extensive, especially for cognitive disabilities.

However, there's two groups that Miller forgot to talk with - parents and admission officers. It may very well be that Miller can't find anyone who knows anyone who actually got an undeserved accommodated test - but that doesn't mean they're not trying. Testing companies track the number of accommodated test requests each year, and the requests for accommodations based on cognitive disabilities has been increasing. Admissions officers, too, might have interesting information on any increase in the number of students who request accommodations after being admitted.

Is it easy to fake such a disability? No. But my guess is that people haven't stopped trying, nor will they.

When your MD needs soft lighting, and extra time

A recent California court ruling has disability advocates seething and accusations flying in all directions:

An Oakland judge has refused to mandate special accommodations for a pair of attention-deficit students scheduled to take a national medical school exam Saturday.

While it would be logistically simple to provide the San Francisco residents with a private room and proctors, allowing states to independently determine who gets preferential treatment might damage the integrity of the standardized national testing system, the judge decided.

A judge in California decided that maintaining test integrity was more important than placating the disabled. How's that for a "man bites dog" story? And let the carping and kvetching begin!

"This is a pretty remarkable ruling, in that it basically eviscerates any state's right to protect the civil rights of their citizens," attorney Stephen Tollafield of Disability Rights Advocates in Oakland said Tuesday. "Judge Sabraw basically said it is more important for a company to have uniform standards than for Californians to be protected under their own state's laws."

Well, yes, that is what he said. In essence, he said that the AAMC has the right to say that all examinees should be held to the same standards of proof of disability, and that examinees don't get to enjoy relaxed standards on a national exam due to their state of origin. The articles claims that these standards are clear, and that 75% of accommodations requests for the April exam were approved. You can peruse the standards here on the AAMC website for yourself (as well as the AAMC's response to the lawsuit).

Of course, the cut-and-dried language of the standards doesn't stop critics from complaining that that they aren't sure what the standards are. And the comments about how well the plaintiffs did under standard testing conditions muddies the waters:

[Dr. Ellen Julian, director of Medical College Admission Test] pointed out to Sabraw that Cashmore, Lebovitz and a third plaintiff, Andres Turner, took the April test under the standardized conditions. Lebovitz scored higher than 75 percent of his peers. Cashmore's score was higher than 70 percent of the other test takers, and Turner finished ahead of 51 percent of the other aspiring medical school students, according to Julian.

Tollafield contends the case is solid, and California law protecting the rights of those with disabilities should be enforced even though the medical school entrance exam is a national test...

"They decide the person is too smart to be disabled and will decline accommodations on the test," Tollafield said. "Intelligence has nothing to do with disability."

Technically, yes. But this isn't a test of intelligence; it's a test of knowledge related to pre-med concepts. MCAT scores may very well correlate with IQ scores, but this isn't an IQ test.

What's more, I don't think Dr. Julian mentioned the plaintiff's MCAT scores as a way of justifying the fact that they did not grant accommodations to them; after all, the AAMC could hardly have known their scores in advance. If the plaintiffs didn't have evidence at the level the AAMC required (no matter what it might be otherwise in California), the plaintiffs don't get accommodations. The scores were most likely mentioned in order to point out the defendants were not that impeded by the lack of accommodations, which does lend some credence to the AAMC's claim that these students aren't disabled. But unless the plaintiffs could somehow present some evidence that the AAMC was incorrect in refusing their request for accommodations in the first place, they don't have a case, and so I believe the judge's ruling was the correct one.

Too quick to label

Joanne Jacobs points us to a couple of authors who believe that dyslexia is rapidly becoming trivialized by people who don't want to work hard - or who are handicapped only by ineffective educational programs.

First up is James Panton, who insists that our lowered standards for academic excellence and increased tolerance of disability labels go hand-in-hand:

In secondary education and at many universities, we place decreasing emphasis on reading books or writing essays, so perhaps we should not be surprised that a number of even the brightest students display weak literacy skills. But the tendency to label these students 'dyslexic' trivialises the experience of those who really suffer from a serious learning disability, and leads to excessive demands on special recourses.

However, the expansive use of the dyslexic label has a far more serious impact upon the educational climate within the university. Our preparedness to define a broad range of students as having special needs encourages a climate of special pleading, and lowers students' expectations of themselves. The hours spent reading books in the library, and the effort required to organise thoughts and ideas into a written argument, can be difficult, but struggling with these difficulties is an essential part of higher education. By labelling students who find such challenges particularly onerous as 'dyslexic', we encourage them to understand these challenges as beyond them. Rather than learning from their mistakes, and being encouraged to overcome their weaknesses, the label 'dyslexia' provides a readymade excuse for poor work.

This commentary at first upset Liz Ditz, until she reconsidered:

At first I raised my hackles, and then I reconsidered. I think he's on to something. There's something about "special" and "victim" running around both American and British culture that is undefinably there.

And let us not, in the American case, overlook the damage caused by whole language reading instruction. I believe that there are a lot of borderline cases--kids who need explicit, direct, structured instruction in the relationship between sound and symbol--but who are not truly dyslexic and don't need nine yards.

All I know is that the number of examinees (on tests such as the SAT) requesting testing accommodations has skyrocketed lately, and that's not because there's a whole lot more examinees in wheelchairs showing up. The most popular request for accommodations is extra time for a learning disability, and there's evidence to suggest that affluent parents are more likely to push for these "boutique diagnoses."

I don't think I've ever encouraged readers to go back and read everything I've written on a topic, but if you've never read anything of mine that concerns test time, accommodated tests, and bias, I urge you to start with the link above and do so (and don't skip the comments). I think it will be well worth your time.

Elitism vs. equality in college educations

Amardeep Singh of No False Medicine comments on a very interesting NYT article on affirmative action, "Diversity's False Solace". (Discriminations doesn't yet have any commentary up on this article, but I'm sure some soon will be.)

Anyway, NYT reporter Walter Benn Michaels believes that AA policies are a "solace" to society because they suggest that racism is the real problem. But Michaels believes economic inequality is the real issue:

This, if you're on the right, is the gratifying thing about campus radicalism. When student and faculty activists struggle for cultural diversity, they are in large part battling over what skin color the rich kids should have. Diversity, like gout, is a rich people's problem. And it is also a rich people's solution.

Singh finds Michaels' arguments to be provocative but wonders where the solutions are:

There is no question that Benn Michaels is pointing to a real problem with affirmative action as it is currently conceptualized by left-leaning academics. But what Benn Michaels doesn't, or won't, address is how to realistically respond to the problem of economic disparity.

An obvious solution (to the larger problem) is to use multi-variable affirmative action, whereby wealthy universities would aim to achieve both economic and racial diversity, preferably not with the same students (i.e., admit wealthy students of color and poor caucasian students)...

Some might suggest a socialized university system, similar to European models, to fix this problem, but it's unreasonable to believe that this could happen in our society:

Sorry, but I don't see it happening. Call it false consciousness, but most Americans (and many people abroad) have a great deal of pride and awe about the wealthy universities, precisely because they are so powerful and elitist. Economically and politically, these universities have never been stronger and more influential -- and that includes the university where Benn Michaels himself taught for many years, the very wealthy and elitist Johns Hopkins University.

The system may be ugly, but it is surprisingly healthy.

I remember the first time I took a Canadian friend on a tour of Princeton University's campus. I thought he'd appreciate the grand buildings, historical sense, and gothic flair of the surroundings. Instead, all he did was puke and whine about "rich people" for an hour. As though everyone who ever attended Princeton (a) was filthy rich through inherited funds and (b) never gave anything back to society by, say, doing research, starting businesses, or creating jobs. As though everyone at Princeton should have given all their money to the poor, dressed in rags, and built a campus out of plywood. As though the money flowing around at Princeton didn't help stellar research organizations like the Institute for Advanced Study develop (even though it's a separate non-profit organization).

No, to him, Princeton was just the playground of the rich, and rich persons were to be hated. But to me, the university is a symbol of a healthy and truly diverse American college society that has a variety of college options for our very heterogenous population. The fact that some of our elite colleges are priced out of the range of most Americans is, to me, not bothersome, because while a college degree is important for advancement in American society, this is not a country in which the particular alma mater determines the rest of one's life.

Good information on testing special education students

An informative and balanced article about the trials of testing disabled students, from the Chicago Tribune:

There is no love lost between many teachers and the push for more standardized tests. But rarely has a required assessment attracted such ire from educators, who say the IAA is too subjective to be an accurate measure of students' ability, takes up valuable teaching time and does nothing to improve student instruction.

Although doomsayers had feared that including special education students in the mix would drastically drive up the number of "failing" schools, that didn't happen in Illinois last year. Just 101 of the state's some 4,000 schools did not meet standards solely because of their special education students.

More time on the LSAT

Abby Rothberg, who sued for more time on the LSAT to accommodate her learning disability has won her case, thanks to a federal judge in Colorado:

The LSAC refused three times to give Rothberg more time despite the recommendations of two psychologists who tested her and determined that she is smart, but slowed by her difficulty in processing visual information and performing tasks based on that information.

[Rothberg] realized she had a learning disability when she was in the second grade and her classmates could read, but she couldn't. She has had tutoring, extra time to finish tests and other assistance throughout her school years in Colorado and at Syracuse University. When she took one test for college admission without seeking extra time, she scored below average. When she took another test with extra time, her score was above average.

This portrait in the Daily Orange is pretty sympathetic to the plaintiff:

Diagnosed with a learning disability, she has a handicap that impairs her to complete the exam in the amount of time her peers would need. It in no way, however, limits her ability to perform well on the exam should she be given appropriate accommodations...

Providing Rothberg with this time does not in any way provide her with an unfair advantage to perform on the test, nor does it guarantee she will be hired as a lawyer or succeed as one.

The LSAT is intended to predict first-year law school grades, so this accommodation also doesn't guarantee Rothberg will be able to finish law school, unless she receives this accommodation for every test she takes.

At some point, Rothberg will not be given the breaks she has been afforded thus far; but that can take place when she is in law school or in the courtroom. On a standardized test, which is itself suspect in its ability to test the taker's skills, Rothberg deserves the opportunity to perform her best. Until she is applying to law firms and must perform in a real world job setting, she deserves the rights that the law has created for her.

Gee, no bias there. The LSAT is "suspect" in its ability to test skills? Really. Would you care to back that up with some data? Would you care to explain how, if the LSAT is a bad test, we can assume that an extended-time LSAT - which is by definition a modification that may fundamentally alter the nature of what's being tested, and one that has not been validated - will somehow be a good test? Rothberg will have her accommodated LSAT, but there's little evidence to support the predictive validity of her score.

And if, as this author suggests, Rothberg will find herself at a disadvantage in job interviews and in the courtroom, why should she go to law school? Does it make sense to give a would-be lawyer accommodations for something she's going to be required to do on a daily basis - process lots of dense, heavy prose - when the real world won't provide that accommodation, thus leaving her fewer opportunities to practice law?

I blogged this in January; it looks like LSAC tried to argue alternative (a) that I theorized, but failed.

Update: Commenting on an another accommodated-testing post a few days back, Devoted Reader Wacky Hermit says what I'm thinking:

A blind person in the intellectual workforce can be just as productive as a non-blind person, given the right computer software. But a person who always needs time-and-a-half to accomplish every thinking task, who cannot be asked to speed up and cannot be assisted other than giving more time? Somehow I don't think that would make a person a productive employee. Someone like that would be better off being encouraged to go into a field that shows off his or her strengths, and doesn't rely on his or her weaknesses.

Being a slow reader is a weakness in the legal field.