Number 2 Pencil

When "adequate" could mean performing below grade level

Jim Sanders of the Sacramento Bee asks, "Is the bar set too high?"

Too many students fail to meet California's standard for proficiency, sparking a simple solution under consideration in the Capitol: redefine "proficient." By changing a few words in state law, legislators could dramatically affect how the federal government rates the state's education system.

"I think it's a totally sensible thing to do," said Assemblywoman Loni Hancock, D-Berkeley. Critics of Hancock's proposal, Assembly Bill 2975, say the state's goal should be to improve schools, not alter words. Hancock counters that both are needed to avoid severe sanctions in coming years under the federal No Child Left Behind Act, or NCLB.

"What all of this needs is for grown-up egos to be set aside and to focus on the young people," she said...

Under AB 2975, proficient students need not necessarily perform at grade level. Rather, test scores must show that they are acquiring adequate skills, year by year, to pass the state's high school exit exam by the end of 12th grade.

The concern here is that the US accepts each state's definition of "proficient," and requires each state's students to reach the state English and math standards of "proficient" by 2014. The bar, in California, is apparently set high, and currently fewer than half make it in each area.

Resetting the bar such that "proficient" is out of alignment with grade level seems rather surreal. If the argument is that it's wrong to judge California by their set standard, there should be data - other than the percent of students not making it - to support the argument that the standard is in fact too high. By twiddling with the numbers to make the percentages look better, the message would be that California in fact has no idea what their standard should be.

CAHSEE battle heats up

The battle over California's exit exam rages in the courts:

On Wednesday, 20 high school seniors and their parents sued the state Department of Education and school Superintendent Jack O'Connell, claiming the exam is illegal and discriminatory. They worry the test may prevent the students from graduating. "I don't think it should hold up your graduation,'' said Wasi, who is not part of the lawsuit but would be affected if it is successful.

The lawsuit was filed in San Francisco County Superior Court. It seeks a court injunction to delay the consequences of the exam for students in this year's class. Defendants also include the state of California and the state Board of Education. Lead attorney Arturo Gonzalez said the lawsuit likely will expand to represent tens of thousands of students who have met all local requirements to graduate except passing both sections of the exam.

Frankly, any student who receives passing grades in all their classes, is promoted all the way through 12th grade, and yet gets stumped by this test should sue the local Dept. of Ed - but not for the reasons the testing critics would think. Not only are the the California High School Exit Examination (or CAHSEE) items at the 10th grade level (math items don't go above Algebra I, for example), but the CA DOE website lists nine alternative paths for students who cannot pass the exam.

If an exit exam is going to be used, at some point, it has to count. This means a class of students will, by definition, be the first, and one could argue that the students this year haven't exactly been caught by surprise. Some enterprising reporters have even discovered that most students don't seem concerned about the exam, or go so far as to support it.

Nonethless, in addition to the lawsuit, we're seeing the usual arguments made that "multiple intelligences" are the issue when students make high grades but can't deal with multiple-choice items. It's tough enough for California to fight the battle for mandatory algebra in high school, but without the exit exam (or some equivalent high-stakes standardized test), there's no way to measure whether algebra is being taught effectively, or consistently.

If you were a parent, would you be satisfied if your teenager had an "A" in Algebra I but couldn't answer a question of the "If x=3 and y=4, then 2x+5y=" type? And would you feel comforted knowing of their "multiple intelligence?"

Feeling good about doing poorly

SeattlePI.com guest columnist Neal Starkman has found the answer to the exit exam controversy in Washington:

Passing the WASL is proposed to be a requirement for graduating from high school, but students would be able to retake the test -- let's see, at last count, 87 times. Plans are also under way to provide alternatives to the WASL, such as the Way Easier Assessment of Student Learning, the WEASL.

People in favor of the WASL say it's important to set standards or else graduation won't mean anything...Opponents of the WASL say the test is unfair to some students, that it puts too much pressure on them, and that students who fail the test all 88 times will feel bad about themselves...

I have a solution that will satisfy those who think the WASL is the greatest measure since the Stanford-Binet "IQ test" and those who have trouble spelling "IQ." It's a degree for students who fail the WASL. It's the A.G. degree -- Almost Graduated.

Here's how it works: I'm a student who doesn't take tests well, principally because I can't read or write. Or I can take tests well, but for some reason the WASL doesn't measure what I learned all throughout school...I take the WASL and I flunk it. Then I take it again and I flunk it. Then someone reads it to me and mouths the answers, and I flunk it. So what happens to me?

What happens to me is this: I walk alongside my fellow students in the graduation ceremony but instead of a diploma, I get my A.G. degree. It's coiled and beribboned like the other diplomas, but instead of saying, "Congratulations, you've graduated!," it says, "Well, you Almost Graduated. Good luck."

I don't get humiliated, I don't feel bad about myself, and, best of all, I can continue with my life almost as if I graduated. If a business wants to hire me after seeing I have an A.G. degree, well, they deserve me.

Takes care of the "self-esteem" part, I suppose. Be sure to read the comments, too.

Facing the finish line in Utah

The Utah high school class of 2006 is down to the wire:

This week, about 5,000 Utah high school seniors will have their last chance to pass the Utah Basic Skills Competency Test.

The class of 2006 is the first to face the consequences of the UBSCT, which was developed to ensure that students are competent in reading, writing and math when they graduate.

To qualify for a high school diploma, each student must successfully pass all sections of the UBSCT or attempt to pass the test three times. Each diploma will note whether the receiving student passed the test.

In other words, if you pass, or if you take the exam three times, you get a diploma. But your diploma will say on it whether you passed the exam. Interesting compromise:

Originally, students who did not complete all sections of the test were to receive an alternative completion diploma.

However, the board heard concerns that graduating with an alternative completion diploma could prevent students from receiving federal funding for college...In January, the board decided to give a full diploma to all students who pass the UBSCT or attempt to take the test three times. Students must also complete their school's citizenship and class requirements. However, every diploma must reflect a student's UBSCT status.

I should note that the exam is called the Utah Basic Skills Competency Exam, which makes it pretty clear that the test isn't supposed to be tough. Nonetheless, the anti-testing hyperbole ran thick last fall:

Dear Class of 2006: You're the first to be held to the state's new high school exit exam. And you look like lab rats. School districts are experimenting on you to learn how to best help students struggling on the Utah Basic Skills Competency Test. Answers will come too late to help you much...

No one really knows what your "basic diploma," "alternative completion diploma" or "certificate of completion" will mean in the real world. They're just waiting to see what happens to you next year...

UBSCT has three parts — reading, writing and math — and you have to pass them all to get a basic diploma. If you don't pass but tried three times, you can get an alternative diploma. Do neither and you'll probably get a certificate of completion. The exam is Utah's only high-stakes test. Your results will follow you for life. You have two more chances to take it: in October and again in February.

Schools hope you'll be prepared.

Funny, I would think it would be "graduating without the ability to read or write at a basic level" that would "follow a student for life." Please note the level of the reading and math items that are presented at the test's website. The schools should be doing much more than hoping students pass this test in large numbers, and it's just sad to hear about parents who are forced to hire tutors to help their child learn to read the instructions on a can of bug spray:

A Salt Lake County mother reports her daughter passed the exam on the fourth try — thanks to extra help never before offered at the neighborhood school...The Salt Lake County mother laments the lack of help available to her daughter until this year. Her family had been paying for private tutoring.

"I think if they want to have a test, they need to prepare the kids way before (their senior year)," the mother said. "She has been depressed this whole fall, and it's her senior year. I've just kind of seen her attitude has gone down, her grades have gone down, and it just makes the kids feel like a failure and that they're left behind. I don't think it's a good thing."

...but I disagree with the testing critics that it would be somehow better for students to receive diplomas (and good grades, apparently) without being asked to demonstrate at least this much knowledge.

California stands behind its exit exam

California superintendent of public instruction Jack O'Connell has held firm, and announced that the only route to a high school diploma is through the state's exit exam:

Seniors who do not pass the California High School Exit Exam this year should be allowed to continue their education, but diplomas will be awarded only to students who pass the test, Jack O'Connell, state superintendent of public instruction, announced at a Sacramento news conference Friday morning.

Within hours, lawyers who oppose the exam said they will sue the state in the coming weeks to try to lift the exit exam as a requirement for this year's graduating class. O'Connell's announcement followed a three-month review of possible alternatives to the controversial math and English exam that was adopted in 1999 and is a graduation requirement for the classes of 2006 and beyond.

To the disappointment of scholars and advocates who have urged the state to develop another path to graduation for students who fail the test, O'Connell said he believes no alternative exists that would show students have learned material tested on the exam.

"I'm convinced the only way to make sure all our graduates have the critical skills is through passage of the high school exit exam," O'Connell said.

Note that all of O'Connell's options for students who fail them exam involve additional schooling, some of which could be at no cost to the student - and yet the testing opponents are still riled. Seems to me like they aren't really interested in whether students get the opportunity to actually learn the material; they just want students who are in school for 12 years to get a diploma, whether they can pass a test - on sixth through tenth-grade material - or not.

Gonzalez said he will argue that the state cannot impose a single test on all students because it has not provided the same learning opportunities at all schools. Research shows that schools with large numbers of students failing the exam also have the most math and English teachers lacking expertise in those subjects. And he criticized O'Connell's options as being out of touch with reality.

"How many of these kids are going to want to go back for a fifth year of high school so they can take a course to pass the exit exam?" he said. "It's not going to happen."

Gonzales, how many of these kids will have a fair shot at life with a diploma that is meaningless, when they could have spent additional time mastering the skills they need? Talk about being out of touch with reality.

Test changes Down East

The Maine Educational Assessment, or MEA, might soon be replaced by the new SAT:

Commissioner of Education Susan A. Gendron has been promoting the idea of replacing the Maine Educational Assessment for juniors with the college-admissions exam. Ms. Gendron has said she thinks requiring students to take the test would communicate the message that any student can go to college and encourage more students to seek a postsecondary education...

Roughly 75 percent of high school juniors in the state already take the SAT. Maine students who took the test in 2004—the last time the old, two-section version was given—scored an average of 509 on the verbal section and a 505 on the mathematics portion, out of a possible 800 on each. The verbal score is a point higher than the national average, while the mathematics score is 15 points lower.

Interestingly, taking the SAT would be required - with the registration fee paid by the state - but there would be no minimum required score for graduation.

Struggling with English

Are high school exit exams "fair" for students with limited English skills?

Overall, states with exit exams are in dilemma -- they've been challenged to hold all children to the same standards, but in doing so, they may withhold diplomas from many kids with limited English. Almost all states with exit exams implicitly require students to know English to graduate, but high schools often find immigrant students are just getting started...

Many high school teachers are not trained to help students with minimal English, which means those children do not receive high quality instruction, said Deborah Short, director of language education and academic development at the Center for Applied Linguistics.

"Do we want a lot of high school students who don't have diplomas -- and therefore have a lot of limited opportunities after high school -- because they are still acquiring English?" she said. "We need more of a policy on what to do with these children."

Actually, the question should be, would we rather have a lot of students with no diplomas because of poor English skills, or would we rather have lots of employers and admissions officers discover that the diploma no longer guarantees its bearer can speak English?

Graduation exams disproportionately affect limited-English students: 87 percent of them will have to pass a test to graduate in coming years, compared to about 72 percent of all U.S. public school students. Most students learning English as a second language live in gateway states for immigrants that have exit exams, mainly California, Texas and New York.

Something tells me that isn't a coincidence. Isn't it possible that these states were quickest to implement the exams because they're trying to prevent turning out high school graduates who are struggling with English?

I agree this is a problem, and if the numbers of limited-English students continue to increase in public high schools, educators are going to be forced to make some tough choices. I just find it worrisome that the debate seems to center on whether the tests are fair to students, rather than whether the awarding of diplomas to students who can't speak English is fair to society.

The plight of the good C- student

Board members in Enfield, CT, are depressed about the idea of exit exams:

Members of the Board of Education don't want to base high school graduation requirements on test scores, but they have no choice in the matter. State law mandates that school systems craft graduation standards beginning in 2006 based in part on the results of the Connecticut Academic Performance Test, a test given annually to high school sophomores. In response, the school board Tuesday discussed revisions to the current graduation policy that would make high school graduation contingent on meeting testing requirements...

Currently, a student must earn a minimum of 22 credits and maintain at least a C-minus average to graduate. Roger Jones, school board member, expressed dismay about changing the policy.

"It's just not right," Jones said. "You've got a poor kid who works hard, a good C-student, and they're just not good test takers, and it winds up they can't get a diploma."

Does the CAPT measure what Jones' think high-school graduates should know? Perhaps not. But if it does, then any student who doesn't pass the exam shouldn't get a diploma. Otherwise, students will receive CT high school diplomas based on effort, not achievement.

The draft policy, which received a first reading Tuesday, includes several opportunities for students to meet the testing requirements. According to the draft policy, students would receive diplomas if they demonstrate proficiency in math and in either reading or writing by scoring a 3 or higher on the CAPT -- or scoring at least 430 on the math and verbal sections of the SAT.

They also could meet the graduation standards by scoring an 18 or higher on the reading and math sections of the ACT, a standardized test similar to the SAT. Students who are unable to achieve the minimum scores on the CAPT or SAT will have the opportunity to take proficiency tests developed by the school system in math and reading.

In other words, the district will bend over backwards to help any kid who struggles with the CAPT. But please note that minimum SAT scores given above identify students who are in the 23rd and 21st percentiles, respectively - and if they can't manage that, they get to take yet another test designed by the school system.

If the district has any faith in tests at all, at some point, they'll have to admit that grade inflation can produce students with C- averages who really don't know enough to earn a high-school diploma. If they're not willing to deny any of those "good C-" students a diploma, then they shouldn't bother with the tests at all.

By the way, we're not talking about large numbers of kids who can't pass the testing gauntlet:

Assistant Superintendent Anthony Torre said the school system has been tracking testing performances for incoming seniors.

He said parents of students who have not yet met the requirements have been notified. Torre said only nine students at Fermi High School have yet to achieve a required score on one of the testing options. He did not have figures for Enfield High School.

Taking their time

A balanced article about exit exams in Alaska contains this odd quote:

This is the second year the exit exam has tripped some seniors' walk down the graduation aisle. Students in Alaska started taking the exam six years ago. But passing all three portions has only been a requirement for graduation for two years. Students first take the exam in the spring of their sophomore year. There are no time limits, and they may re-take it as many times as necessary in order to pass. It is administered twice a year.

In April, some schools reported students taking more than 10 hours to complete the exit exam. Palmer High School principal Wolfgang Winter said at the time that he saw the lengthy test times as a measure of how seriously students were taking them.

Emphases mine. Are you kidding me? Does Winter seriously think that it's a good sign that some students took more than 10 hours to complete an exit exam? I've been all over the web and can't find the item count, but it's hard to imagine that the math and readings sections contain more than 100 items each, tops, or that the writing section contains more than a handful of prompts. Those marathons sounds more like a cry for help to me.

On a related topic, Alaska's DOE website has a nifty list of the exit exams across all the states that use them.

Closing the loopholes in NJ

New Jersey tightens the standards:

New Jersey students who do not pass a standardized test would have a tougher time getting high school diplomas, but would have more chances to take the exam under a plan unveiled Wednesday by the state Education Department.

Education Commissioner William L. Librera is a longtime critic of the process known as the Special Review Assessment, which allows students to graduate without passing the High School Proficiency Assessment. Now, he wants to phase out that path to graduation. "We think the SRA hurts the very kids it's designed to help," Librera said on a conference call with reporters Wednesday. "It erodes the meaning and integrity of the high school diploma."

The special process was introduced in the 1980s, mostly for special education students. But it's become more widely used. Nearly 20 percent of New Jersey high school students follow the alternate path to graduation _ including about half in 31 poor, mostly urban school districts that get extra attention and money from the state.

"More than half." That's quite a loophole. That's a loophole that needs to be closed. The spring '04 HSPA results are summarized here. For a point of reference, the overview of the language arts literacy segments are here, and some sample math items are given here.

Sample OE Item

Every Tuesday, at the Dog Deli, the manager gives away free hot dogs and soda. Every sixth customer gets a free soda, and every eighth customer gets a free hot dog. The Dog Deli served 73 customers last Tuesday.

How many free sodas did the Dog Deli give away last Tuesday?
How many hot dogs were given away?
Did any customers receive both a free hot dog and a free soda?
If so, how many customers?
If a soda sells for 99¢ and a hot dog sells for $1.99, how much did the Dog Deli lose in income by giving away these items?

Justify your answers.

Does anyone believe that one in five NJ students can't be expected to pass an exam of this type before graduation, and thus deserve a loophole so big that, as the state itself admits, "It is nearly impossible...to monitor the conditions in which the SRA is administered"? Justify your answer.

FCAT 2005 results are in

After 18% of the seniors in Florida fail the FCAT graduation test, the wits at Fark have renamed it the "Florida Universal Comprehension Test." A large portion of the failers don't seem to be those "high-grades-low-test-score" sob stories that we hear so much about, though:

Figures from the state Education Department show 27,000 of 150,000 seniors have failed to pass at least one of the required exams. About 60 percent of them also lack credits or have grade point averages that are too low for graduation.

Students can retake the FCAT test indefinitely until they pass to earn a diploma. The state also offers remediation and fast-track GED programs.

That leaves about 10,800 students who presumably have the appropriate credits and a high enough GPA, yet still failed at least one of the required exams. This article has slightly different numbers:

At least 15,100 seniors in Florida, including 1,800 in Broward, failed the math or reading FCAT, according to figures released Wednesday by the state Department of Education. Many of the students failed even though the state eliminated the essay part of the reading test, as well as adjusting the reading and math tests to be entirely multiple-choice.

Must be those eeeevil MCQ's. I went to the FDOE website and found the press release, which says that the percent of students not graduating solely due to the FCAT is 7%, down from 9% last year. The percent passing by school district is given here, although these numbers don't agree at all with the 150,000 number given above, so I'm a tad confused. If anyone has a definitive source on the number of those taking this exam and passing, let me know.

Now, how about those third-graders?

More Florida third-graders passed the state's reading test this year, so fewer will face the wrenching news that they could be kept out of fourth grade, according to test results the state released Wednesday. Reading scores improved for the fourth year in a row for third-graders, who by law can be held back if they fail. Sixty-seven percent now read at or above grade level, the highest percentage ever, compared to 57 percent in 2001...

Only 20 percent of the state's third-graders failed the reading section of the Florida Comprehensive Assessment Test this year, down from 22 percent last year and 29 percent four years ago.

Although more than 40,500 students now face being held back, in the past nearly half of the third-graders who failed FCAT moved to fourth grade anyway. They either passed another state test after a summer "reading camp," showed class work that proved they can read or met some of the state's limited exemptions to the retention law.

The important rules

Mark Goldblatt of NRO proposes an English exit exam:

1) Define the terms "independent clause" and "dependent clause."

2) Find the subject in the following sentence: "Many of my friends drive to school."

3) What are the three principal parts of the verb "to bite"?

4) "Jane has been dating John for two years." Is that sentence written in a present tense or a past tense?

5) "Jane has been dating John for two years." Change that sentence to the corresponding past tense.

6) What three parts of speech can an adverb modify?

7) What is the main use of a semi-colon?

8) "Jane invited John and me." "Jane invited John and I." Which is correct?

9) "He should of told me that I wasn't invited." What's the error in that sentence?

10) "Every person is entitled to their own opinion." What's the error in that sentence?

His amusing take on why such an exam is necessary:

Each question is worth ten points. If you scored below 70, you failed. More to the point, your teachers failed. They've failed you, miserably, for twelve years. Those hundreds of hours spent in classrooms with posters of William Shakespeare and Alice Walker on the walls, those hundreds of hours spent as your teachers prattled on about the joys of creative writing — those hours are worthless, utterly worthless, and you can't have them back. Those A's you received for free-verse poems, those stories you wrote to explore your feelings, those papers returned to you without a single grammatical correction — they're worthless too. You didn't learn what you should have learned, what you needed to learn.

Expect snitty rebuttals from touchy-feely educators to appear soon.

The mysteries of standard setting

The standardized testing version of 12 Angry Men is 72 Standard Setters:

About six dozen people whom you have never met sequestered themselves in a suburban motel late last month and decided whether your son and daughter will graduate from high school. Like a jury choosing between guilt and innocence, members of the panel weighed evidence, deliberated in private, agreed on a conclusion and then parted ways.

But instead of determining responsibility for a particular piece of mayhem or murder, these citizens gave their time to decide the grading scale for the new Ohio Graduation Test, which students will have to pass before leaving high school to get a diploma...

The public usually learns of pass-fail scores for state tests - or "cut scores," in education lingo - when those scores first come before the State Board of Education. That will happen next Tuesday, when the committee's recommendations are considered by the 19-member state board. Whatever pass-fail scale the board eventually adopts will be used to score the test that the state's 10th-graders took in March.

But most Ohioans - including most teachers and administrators - have little idea of the process that determines the thin line between passing and failing.

But I'm sure they realize that, as Arizona State University researcher Gene Glass says, once the committee derives the criterion-referenced cutscore, the state gets to "fiddle" with it to get a standard that is politically popular (and yes, this does make it more of a norm-referenced cutscore at that point).

An exit exam puzzle

Retired USC professor Lorin Anderson wonders about the difficulty of SC's new high school exit exam:

A retired University of South Carolina education professor who specializes in education trends questions whether the minimum score on the exit exam is artificially low. Lorin Anderson points to a 76 percent pass rate in 2004, while the highest percentage of students who passed a now-retired test on the first try was 70.6 percent in 1991. During the 17-year span the old exam was administered, the average number of sophomores who earned a passing score on their first attempt usually fell in a mid-60 percentage range.

“I have reason to doubt the validity of the apparent increase in the passing rate from the old exit exam to the new test,” Anderson said. “There’s something wrong here. It makes no sense"...

Anderson...calls the aversion to disclose the minimum level of performance required to pass the state’s exit exam “the dirty little secret of the testing business.” “I think a fair question to ask is: ‘What percentage of questions do you have to get correct to pass PACT or HSAP?’” Anderson said.

I can understand the state's reluctance to supply this answer, because (a) many testing companies don't disclose cutscores, and (b) it's quite likely that the test isn't scored with a simple number-right total. If item difficulty is also a factor, then an examinee with three hard items correct could end up with a higher score than someone with four easy items correct, and that's very hard to explain to laypeople. On the other hand, it's hard to defend oneself against the accusation that the test has been dumbed down if one isn't willing to release some information about how well someone has to perform to pass.

Hoosier highest-scoring school?

Indiana reports the results of their high school exit exam:

Nearly one-third of 10th-grade students statewide failed on their first attempt to meet the academic standards required to graduate. Test results released by the Indiana Department of Education on Thursday showed 68 percent of 10th-graders passed the English portion of the Graduation Qualifying Exam and 64 percent passed the math portion. State officials consider this a baseline year for the exam because changes were made to more closely align it with the state academic standards.

You know, one could see this as good news. If the exam is meant to measure academic skills above the 10th-grade level, then we wouldn't expect every 10th-grader to pass it, would we?

Regardless of a student’s background, however, the student must pass the GQE before graduating. If a student fails the test the first time, it can be retaken four more times before graduation..

The most concerned administrators are those who worry five testing attempts won't be enough:

Brad Bakle, EACS elementary curriculum and assessment director, said he is trying to figure out what barriers are troubling Paul Harding High School. Scores on the GQE dipped this year after going up last year. “It’s not good,” Bakle said as he looked at the scores. Only 20 percent of 10th-graders passed the math portion and 35 percent passed English.

The Indiana State Board of Education recently approved new high school graduation requirements. ISTEP+ scores for 9th- and 10th-graders are out as well.

The insider's word on education in California

Daniel Weintraub - the Sacramento Bee's "California Insider" and all-around cutie-pie - mentions a note from a local education professor:

In response to this column Sunday defending the state's high school exit exam, I got an interesting note from an education professor at one of our state universities. She opposes the exam, she said, because only 19 percent of students who were not fluent in English passed the test in 2004. Further, she said, only about 60 percent of students who begin school in California after kindergarten and don't speak English as their native language will be fluent by the 12th grade. "In other words," she writes, "if a student is a second language speaker of English, his or her chances of failing the HSEE are somewhere between 40%-80%. This is not fair."

Is this a matter of fairness, or simple fact? Perhaps the situation the professor describes is a reflection of a simple and deliberate policy: if you can't speak English by the 12th grade, you don't get a diploma in California schools...The fact that it's controversial to require students to speak, read and write English before graduating from high school shows just how dysfunctional our education system has become.

Well said. I shudder to think that a professor of education believes the only argument one needs muster against a test is, "It's not fair." Especially when what's allegedly "not fair" is a test requiring a mastery of English for graduation from an American high school.

Daniel's good at pointed, concise conclusions, by the way:

The exam's opponents fear the stigma that will be attached to any student who, failing the test, leaves school without a diploma. Maybe they should worry as much about the prospects for students who for far too long have been leaving high schools with a diploma but without the basic math and English skills they need to survive in society.

Magical thinking in California

In California, as in Arizona, there are those legislators who don't really want exit exams to be, you know, exit exams (registration required):

With the moment of truth fast approaching for California students, a high-powered drive has begun in the Legislature to delay or eliminate tying high school graduation to passing a controversial exit examination. Beginning with the class of 2006, state law requires high schools to deny diplomas to any student who doesn't pass a mathematics and English test, a consequence that was delayed two years ago to give schools more time to prepare.

Three guesses as to what political party the new procrastinators claim, and the first two don't count.

Two new bills, proposed by Democrats, take separate approaches to the issue. But both question the fairness of the high school exit examination and neither would allow imposition of high-stakes consequences next year. Perata and Goldberg, D-Los Angeles, argue that some campuses have been shortchanging children for years, providing inadequate instruction or textbooks. To deny diplomas to students at deficient schools would essentially victimize them twice, they contend.

This kind of thinking fascinates me. How is giving someone a worthless diploma doing them any good whatsoever? Why are these lawmakers convinced that if kids have been shortchanged for years, making sure they receive this piece of paper - without demonstrating the skills that lay behind it - will somehow rectify the situation?

It's a piece of paper, not a magic wand. The exit exam, though, certainly has some magic in it, because the exam allows California to see just how badly their schools are failing the students. If the lawmakers mentioned here care so much about drawing attention to how bad the schools are, you'd think they'd support this exam. Without it, how is anyone really supposed to know how bad things are?

How low can you go?

Devoted Reader and prolific busybody Reginleif sent along a link to a Boston Globe article which describes the horror felt by state Education Commissioner David P. Driscoll when he discovered that - gasp! - raising the MCAS exit exam standard might mean that fewer students will graduate:

Despite pressures from business leaders to set higher standards, state Education Commissioner David P. Driscoll expressed reluctance yesterday to raise the MCAS score that high school students need to graduate, saying that such a move could result in thousands of special-education students failing. Currently, students need only to attain a score of ''needs improvement" to pass the test, but some state education board members and corporate leaders want only those who score ''proficient" to be able to graduate.

But Driscoll said the current passing score is reasonable because ''it's attainable and achievable by a lot of kids." He did not address the possibility that if the passing score were raised, many more minority students, particularly African Americans and Latinos, would also fail to graduate.

Got that? What kids know doesn't matter. What businesses want in their employers doesn't matter. What matters is that Massachusetts sets the standard so that most of their students pass it. Many people believe this exam should be an indicator that students are proficient in high school level skills. Driscoll apparently sees it as a minimum-competency exam, or an indicator that a kid is not entirely uninformed.

Many students who pass the English and math of the Massachusetts Comprehensive Assessment System still require remedial courses in those subjects once they're in college...The current standard requires students only to perform at the eighth-grade level.

Driscoll said he agrees that students need to be better prepared, but said he hopes that can be achieved through encouragement and prodding by teachers, parents, and community members, rather than a state-mandated change in the passing score.

I think teachers have had it with the prodding. What's more, why should they encourage their students to aim higher than the bar that the state has deemed sufficient?

Hoosier standard going to impact?

Indiana has a mandatory exit exam, and a panel recently voted to up the standard for passing - apparently for the sole purpose of keeping the percentage of students passing the same from year to year:

An advisory panel voted Tuesday to recommend that the state raise the score required to pass Indiana's mandatory graduation exam...Officials expect about 32 percent to fail the English portion of the test. About 36 percent are expected to fail the math portion, which included algebra for the first time last fall.

The state agency adjusted the pass-fail scores recommended by a panel of teachers to achieve those percentages, which are about the same as last year's failure rate, said Wes Bruce, who heads student assessment at the Indiana Department of Education.

You don't see many attempts at norm-referencing in exit exams, for the simple reason that an exit exam should represent material mastered, not location on the curve. There's no reason why 100% of students can't pass a high school exit exam, and no reason why 32% failing should be a number that Indiana hopes to see year-to-year. I think this is an attempt at equating the exam across years, but it's not a useful attempt.

The percentage of students who pass or fail is a more important number than the score, an expert said. "With all the games we play, we're just deciding the percentage of kids that are going to fail," said Lowell Rose, a former Kokomo school superintendent and consultant with the Indiana Urban Schools Association.

I have no idea what that quote is supposed to tell us. Yes, percent failing is important, but there's no reason to tweak standards to keep that constant year-to-year.

Raising the passing score on the GQE is the latest move to make the earning of a high school diploma more demanding. The panel in October unanimously approved a plan that by 2011 would make college aid and admissions contingent on students earning a Core 40 diploma — a much more stringent academic path.

That, I have no quarrel with.

Education officials say such measures are necessary to ensure that Hoosiers have the skills needed to get into college and get good jobs. Critics say the tougher standards will make it impossible for some students to graduate regardless of their college plans and doom their chances of making a decent living.

Do the critics stop to think that students who don't master high-school-level material (and the stuff on exit exams tends to be VERY easy) won't have many college plans, or much hope for a decent income? That it's not the test that will hold students back, but their educational deficits?

That much said, the test should only be made more difficult if the curriculum is well-aligned to it. Increasing the difficulty of test items while not focusing on the teaching of those items in school would miss the point entirely, which is to intensify the curriculum, not the items.

The GQE is the cap of a series of annual standardized tests Indiana students must take beginning in third grade. Sophomores who do not pass both sections on the first try are given four more chances to pass before they finish high school. Students also can apply for waivers to graduate despite failing the exam, and people who do not pass the test can retake it after they leave high school, though they might have to take remedial classes at their own expense or take free televised courses.

Remember what I said about how easy exit exams are?

This year's graduating class can still be illiterate, though

By 2009, high school students in Milford, Connecticut will be able to graduate only after passing the very high hurdle of...knowing how to read:

The Board of Education signed off Tuesday on a much-heralded plan to make reading the next graduation requirement for the city's high school students, designating the Class of 2009 the first to be subject to the new rules...

"The board understands that reading is essential to lifelong learning and is in conformance with our performance promises," Politi said.

Reading? Essential to learning? Why, you could have knocked me over with a feather.

The only official of the 10-member board who declined to vote on the proposal was Ronald Funaro, D-2. Attempts Wednesday to contact Funaro were unsuccessful. Last month, Funaro was one of at least two board members who questioned why the district was focusing on high school reading when such learning problems existed in the middle schools, too.

"We are talking about teaching reading in high school. When did we miss it in elementary school? When did we miss it in middle school?" Funaro asked at the Dec. 14 meeting.

Thank God. Someone with sense. If the district doesn't focus on reading until students are past puberty, there's a problem. And why would such a focus be"much-heralded" unless the district is pretty much admitting that they haven't much paid attention to such things in the past...which is what they do admit:

David Larson, executive director of the Connecticut Association of Public Schools Superintendents, said more districts have increased reading standards since the creation a decade ago of the 10th-grade reading exam on the Connecticut Academic Performance Test.

He said mastering such skills can only help students succeed in the real world.

Does that mean we can all agree that schools that once focused, or continued to focus, on things other than the 3R's are not living in the real world?

Quibbling over the MCAS

A Boston English teacher handles letters about the MCAS in the Globe's "Ask the Teacher" feature:

Q. These days, there is much pressure put on our adolescents to pass the MCAS test. In addition, school personnel often need to address youth's other needs, such as social and emotional growth, career exploration, etc. Is the MCAS exam helping or hindering our youth from achieving and being well-rounded?

A. The MCAS has had its fans and foes well before students began taking the statewide test in 1998. Proponents applaud the exam as a means of creating and measuring statewide educational standards, particularly in math and English. Critics, however, take issue with its too-high stakes (a high school graduation requirement), an apparent preference for memorization over knowledge, and a dismantling effect on teaching (faculty teaching to the test).

"Too-high stakes" - meaning Massachusetts students shouldn't be expected to master 10th-grade material in 12 years. And explain to me again exactly how you don't "know" something once you've "memorized" it, and how you can "know" a concept about which you've "memorized" no facts?

Some school districts have kept their focus on educating the whole person...

Translation: Rock-bottom test scores are okay as long as the student learns to play well in groups of properly-distributed diversity...

...while others have reshuffled course content and academic priorities to prepare students for MCAS exams. Karen Harris, who taught English at Watertown High School for 12 years, then became a teacher at Brookline High, has seen the MCAS effect from two distinct perspectives.

"At Watertown High, the MCAS became an obsession for teachers as well as students," said Harris. "Early on, many students failed the exam. After that, every faculty meeting addressed the issue of how to prepare students to pass this one exam . . . What important issues weren't we discussing as a result of this obsession?

What would Harris consider more important than the fact that Watertown students were in class eight hours a day, yet weren't mastering basic reading and math skills? I for one am delighted that the MCAS scores were the focus of every faculty meeting.

"In Brookline, where there's less anxiety about students passing the exam, you have the opportunity to turn the test into a lesson on conscientious objection, for example. I realize that's not a luxury every school system has.

No, it's most definitely not. Could you be a little more condescending about the fact that smart kids at the better schools get to be "conscientious objectors," while the kids at the poorer schools would be better off if they had teachers dedicated to helping them learn to spell those words?

"If teachers are only teaching the test, the students are receiving a very limited education. Plus, with teachers forced to wear more and more hats these days, it's increasingly difficult for many to help create the sort of capable and curious student we all want to see."

I agree that teachers are spread too thin. I also think bad teachers don't know how to teach basic skills without narrowing curriculum. But I fail to see how any student could be termed "capable" if they don't learn these skills.

Students often fare well on standardized tests when their teachers know their subject and show a passion for it. More importantly, their students appreciate that education's worth can't be quantified.

That last statement is true only in the sense that "Money can't buy happiness" is. We all know that a great education can't be wholly captured by a test, and high test scores do not by definition mean a great education. But just as money can buy everything except happiness (and make misery a whole lot more tolerable), the better the test scores, the better off the students are.

High test scores may not mean everything's going all right, but low test scores always mean that something has gone wrong. And there's not a student on earth who has ever been "hindered" by their high test scores.

The truth about exit exams

Here's a devastating indictment of the nation's high school exit exams, as featured in Newsday. I don't think anything in this report, though, will be surprising to some of my more involved Devoted Readers:

From Newsday:

Many high school graduation tests don't measure whether students are ready for college or work, and some states haven't even made clear what the purpose of their test is, a study finds. Of the 25 states that have or plan graduation exams, only one, Georgia, says its test ensures students are prepared for higher education or work. Most of the states gear their tests toward 10th or 11th grade learning, and some gauge pre-9th grade skills, according to a study released Wednesday by the Center on Education Policy, a nonprofit research group.

The center takes no position on the tests, aiming instead to highlight what's working and what's not as state leaders weigh decisions. For its annual report, the center collected data from the states, reviewed research and convened a national panel on the tests' impacts.

I think Newsday's analysis is pertinent, although I disagree with their claim that the debates about exit exams are "quieting somewhat." As they note in the article, the total number of students who failed to earn diplomas because of test scores is hard to track in part because of the number of appeals (and the number of lawsuits, too.)

What's more, we're seeing more and more accommodations, such as those in Alaska that ultimately don't require a student to be able to read before being granted a diploma. Anyone who thinks that allowing students to have test questions read to them, or who are allowed use of a dictionary during an English, is a simple accommodation that doesn't change the nature of what's being tested is fooling themselves. The exit exams are easy enough to begin with, and these accommodations make it possible for virtually anyone to pass.

The report itself is a treasure trove of information, and it's well worth your time. One conclusion startled me:

While several states report conducting studies of the alignment of their exit exams to their standards, fewer report doing studies of the alignment of curriculum and instruction to their exit exam, and almost none report conducting studies on the impacts of their exit exam systems. We recommend that states undertake or encourage others to do studies of alignment between exit exams and curriculum and instruction and studies of the effects of their own exams.

Emphases mine. If schools are doing nothing to ensure that kids are taught what they'll be tested on, and whether the exit exams are holding back only those who truly need more time, no wonder the students and teachers don't like them. Memo to administators: This isn't a touchy-feely project in which you can convince us that there's nothing to measure. The studies are there waiting to be done, and if a diploma hinges on the results, you need to do them. Get over your anti-science bias and collect some data so you can validate the exams.

Fighting the exit exams in Virginia

I meant to post this article about the appalling ease of exit exams yesterday, but it didn't get published until today. And, right on the heels of research suggesting that exit exams are far too easy for today's high schoolers comes this discussion of Virginia's SOL, in which testing opponents insist that the test "makes" students drop out:

Of the more than 98,000 students who began their freshman year in Virginia's high school Class of 2004 -- the first required to pass state Standards of Learning exams to graduate -- fewer than 70,000 are expected to receive their diplomas this month. A child-advocacy group charged yesterday that the exams are partly to blame for the gap.

JustChildren, an arm of the Charlottesville-based Legal Aid Justice Center, held a news conference in Richmond to ask state educators to investigate graduation rates and their connection to the SOL exams. "We support high standards, but we do not support punishing children," said Debra Grant, a Virginia Beach mother of three who spoke at the conference.

Oh, I see. JustChildren supports high standards, but doesn't support actually holding any of these "children" to those standards. If flunking students who don't pass the test is "punishing children," in what way can JustChildren be considered to "support" the standards at all?

Andy Block, legal director for JustChildren, said he suspects that the pressure of the tests discourages students and prompts them to drop out. "Given the significant percentage of students who end up without diplomas, we hope that figuring out why this is going on becomes an urgent priority," he said.

I've yet to see one person making this argument deal with the issue of grades and class exams. Somehow, exams and homework pressures for six classes a semester is acceptable, and yet it's this one standardized test, which probably only lasts a couple of hours, that will force a student to choose a GED over a diploma? Does that make sense to anyone?

And of course there's the issue of how exactly to measure dropouts. Some voices of reason point out that the existing data do not support the claims of organizations like JustChildren:

State education officials said a study has been commissioned, but they said that until individual students can be tracked more accurately, it is difficult to know whether students drop out, move out of state or transfer to private schools.

By that measure, Abrams's study found that Virginia ranked better than 36 states in recent years.

(Thanks to Devoted Reader Jeff B. for the link.)

Exit exams not much of a challenge

I'm sure this is meant to be reassuring, but I find it disturbing:

Maryland high school students may not have too much trouble with standardized tests required for graduation starting with the class of 2009. A new study from a non-profit group shows that the tests are not overwhelming.

It says the material covered on the math exams is the equivalent to what students learn in seventh or eighth grade. The english exams are like another standardized test usually given to eighth and ninth graders.

Uh-huh. Remind me again what the point of this exit exam is? If it's supposed to signify that the student has mastered high-school-level material, it isn't. And think about what this means for those students who still flunk it. (Commenters on Joanne Jacobs' site certainly have thought about this.)

The study mentioned here was conducted by Achieve, Inc.; the entire study is available from their website. Maryland wasn't the only state involved; exit exams from Florida, Massachusetts, New Jersey, Ohio and Texas were examined as well. The conclusion sums it up well:

Achieve reached three conclusions: First, it is perfectly reasonable to expect high school graduates to pass these tests — they are not overly demanding. Second, these exams will need to be strengthened over time to better measure the knowledge and skills high school graduates need to succeed in the real world. Third, states should not rely exclusively on these tests to measure everything that matters in a young person's education. Over time, states will need to develop a more comprehensive set of measures beyond on-demand graduation tests.

Those comprehensive sets of measures used to be called "grades;" apparently those are too unreliable nowadays. And if these test have indeed been dumbed down this far, I agree that states shouldn't rely exclusively on them.

The NYT has more:

The study found that the tests measured very basic material and skills, insufficient for success in university courses or in jobs paying salaries higher than the poverty level, currently about $18,000 for a family of four...

Matthew Gandal, the executive vice president of [Achieve], noted that exit exams were frequently attacked as unfair. "We think it's the opposite," he said. "It's unfair not to expect students to learn what's on these tests. By the time they graduate, if they haven't learned what's on these tests, they'll be really unprepared, and by then it's too late. They won't be able to go to college or to get jobs with which they can support a family."

Amen. How nice to see someone cut through the bull about how the exit exams are the stumbling blocks. They're not. The tests aren't the barriers to achievement, but the skills - or lack thereof - behind the test scores are. Too bad the testing critics in the article, who huff about "one-size-fits-all" exams, don't get this. Apparently, to them it's perfectly fine to expect seniors to perform at an eighth-grade level, or below.

New Jersey is one state that aims to toughen their exams:

New Jersey's High School Proficiency Assessment was among six state tests reviewed by Achieve Inc...The study found the HSPA was among the strongest of the lot...

In all, the study found much of the knowledge and skills in the reviewed tests should be covered by the time a student reaches the end of middle school or ninth grade. For instance, most focused largely on basic math skills that didn't exceed a year of algebra or, in some cases, pre-algebra. New Jersey's was one of the few tests that asked students for any writing sample...

Last year, more than 15 percent of graduates statewide went through the Special Review Assessment, and officials say the number could top 20 percent this year. In some urban districts, the SRA share is as high as 50 percent, and advocates of the alternative exam say it is needed because of what they call the rigor of the HSPA.

Emphasis mine. In some districts, 50% of students are slipping through a loophole that fails no one. And yet critics insist, in the face of evidence suggesting otherwise, that these tests are so demanding that hordes of students drop out in response.

My take on this? First, my assumptions are that exams that are this dumbed down help no one, loopholes that give anyone a diploma help no one, and no test in and of itself will improve the educational process. Also, it's a fact that testing critics will continue to cry "Unfair!" about any test, as evidenced by their yammering about eighth-grade standards for twelfth-grade students.

Given all this, following Achieve's recommendations and New Jersey's lead seems like the best plan - toughen up the curriculum, toughen up the exams, add additional assessments, and say "Too bad" to anyone who just doesn't cut it. We've reached a point where schools are afraid to deny any student a diploma, no matter how poorly that student performs. Unless schools are willing to stand up to the critics, who will raise Cain no matter how low the standards are, exit exams aren't going to be of much use in improving high school education.

HSA critic responds to N2P

Elliot Wolf, the senior quoted in the WaPo article about the Maryland High School Assessments, has requested via email that I post a link to his entire testimony about the HSAs. The WaPo article gave little context to indicate that Elliot was well-informed about the tests, but his testimony, along with articles listed on his website, indicates that he has definitely done some research when it comes to standardized tests.

I thought this part of Wolf's testimony was particular interesting:

Blair is in a unique position, one that demonstrates the effects of the HSA on a wide range of students. On one end, we house the Math, Science and Computer Science Magnet Program – one of the highest regarded public secondary school programs in the country. On the other end, approximately one third of all students at Blair are recent immigrants to the United States, and cannot read, write or speak English fluently [Note: While Wolf's testimony puts this number at approximately 33%, the WaPo article claimed that a much lower percentage of Blair students - 10% - had limited English proficiency].

Our research has shown that the implementation of these tests has had negative effects on students at all points on this continuum. In the Magnet, we have new freshmen coming out of HSA-aligned Algebra I classes who are severely lacking in even the most basic Algebra skills. They are able to pass the HSA with flying colors, yet many are not prepared for the advanced math classes that their predecessors who completed non-HSAaligned algebra courses were.

Thus, his complaint seems to be that students who are taught Algebra under HSA guidelines in fact learn less about Algebra than those who took non-HSA-aligned courses. As I've said many a time, there's nothing wrong with teaching to the test - if it's a good test. But if Wolf's claim is true, the HSA may not be a very good test at all.

Much ado in Maryland

Maryland is nearing the finishing line in setting a high school exit exam requirement in stone, and the local teachers' union has tossed its hat into the ring in opposition:

The Maryland Teachers' Union is joining other education groups in a last-ditch effort to oppose the state's plan to make passing standardized tests a requirement for graduation.

The union president said there's much more to learning that can't be assessed by a single test, 11 News reported.

Oh, yes, so much more. Because, as we all know, exit exams tend to assess nuclear physics and organic chemistry, rather than (usually 10th-grade-level) basic reading and math skills.

Looking further, though, I'm wondering why they're bothering with this last-ditch effort. The Maryland BOE voted last year to implement the exit exams:

Maryland's Board of Education has approved a plan to require students to pass the state's High School Assessments in order to receive a diploma. The requirement begins with the class of 2009, making Maryland the 19th state to adopt an exit exam...

Some board members who voted in favor of the graduation exam said they were doing so reluctantly. "We've never generated the reality of what will happen when we do this," said JoAnn Bell, the board's vice president. "We are going to lose kids."

Work with me here, Ms. Bell. You're going to "lose" kids who spend four years in high school without mastering basic skills in algebra, English, government and biology. These are kids currently in seventh grade, so it's not like they're not forewarned. All the school can do is teach the classes well, and give the tests. Some kid will fail them. This doesn't mean the tests will have blocked them; it will mean they never learned the material, and thus won't have some necessary skills for success later on.

The Washington Post has more on the "last-ditch" efforts:

Critics worry that schools might place too much importance on the tests and that students who think they cannot pass them might drop out. For the past two years, high school students have been required to take the Maryland High School Assessments, but results have had no effect on their graduation status.

Do these same critics also constantly fret that the existence of grades for high school classes are too "important" and might make kids drop out? We're talking about the same thing here. High school students already have to write papers and cram for tests. Sure, there's grade inflation to prop up the lazy, but to hear these critics complain, an exit exam is the only high-stakes academic situation students will encounter in their four year.

Preliminary results of a study on exit exams in six states, to be released next month, show that "there's nothing in those tests that it would be unreasonable for a high school graduate to know," said Michael Cohen, president of Achieve Inc., a nonprofit group that helps states improve and coordinate testing efforts. Cohen outlined Achieve's study findings at yesterday's Maryland school board meeting.

The maximum score on each exam is 800, and students must receive a combined score of 1613 on all four to get a diploma. However, there is a catch: Students will not receive a diploma if they score lower than a minimum target -- yet to be set by the board -- on any of the tests.

So, they can retake many times. They can pass separate exams on separate takes. And they have to score, on average, a little over 400, or 50%, on each exam. To the union member who said, "there's much more to learning that can't be assessed by a single test" - you're right. With standards this low, this test might not be assessing much of anything.

And yet, the hysteria continues:

Elliott Wolf, a Montgomery Blair High School senior, told board members yesterday that the exams could have a devastating effect at his school, where about 10 percent of the students last year spoke limited English. None of them passed the English portion of the 2003 state assessments, according to state records. "These tests are . . . badly implemented, and the students are suffering as a result," he said.

Hm. So we have to take the word of a high school senior that the tests are badly implemented. Why? Because they're given in English? It's not surprising that limited-English proficiency students had trouble with them; it's absurd to say that English tests are "badly implemented" for people who don't know English.

It'll be up to the state, or individual districts, to decide how to deal with recent immigrants who haven't had time to learn English. But everyone who is currently in Maryland's seventh-grade classes has been warned - learn English within the next five years. I don't think that's a "devastating" requirement.

Something's not adding up in Utah

The Salt Lake Tribune reports that Utah's sophomore's are struggling with the math component of the exit exam:

Two out of five Utah sophomores will have to retake and pass at least one section of the state's three-pronged high school exit exam to collect their basic diploma two years from now. In almost all of Utah's 40 school districts, students struggled most with the math test, according to statewide results released Monday...

The class of 2006 is the first required to pass all three sections of the Utah Basic Skills Competency Test (UBSCT) to earn a basic diploma. The Legislature mandated the exam in 1999. Three out of five sophomores passed all three sections, while 16 percent passed two sections, 9 percent one section, and 14 percent failed all three sections.

All the failing students have four more opportunities to pass. The results for all 40 districts can be found online. Summer classes and special remediation are in the works. And officials are saying that students just aren't motivated enough:

"There were still some students who looked at the UBSCT testing as something that probably was not going to count, so when they see they're going to be taking it again, hopefully they'll see it's a little more serious," said Garett Muse, principal of Cottonwood High School...

"Anecdotally, what we hear from other states is that there's lots of senior-year repentance, but that's so scary, to wait until your senior year to get serious about the test," said Louise Moulding, the state Office of Education's director of evaluation and assessment.

Agonizing in Ohio

The Akron (OH) Beacon Journal is a bastion of obviousness, I tell you. Check out the headline on this article:

New state tests could keep more from diplomas: Newspaper report says difficult questions designed to ensure that more high school students will fail

Wow. Who at the BJ thought this needed to be spelled out for readers? For those readers who have repeatedly flunked such an exam, I suppose.

Beginning next spring, sophomores must pass the new Ohio Graduation Test in five subjects if they're to graduate on time in 2007. The new test requires achievement at two grade levels above current graduation exams, which means thousands could fail, the newspaper said.

Three-quarters of sophomores flunked a sample version last year, and nearly one-third failed this spring after the department shortened the test and lowered the recommended score for passing.

So this new, "tougher" exit exam has already been shortened and the standards have been lowered, and two-thirds of 10th-graders can pass it. Isn't it supposed to be a test for 12th-graders? What's the uproar about, then?

Scoring mistakes will become more common as states rush to meet deadlines, said W. James Popham, professor emeritus at UCLA who ran his own testing company. "But scoring mistakes can be corrected,'' he said. "What worries me more is the harm that will be done to children because of lousy tests.''

As compared to the harm done by lousy teaching?

And testing companies are trying to program computers to score essay questions to save money. A Dayton Daily News reporter composed a deliberately nonsensical essay that one company's program awarded a perfect score and declared "effective writing.''

Finally, a valid point. Automated essay scoring programs that would catch something like this do exist, but I'm sure there are programs that don't catch this method of "gaming" the system. But a good reporter would ask why essay questions are being introduced onto these types of standardized exams, when such questions are so expensive and difficult to score.

It's because of the anti-testing types who insist that multiple-choice items are not "authentic" enough and don't measure "real" learning. Somehow, test developers who respond to those charges end up getting blamed for iffy essay scoring procedures as well. Gee, it's almost like there's no test that would please some of these people. You think?

George Madaus, a senior fellow with the National Board on Educational Testing and Public Policy at Boston College, said more attention should be paid to whether achievement tests accurately predict academic success.

"That is wrongheaded and makes no educational sense,'' he said.

This must be a misquote. Madaus is right when he says failures should raise alarms. But surely he doesn't believe that no one should fail these types of tests. Perhaps he means such tests should not be norm-referenced with a standard set to guarantee failers, which would be correct. But the reporter should explain that.

The state adjusts the difficulty of questions so that most students get average scores and a small number get the very highest and very lowest. The idea was to identify struggling students and get them extra help.

Depending on where and how the standard is set, though, this doesn't mean a certain percentage of students are guaranteed to fail each time. It just means most of the questions are of average difficulty; thus, students who perform below average are more likely to fail. There's no way to avoid this other than to (a) put only easy items on the test, which doesn't help identify those who have trouble, or (b) keep the difficulty as is, but set the standard so low that no one fails.

Look, when a state implements an exit exam, some students are going to flunk it, because some students are either unmotivated, unintelligent, or underachievers. States should pay attention to how many flunk and why they do so. But all this hand-wringing because Ohio has created a test in which students who perform below average may flunk is ridiculous. The majority of the 10th-graders passed the exam. Why should we worry about the 12th-graders who don't?

Some students who excel in the classroom don't do well on achievement tests. At Dayton's Meadowdale High School, 18-year-old senior Tynisha Edmondson makes A's in science classes but for four years has failed to pass the science proficiency exam -- despite coming achingly close. In a retest in March, she scored 198, two points shy of passing. If she failed the test she took this month, she can't graduate and might not be able to attend Wright State University in the fall.

"I'm scared,'' Edmondson said.

The article ends with this heart-wrenching anecdote. Tynisha has taken science classes for four years, but the article doesn't mention which types of classes they are. Were they the right classes to be prepared for the science exam? Here's the new guide to the graduation exam - skip to page 11 for the Science stuff. The sample item given is rather inventive. It's certainly not pure recall; it requires students to actually think about the task and synthesize known information.

The items may very well be challenging, and rather than tugging at our heartstrings with anecdotes about poor test-takers, the reporter should be asking why someone who's made straight A's in four years of science is struggling with this material.

Thanks to Daryl for the link.

Dumping the TAKS?

Testing reform got snuck into a Texas finance bill: The high school exit exam TAKS may be replaced.

A little-noted section buried in the finance bill given final approval by the House on Wednesday would kill off the high school TAKS entirely. Its replacement: a new series of 13 course-specific tests tied to classes such as world geography and English II.

"A lot of students are going to find themselves in an awkward position at the end of their high school career," said Rep. Fred Hill, R-Richardson, who said the expected high failure rate for juniors was a major reason he supported the change.

You mean, even more awkward than they already were, given that they had yet to master basic skills? I know, I know, the TAKS isn't perfect. But unless the consensus is that it was completely irrelevant, dumping it solely due to the fail rate isn't necessarily the wisest move.

The bill would require the Texas Education Agency to have these new end-of-course exams – four in science and three each in English, math and social studies – in place by the 2008-09 school year.

Once the end-of-course tests were in place, students would have to pass at least eight – two in each of the four subject areas – to graduate.

Does that mean someone could pass only the 9th and 10th grade science, English, math, and social studies exams and still graduate? I suppose if the TAKS was on a 10th-grade level, this really isn't that different. End-of-course tests certainly have their benefits, including a more relaxed testing schedule and, ideally, a greater link to course content. This won't stop the battles about "teaching to the test," though, not by a long shot. And if students flunk these tests at a high rate too, what's next?

The results of Alaskan exit exams

In Alaska, one high school struggles with the consequences of an exit exam:

Fourteen [out of 347] Juneau-Douglas High School seniors who haven't passed the state exit exam will receive certificates of achievement rather than diplomas on graduation day, Juneau School District officials estimate...The Class of 2004 is the first to have to pass the Alaska High School Graduation Qualifying Examination as one of the requirements for a diploma. Students also must earn a certain number of credits by passing courses in various subjects.

The number of seniors who wouldn't earn diplomas this year would have been higher. But the state has exempted special-ed students from the requirement as the state looks for a way to settle a class-action lawsuit.

So that's only 4% of seniors who are not special-ed who flunked the exit exam while fulfilling other graduation requirements. That's not a huge number, but there's plenty of fretting about them nonetheless:

Students who haven't passed the exit exam - which consists of separate tests in reading, writing and math - can retake the portions they failed. There's no limit on the number of retakes. But the test is offered only twice a year, in early October and early April, and it takes several months to get the results.

Employers and institutions such as the military and colleges vary in their stance toward people who hold only a certificate of achievement. Without a diploma, students aren't eligible for apprentice programs in the trades, said Jim Williams of North Pacific Erectors.

Again, this is 4% of the non-special-education seniors. That means 96% got through just fine.

Not having a high school diploma might affect the admission of marginal students, said Eastern Washington Director of Admissions Michelle Whittingham.

"We are looking at creating some different types of courses or study sessions," JDHS Principal Deb Morse said.

Emphasis mine. One way to have students take the exit exams seriously is to withhold their diplomas if they fail it. Offering special programs to help those who fail is fine - and the school should certainly look hard at why students who fulfill other requirements fail the exam - but at some point, the school has to send the message that it's up to the student to work as hard as possible to learn the material to pass this exam. And the school should unapologetically withhold diplomas from those who don't pass the test.

Councilwoman, educate thyself

NYC Councilwoman Margarita Lopez doesn't agree with Mayor Bloomberg's policy on testing third-graders. Unfortunately, in opposing the policy, she ignored the First Rule of Making Public Criticisms, which is to use thy spellchecker and grammar checker:

A City Council member who was blasting the Bloomberg administration's social-promotion policies sent out two press releases containing spelling and grammatical errors.

"I take total responsibility . . . The member of my office who committed the mistake is going to be protected by me, the same the way that I protect the children of the City of New York," she said.

Unfortunately, Councilwoman Lopez "protects" NYC's children by insisting that these tests discriminate against minorities. How is it protective, as opposed to racist, to insist that kids of certain races just can't be expected to learn to read and answer simple multiple-choice items?

NYC kids survive math exam

Third-graders in NYC thought yesterday's math exam was a snap:

Free at last, third-graders burst out of Jamaica's PS 117 with smiles Tuesday after weeks of no play and all study for a city math exam that could hold them back from fourth grade.

Education officials said 98.3 percent of third graders took Tuesday's test, compared to 97.8 last year.

Can't wait 'til the scores come in.

Put to the test again

Last week, NYC's third-graders took the much-ballyhooed reading test; today, the math test awaited them.

Thousands of third-graders in the Bronx and city were put to the test once again Tuesday, this time in math...

Students must pass both the math and reading exams in order to move onto fourth-grade. It's part of Mayor Bloomberg and the Department of Education's plan to end social promotion. The new policy has divided parents, with some threatening to keep their kids home on the day of testing.

Well, 98.2% of NYC's kids took the reading test, so I doubt any fewer than that will take the math. And what's up with the final sentence of this WABC news article?

It is estimated that 15,000 kids could fail, which is nearly one in five of the city's third graders. In fact, last January, the city's department of education issued letters putting nearly 32,000 students into the "promotions in doubt" category. That uncertainty could continue for months.

Low scoring kids can move on as long as their teachers can demonstrate that they have tried everything.

"Have tried everything"? What on earth is that supposed to mean? Students who fail both attempts at the exams can appeal the decision in the hopes of being promoted, but this comment suggests that as long as kids try everything, even if they fail, they can pass. I don't think that's what the NYC government has in mind.

Also, third-graders who were recent immigrants and thus exempt from the reading exam still have to take the math exam, which is full of word problems:

The math test will be available in Spanish, Chinese and Haitian Creole, but schools are often on their own in finding oral translators for Bengali, Urdu, Korean and dozens of other languages spoken by public school students.

Several children and immigrants groups have been discussing suing the city if they find Chancellor Joel Klein's third-grade retention policy discriminates against immigrants.

While the definition of "discriminates" is most likely being defined rather loosely here, the critics have a point. There's no reason on earth to load up a crucial math exam for third-graders with word problems, when even the smartest kids are still working on learning to read well. Word problems can be great for measuring how well an examinee knows when and how to apply mathematical rules, but the assumption is that all examinees are on a level playing field when it comes to actually reading the words in the problems. The city would have a better leg to stand on if the math exam wasn't measuring both reading and math skills.

Here's last year's grade 4 exam. As far as word problems go, these aren't too bad. They're not too wordy, and they're pretty direct. But an argument could be made for having most of the test resemble questions 1, 2, and 3, as opposed to all the word and graphical problems that follow.

The testing hullaballoo in NYC

I was in a data cocoon all last week, so I missed the big story: the "make-or-break" reading exams for NYC's third-graders were given on Tuesday the 20th (the math exams will be administered tomorrow). Unsurprisingly, the New York Times ran a lengthy story that depicted every parent as worried about the exams, and practically every student as terrified to the point of physical illness. You mean there isn't a single parent out there worth quoting who supports the exam? Gee, no bias there! (The Daily News, on the other hand, led off the first post-test article with a quote from a confident student).

Interestingly, there's already a big flap about the re-use of items on the exam:

The city's test-maker yesterday defended its practice of repeating questions on its exams from year to year - making the program more susceptible to the type of security breaches that occurred on Tuesday's high-stakes reading test - because it's cheaper than designing a whole new exam...

Stirring the controversy even more, this is the first year where the third-grade test results will largely determine whether a student gets promoted.

The test developers have defended the anchor items (which are indeed common in this type of exam), but it's easy to see why the critics insist some kids might have an advantage. There are legitimate reasons to re-use test items - the anchor items allow for comparison of cohorts from year to year - but when items are re-used, it's crucial to keep the old test forms secure. So principals have been ordered to confess if they let students see copies of old exams, and in a bit of bizarre humor, the makeup exams for students who missed the test were cancelled after one TV station ran a close-up of the test at the behest of testing critics:

Close-up images of the third-grade test booklet were shown yesterday on NY1 News, the news cable station, and possibly on other local stations, as part of a news conference held by critics of standardized testing, who have been among the most vocal opponents of the mayor's tough promotion rules.

Still, officials said that showing parts of the test on television was enough of a security breach to require them to cancel makeup exams that would have been given this week or next Monday and that students would have to wait until after next Tuesday's math test for a special makeup version of the reading test.

The city might sue, claiming copyright infringement. As for those critics, Josh Plotnik of the Cornell Sun - no fan of standardized tests - is having none of their claims:

I've never been a particular fan of standardized tests of any kind. Even as a soon-to-be graduate student, I've never done exceptionally well on the SATs or the GREs -- I'm pretty confident that a NYC third grader could surpass my first GRE verbal score. Standardized exams test irrelevant information, attempt to deceive you, and force you to be so scared of never succeeding in life that you dread taking any exam at all. And yet, I find myself adamantly supporting Bloomberg's new "hold back" policy.

The NYC third grade English exam was created to test third grade reading skills, not to trick third graders into abandoning their career goals.

Plotnik then reports that City Councilman Charles Barron "claimed the reading exam favored white children, and that the entire test was racist." Because we can't expect black children to know how to read and answer test items? Why not? I can't think of anything but racism that would explain such a willingness to excuse any poor test scores on the part of minority students. Much better to continue to allow their schools to continue failing them, I suppose.

All this, and there aren't even any reading scores yet. Sheesh.

New exit exams in the Palmetto State

South Carolina is implementing a new, tougher exit exam:

An estimated 45,000 sophomores at public schools across the state will take High School Assessment Program tests over three days, beginning Tuesday...All students in their second year of high school will take the new tests. The results will go on individual transcripts and also play a key role in school ratings.

Unlike the old pass-fail exam it replaces, the new test has a four-tier grading system. A student must get a “Level 2” score or higher to satisfy a state graduation requirement.

Students will be allowed four retakes - not a crazily large number, I suppose. The new test is untimed. And hey, they got students at my alma mater to comment:

College-bound students at Lexington High School say the new test — they took practice versions of it a year ago — is more challenging, but not overwhelmingly difficult. "On the old test, you could guess and get answers right," said Robby Meldau. "Your weaknesses can be exposed by the new test"...

Burns said the old test, which his class had to take as a graduation requirement, had laughable wrong answers to multiple-choice questions that made the right answer obvious.

Here are samples of the new, presumably non-laughable items for ELA and Mathematics. I didn't look too long at the ELA items; I saw one which requires students to read and assess a poem, the very thought of which gives me hives.

The Mathematics items, though, seem clear and concise (students can use calculators). I sent this one item to my boyfriend:

13. Suppose you have one of each of the following items in your closet.

How many combinations can you make using one item from each category?
A. 9
B. 12
C. 18
D. 24

This item is funny to me because my boyfriend watches me go through many, many permutations of about 10 black sweaters/blazers, 20 black skirts, and 10 pairs of black shoes every morning. You'd think that everything being black would mean I'd get dressed quickly, but no. The expression on his face as he watches me ranges between amused and appalled.

I also have a really, really hard time accepting that a red shirt, brown pants, and plastic sandals constitute a legitimate clothing combination. But that's just me.

No exit exams for the disabled in Alaska

Alaska's disabled students are officially off the hook. After some legal discussions and class-action lawsuits, Alaska Education Commissioner Roger Sampson has announced that Alaska's disabled students need not pass the state exam - this year - if they've fulfilled all the other requirements for graduation:

The lawsuit was filed March 16 in U.S. District Court by Oakland, Calif.-based Disability Rights Advocates. Sid Wolinsky, an attorney with the group, said three-quarters of disabled Alaska students were flunking the graduation test, which assesses proficiency in reading, writing and math.

The Alaska plaintiffs are seeking "reasonable accommodations" for disabled students, such as a read-aloud format for students with dyslexia or judging them on grades, comments in class and performance on projects rather than on a test.

Emphasis mine. Not to pick on dyslexics, but isn't this as much as admitting that students should be allowed to get Alaskan high school diplomas while being functionally illiterate? And why weren't the lawsuit filers demanding to know what the disabled students were being taught over 12 years of schooling, if the result was a high likelihood of failing a basic skills exam?

Dissing the test in Delaware

A little bird named Daryl C. mentioned to me that Delaware's plan to develop a "tiered" high school diploma system - based on standardized tests - may be shelved:

Two state representatives from New Castle County said they will introduce legislation, perhaps as early as today, to put a moratorium on the state's controversial three-tiered high school diploma system set to take effect in June.

If adopted, the moratorium would delay implementation of the tiered system until 2006, meaning that all graduating seniors would receive the same diploma this year and next year.

The gist of things seems to be that the Delaware Student Testing Program is under review, allegedly because students are dropping out when faced with receiving diplomas based on test scores. Many eighth-graders have been retained due to test scores, and if the moratorium doesn't pass, many seniors will recieve "basic" diplomas due to their low test scores (although I find it hard to believe that students would rather drop out than receive some kind of diploma).

What's more problematic is that some of those low scores appear to be anomalies:

What may have grabbed the attention of many legislators, however, is the anomaly that some seniors did not score well enough on the standardized tests to receive distinguished diplomas but are on the honor roll, getting good scores on college entrance exams and being accepted to good colleges.

Yvonne Johnson of Wilmington, co-chair of Advocates for Children's Education, which opposes high-stakes testing, called the proposals a great victory for her group.

I certainly don't oppose high-stakes testing, but these kinds of test scores should correlate to some extent. Grade inflation could explain the honor roll standing, but I would be curious about how many students that did well on the SAT did poorly on the Delaware exams. One could argue that a low correlation between Delaware's state scores and SAT scores points to a lack of concurrent validity for Delaware's exams.

A commenter on Daryl's site claims the Delaware test was never designed for individual student rankings, only for district-level comparisons. If so, that's yet another threat to the test's validity for the purpose of a tiered diploma.

More trouble with exit exams

Just yesterday I posted about some issues surrounding legal challenges to Alaska's high school exit exam. This morning on Fark I see the source item being discussed, and a Farker noted that:

A group of parents are suing in Ontario 'cause their kids failed the standardized literacy test. One parent on the ol' CBC admitted his daughter was illiterate... but he's still suing.

Premier Dalton McGuinty says he's open to changes that would make the test more "sensible," whatever that means.

When are exams discriminatory?

A federal class-action lawsuit against Alaska's exit exam was filed yesterday:

The lawsuit charges that Alaska's exit exam discriminates against students with disabilities, making it difficult -- or impossible -- for them to receive a diploma. The complaint said the state has created widespread confusion by repeatedly changing its regulations for disabled students and what modifications in testing conditions they can receive.

Under the current rules, the lawsuit argues, more than two-thirds of the state's disabled high school seniors will not graduate in June.

I agree that the accommodations issue needs to be resolved. If I were a parent whose child had to have items read to them all through their schooling, I'd be upset if that accommodation was not included on the exit exam. But one key point being left out here is whether accommodations change the nature of what's being tested.

A student who is visually disabled cannot read printed material no matter how smart she is, and it's not changing the nature of a reading test to have someone read the items to her. But when the disabilities become more amorphous, like ADD and learning disabilities, I no longer have confidence that the accommodated form of the test, such as a test read aloud, is measuring the same thing as it would be in its unaccommodated format. Thus, one can argue that student with non-physical disabilities are not being held to the same standards as non-disabled students; this makes me wonder how on earth we can claim that a high school diploma is meaningful if we're willing to different set of standards for it for different students.

And I'm very suspicious about such lawsuits when I read things like this:

But many parents and advocacy groups say such exams illegally discriminate against special education students, immigrants and minorities, who have disproportionately low passing rates.

This tendency to lump together special education students (who presumably cannot learn more than a certain amount), immigrants (who presumably have just not had enough time to show what they can learn), and minorities (who tend to have problems learning, but there's no consensus as to why), and then present these charges in an article about disabled students, is a real problem. The issue of accommodations can, and should, be discussed without random charges of discriminations being introduced; the fact that special ed students, immigrants, and minorities may do worse on these exams has nothing to do with the accommodations issue.

A printed test would discriminate against a blind student, because no matter how smart that student is, she can't read the material. But a test does not "discriminate" against test takers who have not mastered the material but have the physical capability of doing so.

Why are students nervous about this exam?

Los Angeles sophomores are tackling the California High School Exit Exam for the first time:

Thousands of Los Angeles Unified 10th-graders will spend today and Wednesday taking the California High School Exit Exam -- a test they must pass before they can earn their diplomas in 2006...

The exam debuted in 2001, replacing a basic-skills test. But when the first round of scores turned out lower than expected, the state Board of Education postponed using it as a graduation requirement until 2006. For most of the state's 500,000 sophomores -- representing the Class of 2006 -- today will be their first try at passing the test, which includes both math and language-arts sections.

I'll be eagerly awaiting the results, especially because of this line:

While most of the questions are based on eighth-grade material, only 59 percent of last year's sophomores passed the math portion and 78 percent of students passed the language arts section in sample tests.

Emphasis mine. I've been reporting pretty consistently that this exam is at the 10th-grade level, but further investigation shows that while the English portion is geared towards concepts that can appear in classes up to 10th grade, the math portion is sixth- and seventh-grade math, and Algebra I (I should have paid closer attention to the CA high school student who commented on this post.) The math portion has the lower passing score (55% of 90 items) and is made up of multiple-choice items only. Study guides can be seen here; only 12 items relate to Algebra I, and those items are quite simple.

(And don't even get me started on their statistics items, which have the least-distracting incorrect options I've ever seen. The only way they could help the students along more would be to include bright red arrows that point to the correct answers.)

How does mastery of middle-school math material provide useful information about whether a student deserves a high school diploma? And why are superintendents like Bob Collins saying things like this?

"We've spent a tremendous amount of time building their confidence," he said. "A lot of kids are intimidated by this test."

Why? They're 10th-graders. The passing scores are set between 55 and 60%, depending on the section. The math questions don't go above Algebra I. Students have five chances to pass, one of which is during the summer after their senior year. If these students are intimidated, something is seriously wrong.

What's worse, some principals foresee a future with special diplomas for those students who can't pass:

At Cleveland High School, 42 percent of students passed the math test and 64 passed the English test in 2002-03, before it was a graduation requirement. If that trend continues, Principal Allan Jay Weiner said the state may have to start offering different types of diplomas for students unable to pass the exam.

"We have so many different kids that come here that speak different languages," he said. "It's kind of weird that you come into a country, and you have to take a test, and you can't even read it."

That's why they get five shots at it before graduation time. And if they can't read an eighth-grade level exam, how will they pass higher-level material that is presumably taught in English? Will their special diplomas need to be printed in their native languages so they can read them?

Something's fishy in Maine

Well, this is a new twist on the exit exam scenario:

Florida's education chief does not like what he sees as a growing loophole in the state´s standardized graduation exam, allowing failing students to obtain diplomas from a private school in Lewiston, Maine. North Atlantic Regional High, a private school designed to assist home schoolers, is issuing diplomas to seniors who flunk the Florida test.

For a $255 fee, North Atlantic Regional offered to accept her course credits and issue a diploma _ even though she has never attended classes there. Within weeks, Fourron was able to begin classes at Miami Dade College.

And will Fourron graduate from Miami Dade, when she's already demonstrated the inability to read English and do math at a 10th-grade level?

The school sounds mighty fishy, not least because its diplomas are allegedly legit in Florida - but not in Maine, where the "school" is based:

...a spokesman for the Maine Department of Education said the high school may be misleading students when it claims on its Internet site to "have the authority and privilege to grant high school diplomas in the State of Maine."

"The state of Maine does not recognize their grades, credits, transcripts or diplomas," said spokesman Edwin "Buzz" Kastuck. "If you´re home-schooling your children, you can issue them a diploma from your kitchen table _ we look at it the same way."

The school's administrators, Steve and Carol Moitozo, supposedly screens each student's academic accomplishments and converts them to a numerical scale, which is then compared against Maine's graduation requirements. If their total score is over that limit, the school issues a Maine high school diploma, despite the fact that the majority of the school's students live outside Maine. And it's supposed to be a place that provides assistance to homeschoolers, yet it appears to be issuing diplomas to kids in Florida's public school system who can't cut the mustard on the FCAT. Fishy, indeed.

WASL going to happen now?

Someone on the editorial staff of the Seattle Times is aggravated with a state senator's proposals for the Washington exit exam:

State Sen. Steve Johnson, R-Kent, should stop messing around with a well-crafted House bill that clarifies state high-school-graduation requirements, and work instead to pass it without major changes.

Johnson would limit students to two retakes of the WASL. Those who fail both times could take what's known as a norm-referenced standardized achievement test. Both changes would undermine the intent of the state's ongoing standards-based education reform.

I see no point in switching to an alternate assessment after two retakes, most especially not to a norm-referenced assessment. Not only does that, as the author points out, defeat the purpose of setting objective standards, but if only those who fail the WASL twice take the alternate test, how are the norms going to be set? If the norms are set by the general population, kids who fail the WASL will fail the alternate test, because they've already been defined as being below all the other students. If norms are set by the population who takes only the alternate test, then the message being sent to students is, if you can prove you're the smartest of all the dodos who failed the WASL twice, you get a diploma. Ridiculous.

As for how many retakes, yes, one could argue that more than two retakes should be allowed. But schools should also be prepared to set a limit on retakes, or give unlimited retakes but insist that a student doesn't graduate until they pass the test, whether it takes one administration or ten.

Slightly more than one-third of the state's 10th-graders passed the reading, writing and math portions of the WASL last year...

Johnson questions the reliability and validity of the WASL.

I have a feeling these two sentences are not unrelated. The assumption that a test that gives politically-incorrect results must be unreliable or invalid is often made by those who fear (or are ignorant of) tests. This doesn't mean the WASL is reliable or valid, of course, but a third of the 10th-graders flunking doesn't mean it isn't, either.

I also think Johnson's comments are not independent of what was said in this meeting:

Superintendent Dolores Gibbons, Marcie Maxwell, Board Member and Legislative Representative and Dimmitt Middle School principal Kathleen Heaton-Bailey hosted a meeting today with state legislators to discuss the state and federal government’s reform efforts including the WASL...

Kowalis explained that although the goal to fully educate every student is laudable, the reality is children are individuals with very individual needs and cannot all be expected to perform at the same level.

Fine. But if you admit that some 17-year-olds cannot be expected, ever, to perform on the same level as most others, you also have to be prepared to deny diplomas to those students. There's a difference between saying, "Kids may perform at different levels," which is absolutely true, and "No matter what level a kid is at at age 17, they should get a diploma if they stayed in school," which is highly debatable.

“We’ve built an entire education system on a ‘no cookie cutter’ model,” Kowalis said. “Now we present this one test [WASL] as a way to measure their competence.”

Hedeen asked if an alternative test or other form of assessment could be considered for students who could not pass the WASL. In 2008 high school students will be required to pass the WASL before graduating. Hedeen noted that, for students who do not pass the test in the 10th grade, teachers in 11th- and 12th-grade classrooms would have to spend all their time helping them meet that goal. That, said Hedeen, could cause districts to restructure curriculum and possibly exclude subjects important to those students who did pass the test.

All their time? Really? Wouldn't ability tracking negate a lot of this? This wouldn't affect those kids in AP classes, I'm sure. And can anyone explain to me why the material normally be taught in an 11th- or 12th-grade classroom would not help a kid on a 10th-grade exam? The implication here is that teachers will have to "dumb down" 11th- and 12th grade material for everyone, but if that's the case, why are these 10th-graders being promoted to the higher grades? If they're so clueless that teachers will have to spend all their time helping them, why promote them?

These teachers and administrators want it both ways. They want agreement with their declaration that all kids are different, but they want diplomas to be "one-size-fits-all" - awarded to kids regardless of their achievements. They want to focus on those 11th- and 12th-graders that pass the WASL by not dumbing down the material, but they want to promote into those grades kids who can't pass a 10th-grade test. No wonder the idea of an exit exam aggravates them.

Before the exit exam, a pre-exit exam

Freehold Regional High School District in New Jersey is concerned that some students won't pass the High School Proficiency Assessment (HSPA) in the 11th grade (the test is required for graduation). So they want another test, for ninth- and tenth-graders, to identify those kids who are in danger of not passing the HSPA later on. Even for me, that's too much testing:

With March drawing closer, Freehold Regional High School District administrators are taking a look at HSPA testing. The district’s principals put their heads together at a recent brainstorming session and asked administrators to find a standardized measure in order to help identify students in the ninth and 10th grades who will be at risk for not passing the High School Proficiency Assessment when they reach the 11th grade...

...the test can profile each student in order to de­termine if there are areas of weakness, while the student is in grade nine and 10. Instructional improvements can then be deter­mined for individual students.

The ultimate goal, of course, is to have 100% of the districts 11th-graders pass the HSPA. There are a few problems with this plan; here's a partial list.

* There doesn't seem to already be some sort of natural classroom-level assessment that gives administrators a good idea of the potential flunkers. Nor does there appear to be any existing preparation for the exam, because administrators are talking about 11th-graders who are walking in "completely unprepared" for this exit exam. Both of these are issues that should have been resolved before an exit exam was implemented.

* I worry that the 100% goal will lead to a dumbing-down of the exam, more pre-exit exam tests, or both.

* The 9th- and 10th-graders will be taking the tests under low stakes. Thus, the numbers are likely to look much worse than they will for the 11th-graders performing under high stakes. It's possible that the lower-grade scores will be so poor as to be useless for predictive or diagnostic purposes.

This is the HSPA, by the way. The Language Arts section contains a great many open-ended items, with assessment in "speaking, listening, writing, reading, and viewing experiences." That differs enough from your average MCQ-based exit exam that it's hard to understand why a solid preparatory curriculum was not put into place parallel with the exam.

Working with Text (ESPA) - Interpreting Text (GEPA and HSPA)
Working with Text refers to those activities in which students use strategies to interpret or reformulate meaning from the text. Questions and tasks with this focus will ask students to identify main ideas, supporting details, directions, paraphrasing, text organization, and purposes for reading, listening, or viewing.

Extending Understanding of the Text
Extending Understanding of the Text refers to those activities in which students use text already generated, that is, informational and everyday texts, as a springboard for generating their own work and ideas. It is a self-contained component for which students will read a passage containing detailed information and use the information from the passage to make decisions, solve a problem, and create original work through a writing project that is designed to extend their understanding of the text...

The Math section looks equally ambitious:

New Jersey's...eleventh-grade mathematics tests assess knowledge and skills in four content areas or clusters:

Number Sense, Concepts, and Applications
Spatial Sense and Geometry
Data Analysis, Probability, Statistics, and Discrete Mathematics
Patterns, Functions, and Algebra

Here are the "content clusters" in more detail.

It's not that wanting to pretest the younger students is a bad thing, necessarily, but, to use the math portion as an example, a NJ 11th-grader isn't going to have much chance of passing unless they've taken classes that give them a solid foundation in algebra, geometry, statistics. So the logical thing to me seems to be...require NJ students to take those classes, and align the class content to the test content, or revise the test content. Giving another test isn't going to do much good unless what NJ needs even more evidence to support tying classroom requirements to the test content.

Any NJ residents, feel free to contact me with additional information about this...

Putting the exam to the test

The entire sophomore classes at Vintage High School and Napa High School (CA) were among those who took the California High School Exit Exam this week. The test, which is at a 10th-grade level, will be required for graduation for these students in 2006:

Following a decision by the state assembly last year, the state board of education voted to postpone the exam for two years because of concerns that students weren't prepared for it. A report found that about 20 percent of the class of 2004 would fail the test's math portion and not graduate.

This is the third time the schools have given the test to the entire sophomore class, but it's the first time it actually counts toward graduation under the new rules.

The MCAS battle continues

The state of Massachusetts has said to high school students -"Nice try, but no cigar. The MCAS is in your future":

The state's highest court refused yesterday to block the use of the MCAS exam as a graduation requirement, dealing a blow to high school students who are suing to abolish the controversial test. The Supreme Judicial Court denied a request for an injunction to stop the state from giving the high-stakes exam pending the outcome of a lawsuit, saying an injunction "would undermine educator accountability and hinder education reform."

The ruling is the latest development in a class-action lawsuit over the Massachusetts Comprehensive Assessment System that contends that the test unfairly discriminates against minority, special education, and other students who have not been adequately prepared for the exam.

To be honest, it's probably true that some students haven't been adequately prepared. But the fact that minority and special ed students do worse on the MCAS is not evidence that the test "unfairly discriminates." There's differential impact, but the test may still be fairly identifying who knows the material, and who doesn't.

Characterizing the SJC ruling as a minor setback because it dealt with only a small portion of their case, lawyers for the students say they are not abandoning their efforts to have the MCAS declared unconstitutional....Meanwhile, state officials are hailing the decision as a sign of more favorable rulings for MCAS.

Apparently, the lawyers tried to argue the exit exam focused "too narrowly" on English and math. What should it focus on - painting and music? Chemistry and geography? The argument that a test that measures the two most basic educational skills is too "narrow" doesn't make sense to me.

...the judges ruled that MCAS legislation gave state officials discretion to phase in subjects such as history and science "in a reasonable manner and on a reasonable timetable."

In addition, the ruling said, state officials "could properly conclude that a student should have competence in `reading, writing, and arithmetic' before being tested on competence in science, history, and other areas."

Exactly.

Meanwhile, lawyers are preparing for trial in state court on the rest of their case against the MCAS requirement, and are amending their complaint to include arguments that the test also discriminates against students enrolled in schools that were declared "underpreforming" by state and federal government.

Withholding high school diplomas from students who pass all other graduation requirements violates their constitutional rights, he said.

Okay, let me get this straight. The MCAS is allegedly a "narrow" test, and we know it tests only English and math. Kids at underperforming schools are being denied these skills by their teachers, as evidenced by MCAS scores. They are not learning basic English and math skills.

And in response, these plaintiffs wants to claim both that the tests are "unfair," because they're measuring what they're supposed to be measuring, and that kids from underperforming schools, who haven't mastered basic English and math, deserve diplomas. My head is spinning. If the MCAS is "unfair" because kids haven't learned the material, why should we assume they should be allowed to graduate? And where in the constitution does it say that these, or any, students have the right to a diploma?

Yes, Virginia, there is an exit exam

Yet another state creates toughens high school graduation requirements, then refuses to stand behind them:

The Virginia General Assembly is being asked to delay the state's tougher, new graduation requirements indefinitely.

To be fully accredited, at least 70 percent of a school's students must pass SOL tests in English, math, science and history. Based on last spring's tests, 1,414 of the state's 1,823 schools - 78 percent - are fully accredited.

In other words, 22% - almost one-fourth - of Virginia's schools are failing to educate at least 30% of their young charges.

Starting with this year's graduating seniors, students are required to earn six verified credits and 22 standard credits to receive a standard diploma. A verified credit is awarded when a student passes a course and the corresponding SOL test, or another state-approved standardized test.

So, only six courses require standardized exams (and I bet those exams are "minimum-competency level," too.

Van Yahres, a member of the House Education Committee, filed the bill Monday. He said it would correct a situation in which students are being penalized while the schools are not yet required to be fully accredited.

So, we can't expect kids to do well if it's clear their schools are failing them. So how do we impress upon schools that they have to start doing things right? I agree, it's unfair to kids in one sense, but if kids are allowed to graduate from poor schools, it's arguable whether they're better off than if they're held back while their school is forced to make changes.

Richmond School Board Chairman Larry A. Olanrewaju called the legislation "a bad idea."

"It's going back to making excuses for the children," he said. "I think we need to continue to focus on what we're doing and continue to offer local verified credits as state has already done. That will be more helpful than repealing the whole process because otherwise, we will go backwards."

Parents sue to abolish literacy testing

And speaking of the new wave of litigious parents, a group in Canada are threatening to sue unless the 10th-grade literacy exam, required for graduation, is dropped:

A group of parents is threatening to sue the Ontario government if it doesn't abolish the mandatory Grade 10 literacy test that must be passed to graduate from high school. David Baker, the lawyer representing the students and parents, claims the standardized literacy tests were designed to fail 20 per cent of students because they don't take into account the needs of those with disabilities or those in the applied rather than academic education stream...

"We are concerned of the impact of the test on kids," he said. "We've been looking at this for a while."

There's nothing wrong with looking at impact, which is different from bias. A test that is perfectly fair and bias-free can still have differential impact on different groups. If Group A is of high literacy ability and Group B is of low literacy ability, then a fair test of literacy skills will pass more of Group A than Group B, and negatively impact Group B more than Group A. The issue of fairness becomes relevant only if the test measures something other than literacy and still fails more members of Group B than Group A.

One can argue that a valid test of literacy will indeed fail more special education students if they are likely to be less literate. So the problem is not the exam; the problem is that the special education students are either mainstreamed into regular classes or are currently receiving diplomas that are the same as the regular students, despite the fact that their abilities are so much lower. Is this fair to either group of students?

Baker said about 27,000 Ontario students won't graduate this year because they haven't passed the test, which was implemented in 2001.

That year, of the 129,000 students who took the test, 75 per cent passed, said an Education Ministry spokesperson. Of those who failed and took the test the next year, 48 per cent passed, meaning that about 88 per cent of the 129,000 students eventually passed.

Does this mean that 12% of all Ontario high school students are in special ed? That's a pretty big number. Removing the test would allow those in the 12% who are not in special education classes to recieve diplomas despite demonstrating a relatively low level of literacy.

Anna Germain said her 17-year-old son Matt, who has Down syndrome, deserves to graduate from his Toronto high school.

Does he really deserve to receive the exact same diploma if he hasn't mastered high-school level material? I can understand a mother's desire for her son to be rewarded for the work he has done. I just don't understand what these parents think a diploma will mean if a student doesn't have to master high-school level skills in order to recieve it.

Exit Exam "plight" in Arizona

Now Arizona's youngsters will be facing the same "plight" as students in other states:

Thousands of Arizona high-schoolers risk not graduating in 2006, facing a plight similar to that of other students in states with high-stakes tests. Conservative state projections estimate that nearly five thousand Arizona seniors -- or about 10 percent -- will fail the AIMS test in 2006.

So that means that 10% of Arizona's students might not be smart/focused enough to earn a diploma. Call me a cynic, but that doesn't sound like an incredibly large number. The 2002 Census Report on Educational Attainment in the US gives the numbers as 84% of all citizens over age 25 having a high school diploma; the number rises to 88% when you count only those between the ages of 25 and 29. Arizona's graduation rate with the current estimate would be 90% of their senior class.

I know, I'm not counting dropouts here, and I'm sure there are those who believe that any student who makes it to senior year should receive a diploma. But certainly some kids make it that far without having advanced skills, and perhaps they really can't learn or demonstrate those skills in that last year. I'm not saying Arizona shouldn't be concerned about that 10%; the schools should find ways to help those students graduate if, in fact, they're not learning the material in the first place due to bad or inefficient teaching. But if they think the failure rate is too high, why assume it's the "plight" of the test?

"This is the political hot potato because nobody wants to be known as the person that costs kids their diplomas," said Keith Gayler of the Center for Education Policy, a nonprofit group that advocates for public education.

Actually, wouldn't that "person" always be either the teacher, or the student, regardless of whether exit exams are in place?

The Center for Education Policy reviewed all state exit exams and found Arizona's and New York's tests among the toughest, The Arizona Republic reported Sunday. "I heard it was hard," said Derrick Riggs, a Chandler High School sophomore. "It's also kind of scary because you have to pass the three parts to graduate. What if I keep failing one part and don't graduate?"

Why are these kids so scared? It's a test of high-school-level material, not boot camp. My guess is these kids are reading all the hysterical comments from teachers and testing opponents and they believe the state is out to deprive them all of their rightful diplomas. Don't kids have to take tests to pass classes in the first place?

And the statement that Arizona's exit exam might be one of the toughest exit exams around is like saying that Heidi Klum is one of the ugliest Victoria's Secret models. The Arizona exam could be the toughest exit exam and still be pretty easy.

Arizona launched AIMS in 2000, but has twice postponed the graduation requirement after debates about low scores and initial problems with content and scoring...Discontentment over poor performance is leading some states to consider alternative measures, such as waivers, exemptions and alternate test scores, so students can graduate.

I have a better idea - get rid of the tests. States that aren't going to stand behind the exams, and accept the fact that some students will flunk them, shouldn't use them. That way, there aren't all these "alternative measures" to keep track of, each of which comes with its own pitfall anyway. Want to use the SAT as an alternate measure? What about all those cries of bias on the SAT? You can be sure that some activist groups will say the SAT is just as problematic (except when, illogically, the SAT gets a pass so that other exams can be criticized).

Granted, removing the exit exam doesn't do anything to help solve the problem of high school graduates who cannot read, write, or do basic math, but at least the school administrators won't have the anxiety of dealing with (horrors!) a student who can't pass a basic skills exam.

"[Arizona students] have to pass a reasonable test to graduate," said Tom Horne, state superintendent of public instruction. "Doing a class project or something along those lines will not be allowed."

Well, that's reassuring. Makes you wonder who floated the class project idea in the first place.

Civil rights lawsuits have been filed in Massachusetts, Michigan and other states on behalf of poor, minority or special-needs students, whose failure rates on the tests can be two to three times higher than those of other students.

Let's hope those lawsuits are against the school for reasons of quality, rather than for the existence of the exams themselves. After all, it's not the exams that cheated those kids out of a decent education.

Parents also question why a student who did well on the SAT or ACT should be denied a diploma for not passing the state's standardized test. "As a parent, you can't help but worry when you hear of kids that do well on their SATs, but fail their exit exam," said John DeCamp, whose son attends Dobson High School in Mesa. "That's a red flag. Maybe the exit exams are too difficult.

Or maybe those anecdotes are only one or two exceptions in a sea of adherences to the rule. I'd like to see the correlation between the exit exam scores and SAT/ACT scores. Granted, the tests measure different things, so you wouldn't expect to see even a high correlation, necessarily. But you'd hope to not see a negative correlation, and if there are indeed high percentages of students who do well on college entrance exams but poorly on the exit exams, that would be cause for worry. The statements by these parents are not, in my mind, strong enough to be cause for worry.

In other words, let's examine the data before we decide whether this exam is really too hard, okay?

The goods on Ohio's new high school exit exam

The Advocate of Central Ohio has the "New Test, New Worries" information on the new exit exam awaiting the class of 2007:

High school freshmen prepare. The Class of 2007 will be the first that must pass the high-stakes Ohio Graduation Test to graduate. Already, Ohio students must pass the ninth-grade proficiency test to earn a diploma.

So, Ohio currently only requires mastery of 9th-grade schools for receipt of a diploma that signifies mastery of 12th-grade work. I can see why they're implementing a new exam.

However, the OGT is no ordinary test. In addition to more than 30 multiple-choice questions on the reading and math tests, students will have to write answers as short as a few sentences or as long as a few paragraphs on up to seven questions. On the math test, not only do students need to calculate the right answer, they have to support their answer by showing their work or writing an explanation. On the reading test, the selections are long.

And....is there any reason Ohio should not expect its 12th-graders to be able to do this work? "More than 30 multiple-choice questions" is not a phrase that should strike fear into young hearts, and neither should "as long as a few paragraphs" or "have to support their answer by showing their work or writing an explanation."

Several administrators and teachers, including Newark Superintendent Keith Richards, have serious doubts about the test. Richards doesn't oppose standardized testing. However, the OGT should test to see that students meet the minimum academic standards, not the maximum, he said. It's a difficult test that some students know they won't be able to pass. Richards worries those teens will drop out of school.

Since when does a test such as the one described above meet the maximum standards for high school performance? If a student can't pass this exam, whose fault is that? Partially the school's, partially the student's. If the state has decided that students should know this material to be awarded a diploma, then they should put their money where their mouth is. The school should be prepared to teach this material to everyone, but ready to deal with the fact that some kids won't be smart/focused/interested enough to understand the material. And those kids shouldn't receive diplomas.

"It doesn't do anything for graduation rate," Richards said. "It doesn't do anything for society if we tell people they aren't capable."

Um, your job isn't to tell them they're capable. Your job is to make them capable, and you're supposed to let them know whether they are capable or not.

Why are schools tying themselves up in these knots? If a school believes its job is to tell every student that they're capable, regardless of reality, then that school shouldn't give an exit exam. But if a school is going to give an exam, it should not fret endlessly about the fact that at least one student won't pass it.

High schoolers will have five chances to pass the test, which includes sections on math, reading, social studies, science and writing, before they finish their senior year.

Five chances? See? It's not that hard a task.

And there are loopholes, yes indeedy. The article doesn't make it clear if the following requirements must ALL be met to recieve the diploma; I certainly hope that's the case:

Passed four of the five tests and have missed passing the fifth test by 10 points or less.

Obtained letters of recommendation from each teacher in the subject area not passed.

Let's see, the GPA loophole will help ensure that grade inflation will be alive and well in Ohio. The attendance and expulsion loophole could be a bone of contention if attendance and/or expulsion rates differ by ethnicity. And the letter of recommendation loophole allows teachers who oppose the test to help those who flunk a part of it get that sheepskin.

Gee, are there any teachers like that?

Larry Friend, a science and math teacher at Eastland-Fairfield Career Center, took the OGT last week during a meeting of the Ohio Department of Education. Friend called the test "scary" and "discouraging."

"In many cases, if their reading skills aren't good, they're toast," Friend said.

The reading load on math tests is a legitimate point. However, it's my impression that all the new, trendy, progressive maths are the ones that include the heavy reading load to begin with, in their efforts to avoid the "dull, rote" traditional mathematics education. And if his juniors are ill-prepared for math, again, whose fault is that? Has the school failed them? If so, that doesn't justify giving these kids diplomas. Why complain about the test?

The test does not discourage youth. Poor mathematics instruction does.

Some schools are already justifiably worried:

At the Licking County Joint Vocational School, about 50 percent of incoming juniors haven't passed the ninth-grade proficiency test, Superintendent Ron Cassidy said. However, about 97 percent of students go on to pass the test and graduate.

"How can we have our students success with the Ohio Graduation Test?" Cassidy asked. "We want out students to leave here with a diploma."

What does that last question mean? "Our students success?" I assume he means "succeed." Regardless, Mr. Cassidy, if you want students to leave with a diploma, then teach them the material. Worrying about the test is not a useful way to utilize your energy.

The new and enhanced California exit exam

The Queen of Edubloggers pointedly instructs us to remember that a new and easier California exit exam shouldn't be referred to as a "dumbed-down" test. We should stop and consider those feelings of those sensitive educator officials before ridiculing the fact that the more difficult math and English question have been dropped from the exam:

The state Board of Education voted yesterday to remove some of the more difficult math and English questions on the California High School Exit Exam. The questions will be replaced with those that measure more basic skills, board members said.

"I don't think we're dumbing down the test in any way," said board member Carol S. Katzman.

Oh no, removing difficult items in no way affects the difficulty of the test. It in no way makes it easier for the less-able students to pass. It merely enhances the self-esteem of those that take it, and in California, isn't self-esteem everything?

The changes were made after careful review, she said, and will test what the high school exam aims to measure: how well students are mastering basic concepts of math and English.

Actually, it measures whether high school students in California can demonstrate, over eight attempts, that they've mastered between 55% and 60% of the sixth- to ninth-grade material now on the exam. But I think Ms. Katzman's way of phrasing it is so much more polite.

English questions will be pared, deleting a requirement that students write a bibliography of reference materials, develop research questions and methods to "elicit and present evidence from primary and secondary sources" and having students demonstrate proper manuscript formats, such as title page, spacing and margins.

All those who condemn "teaching to the test" can contemplate the fact that California's teachers will no longer be held accountable for teaching students how to do research, or to understand how papers should be titled and formatted.

So, why did California decide to "enhance" the exit exam in this way?

...a report...found about 20 percent of the class of 2004 would fail the test's math portion and not graduate. The report stated that about half of students who aren't fluent in English and three-quarters of special education students wouldn't be eligible for diplomas because of poor test performance.

In other words, making sure students receive diplomas is more important than teaching them basic math skills, or making sure that they become fluent in English during high school. I'm willing to give the special education students a pass here, but the schools really have no excuse for the fact that California's non-special-ed students are not ready for a test of skills this basic.

And if students continue to flunk, what then? Will further "enhancements" be required?

Californians faring better on exit exams

The Desert Sun in Palm Springs (CA) reports good news - the exit exams appear to be working, in part because the implementation was delayed long enough for curriculum changes to take effect:

Good news from the education front: Across Riverside County, students in public schools are doing better on the California High School Exit Exam. Many are able to pass the test, often on the first try. The number of students passing the math portion of the exam increased from 26 percent in 2002 to 41 percent in 2003. During that same period, the number of students who passed the English language portion increased from 48 percent to 64 percent.

But this is not a time for schools to be complacent. This is a wake-up call to all districts to get their standards in line with state requirements so that our students have the best chance at success. Students must take responsibility as well and work hard to absorb the knowledge necessary to ensure a successful path toward graduation.

English not required?

The ContraCosta Times has an article up about those unfortunate Californian students who might be denied high school diplomas because of "limited English skills." Some folks might agree with this situation, because they believe that even recent immigrants should learn English before being granted a diploma; students who aren't recent immigrants have no excuse for the low English skills.

What's interesting, though, is that the high school equivalency exam, the GED doesn't require English skills:

The high school equivalency test known as the GED is offered in Spanish and French. Nearly 6,500 people took the test in Spanish in California in 2001. Those who pass get many of the same educational and work opportunities as high school graduates.

The General Educational Development exam is a national test for adults who didn't finish high school. Federal policy allows assessment in primary languages... But California schools operate under Proposition 227, passed in 1998, which limited bilingual instruction. Schools are expected to teach primarily in English, and most standardized testing is done in English.

The article notes that these non-English versions of the GED are not "without controversy," and it's easy to see why. If those who pass the GED are hoping for a shot at the same "educational and work opportunities" as those who pass California's English-only exit exam, they may be disappointed. English is still the primary language in American universities and American offices, as well as on American standardized tests. Is it realistic for those who take the GED in Spanish to expect to have the same opportunities as those with better English skills?

The ABCNews view of exit exams

ABCNews tackles the subject of exit exams, but while it appears the reporter may have tried to balance out the article, he didn't adequately research the topic, and most of the space is given over to testing critics:

[Teacher] Orlinsky's concern has become an issue nationwide, as more and more states move toward requiring that every student pass a standard statewide test on top of their regular class requirements in order to graduate.

Critics of the move say that there is already evidence that children — particularly minority and poor children — are dropping out or being pushed out of schools as a result of emphasis on high-stakes testing.

The Amrein and Berliner study is quoted as "proof" that exit exams seem unrelated to actual academic achievement, or may actually impede learning - but the subsequent criticisms of the study (and one of the author's own waffling quotes) that appeared in the very recent Center on Education Policy paper on exit exams are not mentioned in the ABCNews article. Accidental oversight? Or does ABCNews want to leave readers in the dark about research showing that exit exams may indeed have a positive impact on other measures of academic achievement, a point which David Berliner himself has conceded?

The final sections of the article cover the topic of dropouts and alternative assessments, and here the reporter's lack of research and knowledge about the topics are apparent:

...research seemed to confirm what critics have said about increased rates of dropouts and so-called "push-outs," cases when poor-performing students are allegedly either counseled to leave school, ostensibly to focus on taking high school equivalency tests, or are intentionally discouraged by being repeatedly failed in the years before they would be required to take assessment tests.

"It looks like these exams may be causing dropouts, and this is really serious, because dropouts have a much harder time throughout their lifetime," Gayler said.

A few sentences later, a researcher is quoted who does note that the research linking exit exams to dropout rates is inconclusive, but it's telling that the reporter is willing to give credence to the theory than an increased dropout rate is a valid criticism of the exam, and not of the schools that are responsible for failing to educate their students. If schools raise standards in any fashion, via graduation requirements, exams, or other options, and more students drop out in response, I fail to see why critics then rush to judge the standards. Can't this also be seen as evidence that the schools are merely failing to meet their own standards?

Finally, the mantra of "holistic assessments" is once again invoked without any concern for the cost and psychometric complexity involved with these types of assessments:

There is strong evidence, though, that different people test differently, critics of the reliance on tests say, and they while the tests might be good tools for certain kinds of assessment, they are not always the best way to judge how much a student has learned...

Rather than putting all students through standardized tests, "richer, more complex types of assessment" such as student portfolios, classroom-based tests and teacher assessments would give a "much more fair and accurate" picture of what a student has learned, said MassCARE statewide coordinator Jackie King.

Does Ms. King have any idea how much it costs to develop and pilot tests assessments? Is she aware of the amount of bias that is introduced by such assessments? Are those who are are concerned about correlations aware that the lower the reliability of the instrument, the lower the correlation it will have with any other assessment - and performance assessments nearly always have lower reliability than multiple-choice assessments?

Student portfolios are expensive to develop, expensive to grade, and introduce the potential for a fantastic amount of rater bias. I've written about this multiple times (see here and here), and yet I've almost never seen a reporter inform readers about the fact that these politically-correct, touchy-feely exam alternatives come equipped with massive drawbacks and pitfalls as part of their standard operating system. This may explain the incorrect-yet-seemingly-widespread view that performance assessments are easier to develop and score - and are by definition more "fair" - than the oft-vilified objective assessments.

The burdens of exit exams

The Center on Education Policy has released a report detailing the unexpected consequences of implementing exit exams, which are now used in 19 states for over half of all US public high school students. It's not all bad news, of course, as the exams do seem to be creating a better alignment between standards and course content, and they are boosting the educational progress of students at the remedial level.

The complaints, however, are not surprising. The "teaching to the exit exam" worry is that teaching is now broad, but with little depth, and the financial burdens are higher than previously expected. There's no consistent evidence to support the notion that exit exams mean higher dropout rates. And while it's obvious that certain groups, such poor students, minority students, and disabled students all pass at lower rates, this can be seen both as (a) a reason to remove the tests, so that these students aren't disadvantaged, or (b) a reason to keep them in place, so that schools can't continue to give up on these kids and give them meaningless diplomas.

A few thing I noted from the report (which is 140 pages and will not be covered in detail here):

1. Despite fears that exit exams will be eliminated, or dumbed down beyond all recognition, the report claims that states are committed to them and are making relatively minor modifications. Some states lowered standards, but some didn't, even in the face of extreme public opposition.

2. The map on page 7 is interesting. Any reason why the exit exam fervor has left the midwestern/northern states relatively untouched?

3. The educators, once again (p. 21), are concerned with the "fairness" of a test which has differential impact. But I've written before about how these educators are essentially arguing that these exit exam must produce equal results for, and thus be considered fair to, students at the group level - but they weren't designed for that. They were designed to be fair at an individual level; if a student has the skills, they pass, and if they don't, they don't pass. These exams were never designed to make sure all groups were treated the same, but all students.

4. The Amrein and Berliner study, which allegedly showed no impact of state-level high-stakes exams to more substantial academic achievements, is mentioned, but it's balanced out by mention of the substantial criticisms that followed the study (even though Berliner is allowed to make a waffling rebuttal in this document).

The Regents Ruckus

The Fark.com headline sums it up well: "Problem: Thousands of New York teens may fail to pass regency exams and graduate. Solution: Lower the standards, of course."

Staring at the prospect of tens of thousands of members of the Class of 2004 failing to graduate next June because they couldn't pass toughened Regents exams, the state is faced with an unpalatable choice: stand firm or retreat.

Current rules, announced amid fanfare nearly eight years ago, call for raising cutoff scores on the five Regents exams required for graduation from 55 to 65, starting with this year's seniors. But testing results so far show that this could well threaten the diplomas of several thousand teens on Long Island, and far more than that statewide.

They've had eight years to prepare for raising cutoff scores from 55 to 65, and this is what happens? Boycotts, protests, and hemming and hawing from the state? And why do journalists keep protesting astonishment at the realization that exit exams, which determine who earns a diploma, will, if implemented, prevent some kids from earning a diploma? It's as thought no one realized until now that flunking the exam means losing out on a diploma. Did people really think that every single student would pass the exam?

One option under serious consideration by the Board of Regents would leave cutoff scores at 55 on selected exams, at least temporarily. Other proposals would go further -- for example, by averaging students' scores together, rather than risking not graduating because of failing a single exam.

"If kids in honors classes are having trouble, you know other kids are going to be in trouble," said Jessica McCaffrey, 17, a 12th-grade honors student at Sewanhaka High School in Floral Park.

Jessica, you have no idea. If students in honors classes can't pass an exit exam - on which they can start as early as ninth grade, by the way - then other kids are indeed in trouble. However, it's the school's administrators whose feet should be held to the fire. No honors student should have trouble passing an exam like this, unless schools are dumbing down honors classes beyond all recognition. If parents are having to hire tutors, the schools aren't doing their jobs.

About 12% of Long Island's students failed to meet the 65 passing standard, including - surprise! - a disproportionate number of black and Hispanic students. Well, by all means, then, let's remove the higher standard and cover up the fact that LI's schools are shortchanging those students.

It's not as though the Regents exams are perfect - far from it - and some of the proposed revisions, including one for averaging scores, aren't bad. But the extreme claims that are being made here - that so many kids have testing phobias, for example, or that these tests somehow disadvantage both honors students and underprivileged kids - are obscuring the fact that these tests aren't that hard, and the passing standards aren't that high. Historically, students had to answer about 65% of the items correctly in order to gain course credit. Now, a more sophisticated scaling system is used, but the passing score is still only 55 (or a proposed 65) out of 100, which suggests that students are still required to answer only a slight majority of the items correctly (give or take some adjustment for form difficulties).

Information on the English exam can be found here. US History and Government, here.

Will New York hold fast to the increased standards (which are equal to what students have always needed to obtain a Regents-endorsed diploma, and far below what students would need to obtain college course credit for the material)? Or will they give in and keep the standard at 55%? The problems with the exams, most notably for Physics and Math, cannot be ignored - but that doesn't mean that the concept of using Regents as an exit exam is flawed beyond all belief, nor does it mean that standards cannot be raised.

High anxiety in Lodi, California

LodiNews reporter Alejandro Lazo is fretting about the fact that high schoolers in Lodi (CA) will have to pass an exit exam that contains - horrors! - items about algebra in order to graduate:

If a dirt-poor journalist has only $4.10 in nickels, dimes and quarters, and he has four fewer dimes than nickels, and twice as many quarters as dimes, how many of each type of coin does this reporter have?

This new district policy ups the ante of a state law taking effect this school year requiring students to take algebra, but not necessarily pass it. The new policy has some teachers and students in Lodi Unified worried that some seniors might not walk across the graduation stage.

Any senior who can't solve the above problem may not be able to find the stage. That counts as an algebra problem that high school seniors are supposed to be worried about? Come on. If those teachers were really worried about their students, they'd have managed to teach them the math skills necessary for the above problem at some point during their four-year tenure.

The new policy comes at a time when algebra skills are considered more and more an entry-level job skill, said John Coakley, coordinator of math at Lodi Unified...Except for the service industry, most jobs will require "at a minimum, some higher computation skills," he said.

The University of California also requires Algebra I, as will the statewide exit exam due to be implemented in 2006. Cal's DOE states that students should have been exposed to algebraic concepts by eighth grade.

Oh, but teachers are skeptical about this whole "algebra" thing:

Some math teachers and students at Lodi Unified don't think the new policy will be successful. "None of us object to the idea," said Doug Smith, head of the math department at Lodi High School. "The problem is that the students coming in are not adequately prepared to do the job."

What? You have four years to teach them basic algebra, and you're the head of the math department. So what if they didn't have pre-algebra in middle school? You can't pass the buck here, Doug. That dog won't hunt. There's no excuse for letting kids out of high school who don't understand how to use variables and algebraic equations - unless, of course, a future in the "service economy" is all you foresee for them.

For 687 students entering Algebra A, only 33 percent of students had a mastery of simple equations. "The 'mastery' level means the minimum amount of skill necessary to have success in the course," Smith said, pointing at one column of numbers.

Sounds like it's time to change the course - add remedial after-hours instruction, slow down the first six weeks, anything. Passing the buck back to the eighth-grade teachers won't do anything to fix this. At some point, some teacher has to say, THIS is where my students will begin to learn algebra.

He is particularly concerned about his special-education students."They can't graduate," he said. "One of my students is at the second-grade level, and he's in algebra because he's a 12th-grader."

Now, that's a different issue. The question then becomes - why has this kid been mainstreamed, and why should he be issued a high school diploma? He's not going to be able to hold a job; why continue with the fiction that he should be enrolled in a mainstream high school?

Tim Stutz, a science and math teacher at Liberty High School, is also worried about what the new requirements might mean for his students. "It's better to let them earn their high school diploma" without the algebra requirement, he said. "Sticking this extra requirement on them is just going to show them they failed at something again."

AARGH. What do you think is going to happen to these kids when they try to go to college or hold a job with this diploma?! You don't think they're going to be told that they're failures then? When they get rejected from UCal, or flunk out, or can't hold down a basic office job, do you really think they'll look back and say, "Well, even if Mr. Stutz couldn't teach us algebra, he sure was compassionate"?

No. They're going to be compared to other graduates, and they're going to realize that their diploma is worth nothing.

One estimate says that only 33% of California's students "graduate with a successful completion of algebra," and I can only conclude that Mr. Stutz is not interested in getting his students out of the bottom two-thirds of California's graduates.

The flunking valedictorian disappears from view

The Bridget Green story hasn't really stayed in the news like I thought it would. A valedictorian who flunks an exit exam should be a big wake-up call, and one that gets a lot of press nationwide. Instead, no newspapers other than Louisiana's Times-Picayune has covered it - although a lot of bloggers got into the act, including Joanne Jacobs, Bowl of Gumbo, and An Age Like This (who gets bonus points for doing some extra research on Louisiana's schools.)

I was hoping we'd see some more serious op-eds and calls for actions based on this. Instead, far as I can tell, all that appears to have happened is that my comments section on the original post was infested by trolls, and a bizarre letter to the editor was published on NOLA:

This young woman has suffered enough pain and humiliation from the system. Return to your year book! Not all who were voted most likely to succeed, succeeded. It is usually the B and C students who try the hardest and they're the ones who often make it big. Watch it! You may be looking at your future mayor.

Joyce M.Y. Armstead
New Orleans

Whaaa?? Since when does the possible future success of B and C students justify the miserable current failure of the valedictorian? If the valedictorian did this poorly, what makes us think the B and C students can do anything at all? What's with the "future mayor" comment?

And why is Ms. Green supposed to just "keep trying" in a system that gave her an A for not learning Algebra II? She should persist in her education, I agree, but the real "resistance" here is obviously Fortier High. And if she attends college, she's going to have to overcome the "resistance" of having attended a high school that didn't manage to teach her two year's worth of material in the four years she was there.

Obviously, both this letter writer and Fortier High are much more concerned with Ms. Green's self-esteem than with her academic achievement, and believe that enthusiastic cliches and meaningless "A"s are the way to produce that.

(And speaking of self-esteem...)

It's Florida, but it's not the FCAT

Found an interesting editorial on TCPalm online (FL) from back at the beginning of August. John Ray of PC Watch was the blogger who posted about it back then. He merely pointed it out as yet another piece of evidence that some people consider any testing at all to be politically incorrect; I thought I'd take a closer look at it.

So, test-happy Florida wants to have a statewide exam for university juniors. News flash: We already have one. It's called CLAST, the College Level Academic Skills Test.

Eh? A test of which I have not heard (or, at least, don't remember hearing of)? Hrm. Here's the FAQ (love the red buttons). It measures essay-writing skills, English language skills, reading, and mathematics. The rubrics for developing and scoring items seem straightforward, but the passing score is no help if I don't know the highest possible score. The FAQ leaves that out, as does this page. What is it with sites that give us passing scores that give no indication of how much of the material was actually mastered?

I finally found one website with the following information:

On the June 2000 exam's Reading subtest, a raw score of 26/36 was the minimum passing score (scaled 295)-this is 72.2% correct....For the Mathematics subtest, the minimum passing raw score for the June 2000 test was 34/50-68% correct. Typically, students need to answer approximately 73% of the questions correct to pass the Reading and English Language Skills subtests. They need to correctly answer approximately 68% of the questions correctly to pass the Mathematics subtest.

Okay, so that gives us a little more information. It's not a minimum-competency exam according to the standards, but let's go back to the TCPalm article for more information on the items themselves:

In their quixotic quest to re-invent the wheel, politically minded reformers let CLAST slip down the memory hole. Then again, so have a lot of college juniors. Since it debuted in 1982, CLAST has been riddled with so many exemptions, loopholes and waivers that only about 30 percent of students actually have to sit for the test. Why? Because some couldn't pass it.

"The core concepts are the same whether you're in community college or a university," Larry Abele, provost at Florida State University, points out. In support of universal use of CLAST, which tests fluency in those core concepts, Abele says, "We've paid for this benchmark. We have it. Let's use it."

Oh, so this is an exam for college students that is required for the awarding of an AA degree from a community college or for admission to upper-division status of a state university - and the items are at a 10th-grade level? And people are claiming that it's biased? Yes, against smart kids, who might fall asleep while taking it.

So now CLAST is apparently riddled with exemptions and waivers, which include:

(1) the presence of learning disabilities or failing the CLAST subtests (no joke), or
(2) meeting a specified SAT or ACT score (500 per section or 21 total, respectively), or by earning a 2.5 GPA in certain math and composition courses. Isn't a 2.5 a C+? I had to earn a higher GPA than that just to be able to stay in the dorm I chose my sophomore year. And one of the classes that allows students to exempt the CLAST is "Liberal Arts Mathematics I". Erk.

Anyway, now I think about it, I think I have written about the CLAST before - but this editorial is timely because this death by many tiny little cuts is indeed what we're seeing with the FCAT, complete with dumbed-down items and claims of bias. Bad enough that so many of Florida's seniors don't pass the 10th-grade-level FCAT; why are 20% of Florida's college students not passing an exam at that level? With a message this bad, no wonder they want to shoot the test conveying it.

Scaling woes in Maryland

Yet another Washington Post letter, from Devoted Reader Richard. The question here is, should kids in Maryland have to pass an exam to graduate from high school? High school math teacher Julie Greenberg believes the jury's still out on that question, and she does raise some disturbing points:

This month the issue [of the High School Assessment exit exam] goes to the Maryland Board of Education. Regardless of whether the HSA tests are a good idea, it's premature to set passing scores for one reason: The scores don't make sense.

Look at the algebra HSA. First, the test is a combination of multiple choice questions and questions requiring written responses, yet it gives no indication of the weight given to the problems or types of problems, and the written responses are subjectively evaluated.

Actually, the combination of multiple choice and open-ended items is the test construction method most likely to guarantee a highly-reliable test score. Most of the unreliability is likely to come from that subjectively-scored section, but the author here doesn't note whether there are clearly-defined rubrics for scoring the written components, how raters are trained, and how rater scores are combined. Granted, she might not have access to that material, but simply stating that the test contains open-ended items, or a mixture of item types, is not, in and of itself, a valid test criticism. And while it would be nice for examinees to know how different parts of the test are weighted, it's not crucial that they know.

I found some HSA Algebra items here. Some items are open-ended in the sense that item responses are not provided; students must bubble the right number into a grid. As far as scoring goes, though, there's no subjectivity involved there, and so those items aren't necessarily going to negatively impact reliability.

By the time you get to item 10, more subjectivity has been introduced into the scoring scheme - but the rubric for scoring the item is provided on a right-hand menu. To me, the rubric reads like so much educational jargon, but there is a rubric. It's not wholly subjective (more rubric information is here).

If the items are terrible, or unrelated to one another, or testing noise rather than valid constructs, then the test will be unreliable and invalid, but just knowing the items types gives us no information about whether this is the case. Thus, the author is wrong in using HSA item types as evidence that the test is flawed. Let's see what else she has to say:

Whatever raw score is produced from this less-than-straightforward process is then subjected to the statistical alchemy of scaling, which theoretically distributes scores along a range of 200 to 800 points. However, the highest algebra HSA score in the state is apparently about 585, although I can't be sure if this is the case because the state has not given the scaled scores to students or parents, only to the schools.

Hmmm. Interesting. The use of the word "alchemy" here is telling; alchemy is the long-discredited belief that one metal may be transformed into another, while scaling is based on sound mathematical principals. Sure, it might seem magical on the other end, but it's not an "inexplicable" or "mysterious" process.

Scaling and equating are often thought of as one process, probably because both are done with most large-scale standardized tests, but they're not the same thing. Scaling is essentially reassigning the number scale of a set of test items so that something other than the number-right score scale is used. SAT items are scaled to a score of 200-800, and the scaling does take into account difficulty of items. That scaling is done on a norm-referenced curve, so that an examinee's SAT scores carries with a percentile that compares that person's performance to previous SAT examinees. With scaling, test forms that are of differing lengths, and that use different items, can all be placed on the same scale.

Equating, on the other hand, is the tweaking that's done afterward to adjust differing test forms so that examinees who took the forms can be compared to one another. Although a test such as the SAT is assembled to specifications to ensure that each test comes close to being parallel to previous tests, equating is done after the fact to ensure that examinees are not penalized if they receive a test that's slightly more difficult than expected.

So, back to the Maryland test. If the highest score in Maryland was a 585 out of 800, then this could mean several things. One, that the test is very difficult, and is scaled to really spread out the top range of scorers, so that it would be very tough to get one of those high scores. It could also mean that no kid in Maryland learned much about algebra, but this teacher notes that students who managed only the high 500's on this HSA made 800s - perfect scores - on the SAT math section, and that some younger students who took the Math SAT also did much better on it than on the HSA.

That, to me, means the HSA might be more difficult than, or that its algebra content diverges wildly from, the SAT, and neither of these situations is satisfactory for a high school exit exam. The teacher, though, thinks that this exam was quite easy, but that the scaling process devalued the scores. That, to me, seems implausible, and she doesn't provide any evidence to suggest how or why that would be true. Scaling is not normally used to massively readjust scores as compensation for unexpected test ease or difficulty.

There's no logical reason to give a test that's far too easy and then scale the test in such a way that all the score are greatly lowered. However, this teacher is right to be skeptical, beause something here is not adding up (pun intended). She wants information on the scaling process; I'd want more information on the items, the passing score, the test reliability, and the score distributions as well.

So - back to the web. I dug up a very interesting, very recent report on the HSA - "High risk or high time? A critical junction in implementing Maryland's High School Assessment as a graduation requirement." The report is lengthy; I'll report more tomorrow when I've had the chance to peruse it at greater length.

Howeve, I can say that the report concurs with the letter-writer's claim, above, that while students receive percentile rankings, they haven't been given their number-right or percentage-correct scores, and they don't know how their performance compares to the standards. There's little reason for a high school exit exam to be norm-referenced; if 100% of students meet a standard, or criterion, they should pass. Thus, it's not anywhere near as useful to give a percentile ranking on this exam as it is to show a student how they compare to the standards.

The report also notes that Maryland's high school standards, which require only Algebra I and Geometry for graduation, are misaligned to the standards of Maryland's colleges, which require Algebra II and some Trigonometry. And page 43 essentially restates the concerns of the teacher above:

The normative HSA data released to date, however, raises questions: the 2002 HSA percentile and scale score data available on the MSDE website (www.msde.md.us) demands further explanation. While psychometricians and statisticians understand the science of adapting raw scores to scaled scores, and the tenuous quality of firstyear scores, how should the general public interpret the fact that the 99th percentile on the English I assessment equates to a scale score of 473 on a scale of 0 - 800? Does that relatively low score mean that, generally, Maryland students did not fare well as measured by the standards? As it stands, the public has received insufficient information regarding decisions that will profoundly impact its children.

I couldn't get the link in the quote above to work - probably not a good sign. This report suggests that something is indeed wrong with the scaling process of the HSA, because it raises so many questions about how it should be interpreted.

Exit exams are here to stay - but will they work?

More than half the public school students in the US must pass an exit exam to graduate, and despite controversies, it appears the exit exams are here to stay. A study by the Center on Education Policy concludes that the exams have led to more rigorous courses, and have helped identify struggling students - but there are downsides as well:

Diploma-driven tests have been used for years, but the 2002-03 school year was the one in which many students and parents learned why the tests are often called "high stakes." Several states withheld thousands of diplomas, or prepared to do so. In some states, tests got tougher. At times, it got messy.

In many states, 65 percent to 85 percent of students pass on their first try. But in states that provided a breakdown of data, scores were significantly lower among blacks, Hispanics, poor students, children with disabilities and those with limited English ability...

Those with limited English ability should, of course, not be passing at the same rate, not if these tests are supposed to measure mastery of English. Hence, it shouldn't be lumped in here with minority students, but it usually is. Also, the study is correct is stating that the schools can't just focus on test development, but must also focus on teacher training and helping at-risk kids before they flunk the exams. Those sorts of efforts will help reduce score gaps between groups.

Implicit, of course, in the list of requirements for making exit exams effective is that schools must be prepared to hold all students to one meaningful standard, to flunk students who don't meet that standard, to not allow grade inflation to take precedence over test results, and to stand fast against protestors who don't like an objective standard with politically-incorrect results. If schools continue to create massive loopholes, stay silent in the face of organized protest, and back down because certain groups don't perform as well, they'll make the exit exams next to useless, while still inflaming test opponents.

If schools are going to do this, they need to stick to their guns and do it right.

Avoiding the alternatives

Governor Jeb Bush has "heard concerns" about the FCAT, according to this article. What a great headline. He has "heard," but did he believe the claims of protestors that the test is inherently unfair? Did he allow protestors to continue their claims that the FCAT alone is holding thousands of kids back? Doesn't sound like it:

...faced with a boisterous crowd outside his office, Bush finally relented, first suggesting that a group of parents and students meet privately with him.

''If the principal and teacher believe that the child has achieved what the expectations are, then that child can be promoted,'' Bush said, suggesting that Wilson, as a school district employee, should be familiar with the regulations...

Heh. Apparently, the protesters who are screaming that the FCAT is holding so many kids back aren't being too forthcoming with the fact that the portfolio option can be used to pass a third-grader who flunks the FCAT. These portfolios provide alternate reading passages and items, along with other test options. But not that many teachers took advantage of this - it tended to be used only with the students with really low FCAT scores who were nonetheless reading well (according to their teachers). Only 256 bypasses were submitted to the Miami-Dade school, and 200 of them were accepted.

The remaining flunkers, we might assume, were judged by their teachers to be unable of passing the alternatives; i.e., not reading at grade level. Thus, their retention in third grade is not the testing-related injustice that the protestors would have you believe.

The valedictorian flunks out

No, despite your high hopes upon seeing the header, this is not a Blair Hornstine story - but it's just about as sad.

There is a young woman named Bridget Green. She was all set to be valedictorian of Fortier High School in New Orleans. But instead, she skipped graduation, because she's likely to be flunked out. Not because of some grading dispute or plagiarism charges, but because she flunked the math portion of the exit exam five times.

Is she just an innocent victim of a flawed standardized test? Well, while the Graduate Exit Exam (GEE) may not be perfect, its conclusion about Ms. Green's true math skills has been externally validated:

Had the GEE been the only test that gave Bridget Green so much trouble, perhaps critics of that test would have a point. But she also did poorly on the ACT, a tool that colleges nationwide use to determine a student's readiness for higher education. Ms. Green's composite score on that test was 11. According to the American College Testing Web site, that places her in the 1st percentile. Of all the 10th-, 11th- and 12th-graders who took the test within the last three years, 99 percent of them did better than Ms. Green.

Emphasis mine. This NOLA op-ed is scathing:

Just this year Ms. Green received an A in Algebra II. In most places, an A in that subject indicates the student has demonstrated mastery in factoring polynomials and dividing them, solving quadratic equations and plotting them and is adept at solving complex word problems. Obviously, Bridget Green hadn't mastered those skills or she would have passed the test...

The teachers and administrators at Fortier High have some explaining to do. If the senior with the best grades can't pass a 10th-grade test, what, if anything, are students being taught?

What, indeed. How on earth can this be? How can a valedictorian place in the lowest percentile of the ACT? That's so low as to believe that she didn't really make an effort, or just filled in the bubbles randomly.

There's a more sympathetic, and informative, article about Ms. Green here:

Green, who failed to pass the 10th-grade level test on five occasions, isn't the only student at Fortier who missed this year's ceremony. Of the 220 students who began the school year as seniors, 125 graduated in May. School officials said at least 30 had the grades to graduate but failed the test...

...Green was feeling confident when she faced the exam for the fifth time this March. Her hopes were fueled by her performance in Algebra II, where she earned an A. When she took Algebra I, she earned a C.

Why on earth did she earn a grade that high if she couldn't pass 10th-grade math items? And math courses are cumulative. Grades tend to go down as one progresses along the math continuum, not up. If Ms. Green didn't have a good grasp of Algebra I concepts, how did she manage an A in Algebra II?

Needless to say, the testing critics are out in force and ready to overlook even this most outrageous example of mis-education:

"I wish our public officials and even (the state Board of Elementary and Sceondary Education) would take a second look at the test," said C.C. Campbell-Rock, a local activist who has long fought to abolish rules that tie student promotion to test performance.

C. C. Campbell-Rock, I think you're a fool. This test is providing a massive wake-up call to a school system that inflates grades and doesn't provide its students with the educational skills they need to go on to college - and you think the tests should be abolished? Not before someone figures how a student could have the highest GPA in her senior class and not understand 10th-grade math.

A middling score on the ACT wouldn't be impossible here - I've known valedictorians and salutatorians who didn't have anywhere near the highest SAT scores in their classes - but the 1st percentile on the ACT for someone who received an A in Algebra II? No. If reality wasn't going to slap Ms. Green in the face here, it was certainly going to slap her later, when colleges refused to admit her despite her valedictorian status. Despite the claim that math is Ms. Green's "only" weak spot, her overall ACT score would have been better if that were the case (she even admits she "eked" out a passing score on the English component). It's hard to believe that she's been cheated out of a real education only in math.

How hard is the GEE? Judge for yourself. A set of 10th-grade sample items is here. Skip to page 5 for the math items. Students are required to "add signed numbers and find the largest sum", interpret graphs, interpret the most basic statistics (mean, median, mode), and use simple geometry for proportional reasoning in the multiple-choice section.

In the constructed response example given, a student is required to find figures with sides of differing lengths that have the same area and write a simple one-variable algebraic expression to find the area of a rectangle with length x.

And the valedictorian couldn't do this? And critics are telling us to take another look at this exam? I'd say we take another look at what Fortier High considers to be Algebra II work. And what about the students who had lower GPA's than Ms. Green? At least 125 of them passed the GEE, meaning they managed to overcome the obstacles of this school's dubious educational tactics.

Do I think Fortier High is unusual among Louisiana's schools? Not after reading this summary from CABL, the Council for A Better Louisiana:

A snapshot of Louisiana's economic and quality of life indicators shows that:

* A significant number of public students score below “basic” levels on academic tests and the although the rate is improving, the state has a significant dropout rate -- with nearly 16,000 high school students leaving in 2002.
* Academically, many of our public middle schools are performing poor or failing levels.
* Academically, many of our high schools are performing at poor or failing levels.
* Four out of 10 college freshmen from our public high schools must take remedial courses.

CABL also notes that the passing rate on the GEE is set at the "Approaching Basic" level, which puts the 75%+ passing rate into perspective. The higher education indicators are much more informative - 39% of Louisiana's college students are enrolled in "Non-credit Developmental Courses", which is a polite way of saying "remedial" - and the 6-year graduation rate is an appalling 39% as well.

None of these facts support the theory that this is an isolated incident - that Ms. Green is merely bad at math or nervous about taking tests - nor do they support the demand that these high-stakes tests should be abolished. Ultimately, I have to agree with this letter writer:

One thing is clear. Standardized tests like the GEE and ACT are a necessary evil. A grade of A or B at some New Orleans high schools does not accurately reflect knowledge or ability.

Wayne L. Johnson
New Orleans

The real "quagmire" - California's educational system

Pundits are nattering on about the "quagmire" in Iraq, but Pacific Research Institute for Public Policy editorial director K. Lloyd Billingsley sees one closer to home:

Data gathered as part of the federal No Child Left Behind Act found that about 70 percent of California schools fail to meet standards for yearly improvement. While shocking enough, that statistic fails to convey the depth of the problem. In California, even the best students read poorly.

The California Education Report Card, a report by the Pacific Research Institute, noted that last year only 36 percent of fourth-grade students scored at or above proficiency on the English portion of the California Standards Test...These results are among the lowest in the nation.

What could help improve the situation in California? Mr. Billingsley thinks ideas such as tougher standards, charter schools, and an exit exam could work - but all of these are opposed by California's "educators":

There is some hope in California's tough academic content standards, which emphasize core knowledge and skills...Standards and accountability, however, are under attack by teacher unions.

Charter schools are another bright spot but also under attack. Teacher unions are fighting to block a charter school for low-income, minority students at the recently closed Sacramento High School, a project backed by former NBA star Kevin Johnson and one with first-rate staffing and strong community support.

Teachers who oppose standards, unions who oppose privately-funded schools for desperate minority youth, exit exams that are dumbed down and then discounted altogether...I'm not sure even the Terminator could handle this crew.

Accountability in the Gem State

Young Idahoians are soon to be jumping through the same exit exam hurdles as kids in California, Florida, and the like. The State Accountability Commission members haven't yet set the year in which graduation exams will be implemented, but they are recommending that this type of exam be put to use:

The State Accountability Commission stopped short of setting the year for implementing the mandatory test for graduation in submitting the proposal to the board for review on Aug. 14. A public hearing was expected to be set for October with final board approval before year's end...

Under the commission proposal, students who fail the test could appeal to the local school board to take a different form of assessment to gain a diploma. But McGee called that provision a "safety net" that she did not expect to be used frequently.

I believe Ms. McGee is being rather naive about the frequency of use for that safety net, given that local educators are already raising a ruckus about the unfairness of holding all of Idaho's students accountable with this one test:

Some school superintendents welcomed the provision for appeals, but Sam Byrd, Idaho Migrant Council director, objected to any high-stakes tests because they put many Hispanic and other minority students at a disadvantage. "We're opposed to any type of test that in and of itself acts as a silver bullet," Byrd said.

Meridian School District Superintendent Christine Donnell expressed the same concern about pinning graduation on a single test..."There are some students that just don't do well on tests," Donnell said. "If we can find an alternate assessment that would measure what they've learned, I think it would be better for all concerned."
In what seems to be part of the same story (but is reported from a different location), the doomsayers have already emerged:

District 25 Curriculum Coordinator Black said she thinks the "high stakes" test will put more pressure on the students. "I don't think it's wise to make any major decisions on a child based on one test and one time," Black said. "There are just some students who don't test well."

Black said she is concerned about Native American and Hispanic students because they traditionally don't test well on standardized tests. "To stop them from graduating is a high penalty," Black said.

No, to give them a high school diploma when they can't demonstrate basic achievement in reading and mathematics is the high penalty. Again, we have here the smoke screen of educrats who want to convince us that the diploma is all that matters, not the work that went into it.

If Native American and Hispanic students are indeed performing worse on these tests, now is the time to ask why, rather than casting about for loopholes. Do they not understand the purpose of the tests? Have they no experience with practicing for exams? Are they buying into the hype that the tests are culturally biased? Or are they stuck in classes with teachers who don't believe in drilling them on basic skills?

McGee said she has been looking at past Idaho Standards Achievement Test scores and only 36 percent of the Hispanic students have passed. She said the percentage is not acceptable and no student should be left behind from getting a minimum education.

No student will be left behind when teachers improve teaching, not when tests are removed. Removing the exit exam only allows bad teaching to continue. And students are expected to have about ten chances to pass the exit exam, with the option to bank the sections they pass for use on subsequent tests. If only 36 percent of Hispanic students pass under those conditions, something is seriously wrong, and it probably isn't with the exam.

Although the date of implementation has not been set, the classic exit exam phenomenon symptoms are already evident in Idaho. The passing requirements have been set to be extremely loose, a safety net is in place for those "test-anxiety" sufferers, and the racially-focused pessimists are already issuing dark predictions about the wholesale failure of certain minority groups to pass even these extremely low requirements. It's all there. Only thing left to come are the lawsuits.

The exit exam debate

Former teacher Shirley Hickman muses about the effectiveness of exit exams in the Porterville (CA) Recorder:

Do the tests really measure what they say they measure? For example, the STAR Cat 9 test, which was given to high school students, was supposed to measure if students have met the California Standards in language arts, science, social science and math.

Students were more willing to work hard when they knew they had to pass those tests to graduate. The California High School Exit Exam may have the same effect on many students...The aim [of NCLB] is praise-worthy, but the target will only be hit if there is a quality testing program that accurately measures what students need to know.

In contrast to Ms. Hickman's optimistic statements, here's an article from one very pessimistic and dissatisfied Alaskan high school junior, Luisa Walmsley:

It bothers me that it is so easy to graduate from high school without knowing much at all. The scores necessary to pass a course can be obtained by sitting through the classes and doing a minimum amount of work. I see students doing this in many of my classes, and I have been guilty of it myself. It is easy to take advantage of the system...

The state of Alaska has sought to remedy this problem by instituting the High School Graduation Qualifying Exam, which requires students to pass tests in reading, writing and mathematics in order to receive their high school diplomas...This exam doesn't even begin to improve the situation. We spend far too much time in school to limit our learning to basic skills. An exit exam does nothing to encourage more than just that. If we must have a test, it should be one that motivates students to seek more knowledge.

While Ms. Hickman seems to be dealing with students who rebel at learning just the basics, Ms. Walmsley thinks that the exams are too basic, and should be more difficult in order to be motivating. The exams weren't really designed to motivate students to learn more than the basic skills, though, so I can see why they'd be frustrating for students who want a challenge.

"The class that got away"

Joanne Jacobs noticed a fine article in the Sacramento Bee about the postponed California exit exam. Columnist Daniel Weintraub correctly judges what the fuss about exit exams are all about - when bad teaching on part of a school leads to negative consequences for the students, it's hard for the school to defend its choices:

The alternatives [to postponing the exit exam] the board considered were worse than delay. One was to make the test easier. The other was to lower the passing score [These could be considered the same thing]. Instead, other than dropping one essay requirement to shorten from two days to one the time it takes to administer the language portion of the exam, the board stayed the course. This is good news.

But fans of reform should not rest easy. The test's opponents will see this decision not as a momentary pause but as a crack in the door. They will continue to push to weaken accountability because they do not believe in it....The high school exit exam has become their primary target because, of all the tests the state administers, this is the only one that truly counts...It means something, and that meaning makes it dangerous. When kids fail, people start asking questions. Did the child try hard enough? Did the parents push hard enough? Did the school provide the proper coursework and materials? Was the teaching sufficient?

All of those questions are uncomfortable for a segment of the education establishment that would rather fuzz things up, pat kids on the head for making a good try and send them on their way with no concrete sense of what they have taken with them after 13 years of seat time in the public schools...

Mr. Weintraub also correctly picks up on the recent eduational, political, and cultural philosophies which say that no child can ever fail. The exit exam, by definition, is going to identify those who fail, and unless it is dumbed down beyond recognition, some kids in even the best schools are going to fail. Unless we're willing to say to those kids, "We gave you the best opportunity, but for whatever reason, you didn't perform up to the standard" and refuse to issue a diploma, our support of accountability is empty talk.

Mr. Weintraub estimates that perhaps 80% of California's seniors would have eventually passed the exit exam. Are we truly comfortable with flunking the other 20%? And are we willing to defend the results even if the 20% group contains disproportionately large numbers of minority students? That's the first issue the anti-testing crowd will attack - indeed, it's often the crux of their claims that such tests aren't "fair" - and exit exam supporters should be ready for it.

California seniors are off the hook

From the July 8th Mercury News (OK, so I'm a little behind here) - Exit Exam Likely to Be Postponed For Two Years:

California's high school seniors have been told since they were in eighth grade that they would be the first class to have to pass an exit exam to get a diploma. Now, the State Board of Education appears poised to deliver a revised message: You're off the hook.

The board is expected to vote Wednesday to delay enforcing the high school exit exam requirement for at least two years. It is a move welcomed by some educators who want more time to get students ready to tackle the test but is seen by others as a setback in the state's aggressive push to establish stricter academic standards.

On July 9th, the board did indeed vote - unanimously - to delay the exit exam for two years. Seems they haven't quite figured out how to best tackle the high failure rate which indicates that California's high schools aren't doing as good a job as they should:

Under legislation passed in 1999, the Class of 2004 -- which had 428,117 students enrolled statewide as of October -- was supposed to be the first that would have to pass the exam to graduate. Students can take the test up to eight times during their high school years.

But as of January, only 62 percent of students in the Class of 2004 had passed the math section of the exam, which covers algebra as well as some statistics, geometry and probability. Eighty-one percent had passed the English language arts section

That's just a little over half of the students passing the math section, on an exam that allows eight tries. And the board thinks this situation will be rectified within two years? I'm beginning to think that the board might have wanted more information about how their students are doing when they instituted this exam, but they didn't want, or expect, this stunning truth about the students' poor performance.

The letters to the editor have begun arriving at the Merc (see here as well) - and some are from students opposed to the delay:

California is one of the lowest ranking states in terms of education -- and now the state has given students a break and will let us take it easy. The state should pressure students to work hard to pass the test.

California is already one of the weakest states in the nation in terms of education. With the likely postponement of the high school exit exam (Page 1A, July 8), this view will be reinforced. There is no reason to delay this graduation requirement. I am a student at Andrew Hill High School and a member of the Class of 2004. When I took the exit exam, I found it to be simple. Its level of difficulty pales in comparison to the SAT, or even to the standardized tests proctored by the state each year.

Would I be rude to suggest that perhaps the students who are in favor of dropping the exam, which measures skills at the 10th-grade level, are perhaps not literate enough to write letters to the Merc supporting the board's decision? Yes, that would be rude. Forget I said anything.

Another reader, in fact, suggests dumbing the test down even more, although she defines it as "aligning the test" with the real world:

As long as the state Board of Education is postponing the high school exit exam, it should rewrite it to test only skills needed for survival in the real world, and eliminate the college-prep questions...

Given the prices, which is cheaper -- one 28-ounce bottle of ketchup or two 14-ounce bottles? Given a bus map and schedule, when and where should you wait for the bus? Given the descriptions of three different car loans, which offers the better deal? Given a blank job application form, fill it out.

Educrats already scream that our tests only measure basic skills, and fail to measure the much-touted "higher-order thinking" and such intangibles as creativity and intuition. Can you imagine the outcry if the board operationalized an exit exam that as much as admitted that California's students are incapable of understanding literature, art, science, and higher maths, and can only be trained to get through life's daily errands? If "teaching to the test" is such a concern when the test measures basic educational skills, I shudder to think of the backlash that would occur if the test measured only basic life coping skills.

The Merc had this to say about the board's decision:

...It's a wise move for the state board of education, as it's expected to do today, to delay imposing that graduation requirement for two years.

First, not all schools offered the appropriate classes and subjects because state content standards weren't in place early enough. Second, the state was likely facing a costly lawsuit had it forced unprepared students to pass the test.

So the state shouldn't do anything that might generate a lawsuit, even if the lawsuit is filed for the wrong reasons? And is two years really enough time to put the content standards in place? The legislation dictating that the Class of '04 should pass the exit exam was passed in 1999. If five years hasn't been enough to get everyone ready, will another two really make a difference? And does the board have any suggestions as to how the schools should fix things in two years that they couldn't fix in twice that amount of time?

Surely the board is aware that they will be sued, regardless, by the parents of some students who feel they deserve to pass, but don't. Given that some students will always fail, unless the test is dumbed down beyond all recognition, fear of lawsuits shouldn't be a legitimate reason for postponement. If anything, I feel that the the board's unwillingness to defend the test for use this year will make the test more susceptible to lawsuits in the future, not less.

As one superintendent aptly puts it:

"This is a giant PR mess," said Bill Kugler, deputy superintendent in the East Side Union High School District in San Jose. "We have been telling kids since they entered high school: 'You are the ones and you need to be ready.' Now we have to say, 'You don't have to do that.' It's a credibility problem."

Indeed.

A "special avenue" for Mass. students

For the first time this year, the MCAS is required for a high school diploma in Massachusetts, and already a Democratic state senator has inserted an amendment into the most recent state budget which would allows special-needs students who don't pass the MCAS to receive a high-school diploma through a "special avenue". Currently, those that don't pass the MCAS must accept the "certificate of attainment" alternative, but that may soon change:

Spearheaded by state Sen. Cynthia Creem, D-Newton, the amendment targets the state Department of Education's decision to award "certificates of attainment" to special education students who meet their high schools' requirements for graduating but failed the MCAS...

The appeal, which must be filed by the school superintendent at the request of a parent, will have to meet several criteria. For example, there must be proof that local graduation requirements have been met. A recommendation from the student's individualized education plan team that he or she is ready to graduate is another requirement.

This plan appears to have some checks and balances to prevent abuse of the system, but it's also likely to be used by the anti-testing crowd as one more piece of evidence that standardized tests are "unfair".

Can graduation exams be "refined"?

The Education Gadfly warns against "the siren's song" of believing that graduation standards and exit exams should be "refined" with more flexible standards. He notes that the real problem with those "heart-wrenching tales of 'B' students" who flunk graduation exams is that the schools have either inflated the grades or failed to teach the necessary content:

...these exit exams are pitched at a rather modest level and offer students multiple chances to pass, which leads to an entirely different explanation: perhaps a nontrivial number of students actually lack mastery of essential knowledge and skills. If "B" students are failing, either their school grades are too high or they have not learned basic content...

...Standards-based reform is alluring because it promises that all graduates will master critical knowledge and skills. Setting bona fide performance standards makes it inevitable that some students (and schools) will fail to meet them. This poses a daunting political challenge in a democratic society where those who fail have specific incentives to challenge the legitimacy of the system...

He also notes that the testing critics will soon find themselves in the "unenviable", and presumably untenable, position of attacking established standards that are good indicators of how well our schools are teaching and how well our students are learning. I believe he gives testing opponents too much credit, though, in this assumption that they will shy away from this approach. Too many test critics, such as the group in Florida, have bought into the "all tests are biased/racist/sexist" mindset, and they believe that certain kids should be exempt from the objective standards measured by such exams.

Joanne Jacobs has more on this.