Howdy, folks! Starting on April 6, 2008, I will be putting up a "testing roundup" post every weekend at Joanne Jacob's site. Swing on by and put in your two cents!
The first post is here.
UPDATE: And by the way, I have to point out this article in the NYTimes about psychometricians, which many of you probably saw two years ago. I just love that it came out on - yes - the day before my wedding day. Which is why I missed it at the time.
Is this good news or bad news for Indiana? You be the judge:
More than half of Indiana's seventh-graders passed the state's mandatory science exam, administered to that age group en masse for the first time last fall. Fifty-two percent of the state's 80,863 seventh-grade students passed the science assessment, a new section in the annual Indiana Statewide Testing for Educational Progress-Plus exam. That left 47 percent below the benchmark, said Mary Tiede Wilhelmus, spokeswoman for the Indiana Department of Education; 1 percent of the results were unscoreable."We certainly know we need to do better, but this is a starting point," said Tiede Wilhelmus. Two areas that gave kids trouble, she said, were sections on the nature of science and technology -- which included scientific investigation -- and one focusing on the physical universe.
Indianapolis schools fared much worse, with only 18% passing the exam. A sample exam is here. It's pretty open-ended. I had to guess on the first one - I remembered the orbits as being circular rather than elliptical, and that is one of the correct answers.
One suburban Philly teacher leads tai chi classes to help prepare kids for the state standardized exams:
A Centennial teacher is utilizing the ancient martial art of Tai Chi Chuan to help his students get ready for the state's standardized mathematics and reading tests.“I want you to concentrate. Think about what you are doing. Breathe in and breathe out,” Joseph Pisacano, a fifth-grade teacher at Everett A. McDonald Elementary School in Warminster, said Thursday morning as he and the students made smooth, circular motions with their arms and hands.
The class went through the Tai Chi relaxation exercises as calm, soothing music played quietly in the background.
“Think about how wonderful you are going to do on the test and how easy it's going to be. Think about all that you have learned this year. Take all of that, and use it,” the teacher said as the class wrapped up the exercise session by sitting in a meditative position on the floor.
As long as he's making sure to actually teach the material along with some relaxation exercises, I'm all for it. This approach is worlds better than the chicken-little squawks you see from some educators who are so stressed out about testing that they end up stressing out students as well.
The idea of standardized testing in college is still gaining attention - most of it negative:
A parade of college presidents will appear before a federal higher-education commission meeting in Boston tomorrow, and early signs suggest it will be a lively, even contentious scene. Texas businessman Charles Miller, chairman of the commission appointed by Education Secretary Margaret Spellings, has made waves by suggesting that some kind of standardized testing would help measure whether college students are taught well. There is no formal proposal yet, and Miller has stressed in press interviews that there would be no single test for every school. Still, the idea has alarmed many educators.Susan Hockfield of MIT, who had lunch with the Globe editorial board last week, didn't mince words when asked about the testing notion. ''I think it's a terrible idea," said Hockfield, who is scheduled to testify tomorrow. ''Higher education needs help, but what is really broken is K-12 education. We need more high school graduates who can understand and do math."
In other words, there's a problem, but it can't be fixed in college. I agree with Dr. Hockfield, but I wonder if she knows how strong the opposition can be to "fixing" anything in the K-12 system:
At School Without Walls and two other high schools where I am a guest teacher -- Wilson High School in the District and Bethesda-Chevy Chase High School in lower Montgomery County -- I have never given a test. I respect my students too much to demean them with exercises in fake knowledge.Tests represent fear-based learning, the opposite of learning based on desire. Frightened and fretting with pre-test jitters, students stuff their minds with information they disgorge on exam sheets and sweat out the results. I know of no meaningful evidence that acing tests has anything to do with students' character development or whether their natural instincts for idealism or altruism are nurtured.
I have large amounts of evidence that tests promote the opposite: character defects.
Good luck convincing this teacher that reading and math basic skills can and should be assessed. She's much more concerned about "idealism" and "altruism." How far do those get one at MIT, I wonder? As Betsy's Page notes, this teacher teaches a class on peace. Most of his students love him, it seems - and wouldn't you, if testing was banned? - but not all:
At Bethesda-Chevy Chase, Peace Studies is taught by Colman McCarthy, a former Washington Post reporter and founder and president of the Center for Teaching Peace. Though the course is taught at seven other Montgomery County high schools, some say B-CC's is perhaps the most personal and ideological of the offerings because McCarthy makes no effort to disguise his opposition to war, violence and animal testing.Saraf and Avishek Panth, also 17, acknowledge that with the exception of one lecture they sat in on this month, most of what they know about the course has come from friends and acquaintances who have taken the class. But, they said, those discussions, coupled with research they have done on McCarthy's background, have convinced them that their school should not continue to offer Peace Studies unless significant changes are made. This is not an ideological debate, they said. Rather, what bothers them the most is that McCarthy offers students only one perspective.
Of course he does. This is a crusade for him. One that doesn't involve anything as nasty and dehumanizing as testing (and that's even funnier than "high comedy," according to one Devoted Reader who forwarded the link). After reading his WaPo diatribe about testing, it's hard to believe that he actually is "welcoming of conservative dissention" on any topic.
When the stakes are high, do the results make sense? Jay Greene, Marcus Winters, and Greg Forster address the issue:
Several objections have been raised against using standardized testing for accountability purposes. Most concerns about high-stakes testing revolve around the adverse incentives created by the tests. Some have worried that pressures to produce gains in test scores have led to poor test designs or questionable revisions in test designs that exaggerate student achievement...Others have written that instead of teaching generally useful skills, teachers are teaching skills that are unique only to a particular test...Still others have directly questioned the integrity of those administering and scoring the high-stakes tests, suggesting that cheating has produced much of the claimed rise in student achievement on such exams..Most of these criticisms fail to withstand scrutiny...This study differs from other analyses in that it focuses on the comparison of school-level results on high-stakes tests and commercially designed low-stakes tests. By focusing on school-level results we are comparing test results from the same or similar students, reducing the danger that population differences may hinder the comparison. Examining school-level results also allows for a more precise correlation of the different kinds of test results than is possible by looking only at state-level results, which provide fewer observations for analysis...
The conclusion? That, within a school, correlations between high- and low-stakes tests tend to be large and postive:
The finding that high- and low-stakes tests produce very similar score level results tells us that the stakes of the tests do not distort information about the general level at which students are performing. If high-stakes testing is only being used to assure that students can perform at certain academic levels, then the results of those high-stakes tests appear to be reliable policy tools. The generally strong correlations between score levels on high- and low-stakes tests in all the school systems we examined suggest that teaching to the test, cheating, or other manipulations are not causing high-stakes tests to produce results that look very different from tests where there are no incentives for distortion.
Maine is now testing its third-graders regularly, and teachers are having to walk a fine line of test prep and emotional support:
They want the students to take the tests seriously, because they're used by the state and federal government to measure whether schools are teaching students math and reading effectively. But they don't want to pressure young students new to the game of high-stakes testing."We tell them to just do their best," said Hall-Dale Elementary School third-grade teacher Maureen Mathews. "You want to them to know it's important. But you don't want to make them nervous."
Love the photo of one neophyte examinee, who appears to be a direct descendant of Dame Judi Dench.
Critics are howling about ten-buck-an-hour temps grading FCAT essays:
Critics were fuming Friday after learning the FCAT -- the standardized test that will leave a permanent mark on the academic future of thousands of Florida students -- will be graded by $10-an-hour temporary workers who are required only to have a week's training and a bachelor's degree."It's just incredible to me that after all of the pressure that is placed on me to maintain my teaching credentials, the countless hours spent in workshops, and then they turn around and hand these tests off to be scored by a bunch of temps," said David Worrell, president of the Leon Classroom Teachers Association. "It's just insulting."
The DOE says that many of the workers have teaching experience and are very familiar with the exams. Keeping permanent full-time graders would certainly up the costs of the exam. Others say that criticizing the temps misses the point:
Of the many legitimate concerns raised about the FCAT over the years, this is the least of them. Temps are used to grade other important exams, such as the ACT college entrance exam, and the FCAT graders will have bachelor's degrees and training. Those grading essays will handle questions that relate to their college degree. Essays are to be graded twice, and possibly a third time.The real problem is that, under Gov. Jeb Bush, the FCAT has been wielded like a cudgel in the evaluation of school quality - and now, teacher quality.
In other words, they're fine with how the essays are graded; they're just not happy with the FCAT use as a whole.
Hey, look, it's a foolproof method for making college educations cheaper!
Oh, wait, that's not what they meant. Never mind. I'll just point out that the author is against college-level standardized tests, in part because, "achievements in math and science will speak for themselves." Couldn't that theory be used as an argument to test everyone who doesn't take those kind of courses in college?
For more than 20 years, FairTest, a small nonprofit group headquartered on the second floor of an old house here, has been the No. 1 critic of America's big testing companies and their standardized tests. In 1987, when FairTest began publishing its list of colleges that did not require applicants to submit SAT's, there were 51; today there are 730, including Holy Cross, Bowdoin, Bates, Mount Holyoke and Muhlenberg....for all FairTest's impact, its days may be numbered. Never before has standardized testing so dominated American public education, thanks to the 2002 federal No Child Left Behind Law. Every child from grade 3 to high school must now take state tests. And the Bush administration is considering extending those tests to colleges.
"With N.C.L.B., a lot of people feel the debate is over," said Monty Neill, director of FairTest, officially the National Center for Fair and Open Testing. "The attitude seems to be, 'Testing is so pervasive, what's the point?' " Support from foundations has virtually dried up and individual donations have not made up the difference. "Our board has seriously discussed whether to fold the operation," Mr. Neill said.
I find this a pretty revealing comment. There's always a need for testing to be scrutinized, for tests to be evaluated, and for the public to be informed. But I've always sensed that FairTest's commentary was always anti-any-testing, not pro-good-testing. Now that testing is so pervasive, it's not helpful to bash tests rather than inform the public. ETS's president seems to agree:
Kurt Landgraf, the president of the testing service, which administers the SAT, wrote in an e-mail message: "Perhaps if they had been more attuned to the public's support for using tests to help teachers teach and students learn, then they might have had wider support."
Further along in the article, I don't quite get the point of NYT reporter Michael Winerip listing this as though it's a bombshell:
In a recent newsletter, FairTest printed an analysis of SAT results, using, and crediting, College Board research showing the direct correlation between family income and SAT scores. For every extra $10,000 a family earns, children's combined math and verbal scores go up 12 to 31 points. So children whose parents earn $50,000 score better on average (a combined 996 SAT) than students from families who earn $40,000 (967) but worse than students from families who earn $60,000 (1014).
Okay, who alive today doesn't know that kids with more money tend to have more educational advantages? It makes sense to me that kids from wealthier homes do better on all educational indices; if they didn't do better on the SAT, parents would question the efficacy of private schools and tutoring. Why this is being mentioned here as though it's surprising knowledge - or a valid test criticism - is beyond me.
At the same time, correlation doesn't equal causation. Just because A and B are correlated, that doesn't mean A causes B. B might cause A, or some C could be causing both to happen. Smarter parents might make more money, and their kids get both the nature and nurture benefits. We are trying to close the gap by offering all students better opportunities, but a test that doesn't reflect when kids know more material, either by virtue of schooling or parental largesse, is a pretty useless test.
Jay Mathews offers a spirited defense of a process much maligned in the education world - "teaching to the test":
When we say "teaching to the test," we should acknowledge that we are usually not talking about those drill fests. Rather, we often use the phrase to refer to any course that prepares students for one of the annual state assessment exams required under the No Child Left Behind Act. For reasons that escape me, we never say a teacher is "teaching to the test" if she's using a test she wrote herself. We share the teacher's view that what she is doing is helping her students learn the material, not ace the test. But if she is preparing the class for an exam written by some outsider, the thinking goes, then she must be forced to adhere to someone else's views on teaching and thus is likely to present the material too quickly, too thinly, too prescriptively, too joylessly -- add your own favorite unattractive adverb......Conversations about this would go more smoothly if we didn't have such distorted views of what teaching to the test means. We might instead turn the discussion to what methods of instruction work best or how much time our children should spend studying.
The more-pay-for-higher-scores plan in Florida has passed the Board of Education approval stage:
The Florida Board of Education unanimously approved a plan Tuesday that will give some teachers bonuses based solely on their students' performance on standardized tests.As early as next year, the plan will award the top 10 percent of teachers in each school district a 5 percent bonus based on learning gains shown on the Florida Comprehensive Assessment Test.
If districts want to reward more teachers, they can. But there may not be state funding for it, officials warn. The plan also will require the state to create exams or other assessments in every subject not covered by the FCAT.
Whew. Possibly no funding and definitely more testing? Gee, wonder if this plan is causing any controversy. (Obviously, I'm being sarcastic here.)
Florida Education Commissioner John Winn wants to get serious - about tying teacher bonuses to FCAT scores:
In Miami-Dade and Broward counties, teachers could earn bonuses ranging from $1,710 to $4,150 per year on salaries that range from $34,200 to $83,070.The policy would add another set of consequences to Florida's high-stakes accountability system, which already determines school grades, high-school graduation and whether students can progress from third to fourth grade.
Winn said the Effective Compensation plan, which he dubbed E-Comp, would encourage teacher improvement, reward excellence and bolster recruitment and retention.
The critics, they disagree:
E-Comp would replace existing bonus systems in Miami-Dade and Broward, both of which Winn said were unacceptable.In Miami-Dade, a 5 percent bonus is given to all teachers at 28 schools that have the largest gains in FCAT reading and math scores. UTD President Karen Aronowitz said that system is fairest because many teachers contribute to a student's success. ''When we send firefighters to a fire, do we pay them differently based on who handles the hose?'' she asked.
No, but we do tend to get rid of those who don't pick up their end of the hose, especially if the end result is a house burnt down to the ground. If the public were secure in the knowledge that bad teachers would not only not share in school-wide compensation, but would get dismissed to boot, I'm not sure that merit pay for especially-good teachers would even be an issue.
The new GRE grows ever closer:
According to the ETS Web site, the changes will better gauge students' preparation for graduate school by measuring general academic skills with more precision than in the past. A single 30-minute verbal section will be changed to two 40-minute sections. Sections on analogies and antonyms will be removed, while new sentence equivalence questions will be introduced and critical reading sections will be expanded. Quantitative reasoning -- lengthened from one 45-minute section to two 40-minute sections -- will include less geometry and more data interpretation and word problems. The test will be graded on a scale of 120-179, as opposed to the current 200-800 scale.
Some students resent the newer, longer length:
...Sam Penziner '07, who is also planning to attend graduate school, said he thinks the longer exam will measure test-taking stamina rather than skill. "Making the test longer emphasizes factors like endurance and stress that affect performance," Penziner said.
And graduate school doesn't require endurance and good stress-coping strategies?
University of Arkansas at Fayetteville’s education researchers doubt that standardized test scores are the best indicators of school district performance:
The statistical analysis — which will be presented today to the Arkansas Association of Educational Administrators — concluded that Arkansas students are achieving slightly better than the nation as a whole. And several Arkansas districts not typically recognized for their academic excellence top the scale of high performers...“The School Performance Index in Arkansas” takes into account student demographics and the levels of affluence and education in a community. It predicts what student achievement in a school or district should be on the basis of those factors and then compares those projected achievement levels to actual standardized test results.
So, if I understand this, they're predicting how well a school should do based on various demographic and SES levels, and then comparing those predictions to the real test scores. They're concluding that raw test scores shouldn't be used to compare schools, but instead should be adjusted to show how well the school is doing given all these predictor variables, so that schools with students who are predicted to do poorly should not be considered bad schools if they produce mediocre test scores. I'm not sure I agree with that conclusion.
One interesting side finding:
The school analysis, which Greene said could be further refined by the state, showed that school performance on the Iowa Test is “partially” affected by the level of household income, educational attainment of adults, and the percentage of married families in a district. The scores are “substanially affected” by the percentages of black students and students who qualify for reduced meal prices at a school.In contrast, the study concluded that school performance is not affected by the size of a school or a district or by the amount of money spent in a district.
Gee, wonder why the headline for this article wasn't, "Schools don't need more money to perform better?"
A UMass-Amherst research claims the Connecticut Academic Performance Test is extremely predictive of college success - even more so than the SAT:
Stephen Coelen, a researcher from the University of Massachusetts at Amherst, tracked 32,653 members of the Class of 1998, comparing how well they did as sophomores on CAPT to how many applied to, enrolled in and did well in college. On every measure, he found the higher the CAPT score, the more students were likely to go to college, avoid remedial courses in college, get higher grade point averages in college and graduate.When matched against SATs — the College Board exam students take to predict college success, Coelen said both exams helped explain student success in college. Of the two, Coelen said CAPT "was always correct. SAT was not always correct.
Interesting. The "going to college" part could, I think, be affected by the possibility that those who score high on CAPT in 10th grade spend the next couple of years being groomed by teachers for college. If their CAPT scores affect their high school class placement or treatment in any way, then it wouldn't be surprising that CAPT would correlate with college attendence - it would be one of the predictors of it.
Interesting also to see that the CAPT apparently has a high positive correlation with college grades, but given the outcry we hear these days about grade inflation, one wonders if this is really a positive thing about the exam.
Commissioner of Higher Education Valerie Lewis said the study proves the value of CAPT in predicting college success and should be recognized by college admission staffs as a valuable piece of information when they admit students.Lewis also found it startling that 10 percent of students who score very high on CAPT never show up in college. That means some talent is going untapped and underdeveloped...
I find 10 percent startlingly low We're not being told what "very high" means, nor do we know the shape of the distribution. The study was composed of around 32,000 kids; if "very high" means the top 5% of scorers, we're talking about less than 200 smart kids from Connecticut in that graduating year who passed on college. I would think that family issues, financial issues, health issues, and lifestyle issues would affect that preclude college would affect at least 10 percent, maybe more. These are people, not automatons, and if they were that smart, they may have well decided that they wanted to do something other than pay thousands of dollars a year for additional education.
Third- through eighth-graders in New York must now take state standardized exams, and for three of those grades, the tests will determine promotion:
Today, grades three, four and five had a multiple choice test. Tomorrow the same grades will listen to stories, and then write about what they've heard. On Thursday, 4th graders only will be asked to read a passage and then write about it.This year marks the end of citywide tests and the start of statewide testing for all students in grades three through eight because of the federal law 'No Child Left Behind.' Also this year, state standards are higher.
You can look through the PowerPoint presentation here that gives an overview of the exams. Core curricula in English and Math are also available. A sample exam for Grade 3 looks pretty simple
More testing might be on the way in Florida - and that could be a good thing:
Florida high school students may someday have to take end-of-grade tests in history, literature, biology and other key subjects -- possibly in addition to the FCAT. Members of a state task force on high school reform are suggesting the tests as a way to make sure students are really learning what the state says they are supposed to learn...New York and Texas already use similar tests, and some Florida school districts have adopted them, too. ''We're looking for something that's going to help students achieve at a higher rate, not looking to multiply the number of tests out there,'' said state education Commissioner John Winn. "But an end-of-course test is a way to have some consistency in proficiency level.''
Chett at ReformK12 just pointed me to a nifty online list of psychometric resources - The Statistics Resources for the Center for Research, Evaluation, and Program Development, Boston Graduate School of Psychoanalysis. I'd recommend the books on IRT, except that I haven't read them - I'm old school and rely on the classic Hambleton & Swaminathan IRT bible. I actually rely on a lot of "bibles" that aren't listed here, like Educational Measurement and Psychometric Theory. A tad ironic that I mainly read these old books, since I'm the book review editor for a measurement journal, but oh well.
I also would be remiss if I did not mention the IRT software packages BILOG and *cough*my-advisor-created-this*cough* MULTILOG. Even I can't claim they're easy to use - unless you're fluent in FORTRAN, and a frightening percentage of psychometricians are - but I see them get a lot of use in testing organizations.
The tweetles, trills, blats, waa-waas, and crashes of Florida's band students may soon be measurable via a standardized exam:
It won't resonate as loudly as the FCAT, but Florida schoolchildren could soon face a new test of how well they're learning music. The test is being developed by two statewide groups of music teachers who see it as a way of reinforcing the importance of music in a well-rounded education and measuring how well it is being taught around Florida."Music educators are accountable every time their students step up on the stage, but they felt they needed a more formal way (to measure their learning)," said Timothy Brophy, a University of Florida assistant professor of music education, in a telephone interview from Gainesville...
The first phase of the music tests could begin in fourth and eighth grades as soon as 2007 if all goes as planned. It will consist of a paper-and-pencil test in which pupils respond to a series of questions recorded on a compact disc, including musical passages.
Brophy said the next two phases of the testing program are expected to include actual musical performances that would be recorded and compared with a standard for the appropriate grade level.
Tests for music students in other parts of the country should be modified to conform to local standards and tastes. For example, band geeks in South Dakota should be tested on their awareness of just how seductive the saxophone can be.
Testing marches onwards, as 23 states expand their testing programs:
Forty-eight states and the District of Columbia will give standards-based tests in reading and mathematics in grades 3-8 and at least once in high school this school year, as required by the nearly 4-year-old federal law, according to a survey by the Editorial Projects in Education Research Center.The holdouts are Iowa and Nebraska. Districts in Iowa give the Iowa Tests of Basic Skills, a national test not designed to measure state or local content standards, while districts in Nebraska craft their own tests, except for a state writing exam.
In devising the new tests, most states have defied predictions and chosen to go beyond multiple-choice items, by including questions that ask students to construct their own responses.
Hoo boy. Could be good, could be very problematic (and expensive) to score. The entire article is worth a read.
Opinion Journal's Best of the Web wonders if there's been an interesting spin on test scores in Michigan. Here's the bad news:
Michigan African-American fourth- and eighth-graders scored much worse in reading and math than African-American students in the United States as a whole, according to national test results released Wednesday. The 2005 National Assessment of Educational Progress results show significant improvement in math. But the test results show little improvement in reading scores overall since 1992.
Now, the good news:
The good news is that the gap between black and white students' scores in math and reading within Michigan has decreased.
BotW assumes that the gap could not have closed without white students performing worse, and this article (interestingly) does not mention white student performance. But it does say that black students have improved in math and stayed level in reading. The "reverse Lake Wobegon" effect might be happening for reading, but not necessarily for math. If the black students have improved in math while the white students stayed even, and stayed level in reading while the white students declined slightly, we'd see this pattern.
I couldn't find change information by ethnic group and state in the 2005 report, but if you manage to see that anywhere, let me know.
The GRE, which made waves in 1993 by transforming into a computer-adaptive exam, is being revamped again:
Although the test will still include sections on verbal reasoning, quantitative reasoning and analytical writing, every section is being revised, and the test lengthened to about four hours, from two and a half hours. About 500,000 students, 20 percent to 25 percent of them foreigners, take the general G.R.E. each year. E.T.S., which administers the test, also offers subject-matter tests in such fields as biology, mathematics and physics, but those tests, taken by far fewer students, are not being changed.To enhance security, every question on the new exams will be used only once, and the test will start at different times in different time zones, so students who have finished cannot pass on questions to those in different zones...
As of next year, the test will no longer be "computer adaptive," with test-takers getting questions tailored to their performance on previous questions, so that each gets challenging questions that provide a clear picture of what they can do. Instead, every student taking the test on a particular day will get the same questions, and those questions will not be reused.
Computer-adaptive tests, as ETS and others have discovered, require item reuse and enormous item pools to prevent any one item from being exposed too often. Good GRE items are not cheap nor easy to come by, and this change addresses security questions and helps to ensure the relationship between the items and the construct by reverting back to the one-use-per-item model. Of course, removing the adaptive algorithm also involves lengthening the exam, because the range in item difficulties for each form once again must be wide enough to adequately assess the geniuses and those who should probably strike graduate school off their "to-do" lists.
Update: Ah, the joys of knowing so many experts. I think this post was up for about two seconds when another psychometrician emailed me to remind me about the CAT version of the ASVAB, the Armed Services Vocational Aptitude Battery. That actually came out before the GRE-CAT, in the late 1980's, and I certainly should have remembered that, considering the truly phenomenal and exhaustive primer that exists on the subject, Computerized Adaptive Testing: From Inquiry to Operation. I can't recommend that book highly enough for anyone who wants to learn more about developing, testing, and implementing a computer-adaptive test.
The advent of computer-based testing is worrisome to would-be med students:
As if pre-meds did not already have enough to worry about, recent changes to the MCAT have some pre-med students worried about more than just mastering its content.The Association of American Medical Colleges announced recently in a press release that it will convert the MCAT to a computer-based format within the next two years, a move that will force both students and test-prep companies to sharpen their strategies for tackling the test, rather than their pencils.
The paper format of the test will be administered through 2006, though trial versions of the computer-based test will be given at the August 2006 testing date.
There are many advantages to computer-based tests; three of the biggest ones are the shorter test length (I'm assuming here the new MCAT is adaptive as well as computerized), more testing opportunities, and shorter score report turnaround time. Based on my moderate experience with CBTs, I think that some of the fears of future examinees will turn out to be unfounded:
"With the computer-based test you can't underline passages and put notes next to text, so it's hard to map out the progression of a passage. You can't keep track of it when you have to keep switching from a computer to your notes," said Patrick Wiita, a fourth-year microbiology, immunology and molecular genetics student who has taken the MCAT.
Underlining text onscreen is a fairly simple feature to add, if it isn't already included. And it's also possible to allow a typing area on-screen for notes or comments, for those who don't feel comfortable using scratch paper.
There's also concern about computer glitches, Wiita said. "Who knows what errors could occur in programming. It's the same reason they haven't switched to online voting. If there's an error, there's no paper trail," Wiita said.
Why on earth would you want a paper trail when you can capture every keystroke? Certainly, big programming errors can occur - but paper tests can vanish in transit just as easily. If anything, the software that processes computerized testing errors tend to provide more information to the testing companies than paper errors. There will be exact records of when screens go down, where examinees were when the error occurred - and most CBT providers that I've seen have little problem restarting a test on the correct screen after an error occurs. Any testing company thinking about CBT administrations correctly places a lot of focus on error prevention and recovery.
Mustafa said new testing formats create a lot of work for test-prep companies, especially research into how students feel about the test. "We did a survey on 4,000 (students) to see how they're feeling, what they think. Eighty-two percent said they would do worse on a computer-based test," Mustafa said.
I've yet to see any research showing that examinees in fact do worse as a whole on computer-based tests. Some subgroups, in fact, do better in certain construct areas. Regardless, this is an easy enough question to answer; matched examinee groups can be compared with respect to P&P and CBT scores.
Of course, examinees may experience a feeling of doing worse if they're switching from a P&P test to an computer-adaptive test (or CAT), since the items will be tailored to their ability level and they'll see more of what they consider to be hard items. But CAT scoring scheme takes item difficulty into account, so that two examinees could miss the same number of items but end up with different scores, based on the difficulty of items that were answered correctly.
CATs have been operational in a high-stakes, large-scale environment since 1993 (when ETS pioneered the GRE-CAT), so there's quite a bit of theoretical and operational research out there to guide the development and refinement of the MCAT-CAT. Good luck to them.
An op-ed in the New York Sun and the folks at The Daily Howler focus on a recent article praising Wake County (NC) for raising test scores. The real picture, it seems, is not what the NYT the made it out to be.
Here's what the NYT printed:
Over the last decade, black and Hispanic students here in Wake County have made such dramatic strides in standardized reading and math tests that it has caught the attention of education experts around the country.The main reason for the students' dramatic improvement, say officials and parents in the county, which includes Raleigh and its sprawling suburbs, is that the district has made a concerted effort to integrate the schools economically.
Here's what the Daily Howler concluded, after a spot of online digging:
Wow! Times readers felt a familiar glow; 80 percent of Wake County black kids scored at grade level on last spring’s tests! But here’s what Finder didn’t tell you—across the state of North Carolina, 77 percent of all black kids scored at grade level on those same tests! That’s right; the Times devoted this front-page story to a three-point difference in passing rates—a three-point difference in passing rates on tests almost everyone passes!...WHY YOU’RE BEING PLAYED THIS WAY: Finder’s piece has an obvious sub-text. Wake County is busing to achieve economic integration—and this is producing big score gains.
For ourselves, we would favor such a program as long as the voters were willing. But these Wake Country test scores provide little evidence of big pay-offs in minority achievement if you enact such a program. Yes, Wake has shown good score gains (most likely on easier tests)—but so have schools all over the state! How can Wake’s program account for gains which are happening in all the state’s districts?
Yes, the gains are occurring all over the state...But apparently, Finder didn’t want you to know that. As good pseudo-liberals have endlessly done, he just wanted you feeling real good about a type of program he favors. As good pseudo-liberals have shamelessly done, he wanted you thinking something bogus and cruel: When it comes to the education of poor black children, success is right there for the taking...
And here's what the op-ed writer concluded:
Intrigued by the story's claim that the percentage of Raleigh's students achieving proficiency had risen dramatically over the past several years, my research assistant, Mark Linnen, took it upon himself to check out the data available on the North Carolina Web site. Over the past 10 years, the percentage proficient or better in grades 3-8 in Raleigh (Wade County) had in fact risen by 13% in math and 12% in reading between 1995 and 2005. That seemed to confirm the bragging of local officials - until it was discovered that, statewide, proficiency rates were up by 21% in math and 19% in reading - gains that outstripped those in Raleigh by over 50%. Nor did the proficiency rates of Raleigh's black and Hispanic students climb any faster than the statewide average for these groups. In fact, the gains were somewhat smaller.Not that proficiency rates in North Carolina mean much. The state has some of the worst state standards in the country. Last spring, my Education Next co-editor, Rick Hess and I gave North Carolina's proficiency standards one of the worst marks in the country - a D minus. (By comparison, South Carolina got an A.) So low were the standards that 85% of all North Carolina eighth graders was said to be proficient in reading, despite the fact that only 29% of the state's eighth graders was found proficient on the National Assessment of Educational Progress, the nation's report card.
One can only suspect that the allegedly astronomical gains in North Carolina - in both Raleigh and elsewhere - were simply a function of a dumbed-down scoring system.
Does the NYT even realize that all the NC scores are online, and that readers can do the math for themselves?
Parapundit has a few things to say on the topic as well
How well would you do on the FCAT? Come Wednesday, you can find out:
State officials will put an old edition of the Florida Comprehensive Assessment Test online so people can see what it's like to be in 10th grade.If you don't know much about history, don't know much biology, you might still do well. The test measures ability in reading and math. The questions were from the 2004 exam, and most likely won't be seen on any test again any time soon.
State education officials said Friday the test will be on the department's Web site. People can download or print the test and take it like students would, then get the answer key to see what they got right and wrong.
The Lodi Unified school district (CA) is unhappy that membership in the gifted-student program Gifted and Talented Education (GATE) is down, and doesn't seem to reflect the ethnic diversity of the local population. So, of course, they've suggested tinkering with the admission formulas:
Trustees recently approved changing the requirements of the Gifted and Talented Education program to allow more opportunity for minority students to join its ranks.Students are now eligible to receive extra points on top of their standardized test scores for limiting factors present in their lives. So, for example, a child who has a learning disability, is an English language learner or comes from an impoverished or culturally diverse background will receive special consideration.
"We've been seeing lately our (GATE) enrollment is down," Lodi Unified Superintendent Bill Huyett said Monday. "That's an indicator that it needs to be opened up a little"...
Despite past attempts to diversify its gifted programs, the district found GATE enrollment did not reflect the makeup of the district.
Imagine that. Schools tend to find that academic grades, test scores, and other measures of academic achievement are not randomly dispersed among ethnic group members. Seen in that light, the fact that GATE participation doesn't mirror the "diversity" of the district is a validation of the GATE admissions criteria. Unfortunately, "diversity" trumps all, so these educators would like to muddy the waters by including some very fuzzy criteria.
Who's going to define what a learning disability is? What criteria will be used? How severe does the disability need to be? Why would the district expect that a student with any sort of learning disability would be able to handle the GATE curriculum? How long can a student be living in the US and still be considered an English language learner? How are we defining an impoverished background? And why would any of these factors, which could be considered a disability, be lumped in together with being from a "culturally diverse" background, which is not? Does this mean Asian and Indian children who are already off the top of the testing charts will be even more likely to be accepted? Or will some cultures be considered more diverse than others?
I agree that the codes are in conflict; it's ridiculous to define a program as being for kids who are on the top end of the intelligence scale and then also demand that the kids in that group be diverse in any sort of multicultural way. They will be who they are, and if the district is truly unhappy to find out that any particular group seems underrepresented, the solution is to investigate why that might be, not to fudge the numbers afterward with some ill-defined and highly-unreliable admission criteria.
What really frosts my shorts about this whole scenario is that the programs are for the benefit of the students enrolled in them. Therefore, the adults involved should be committed to developing admission criteria that guarantee, as much as possible, that any particular student is ready for more challenges, will benefit by more challenges, and will complete the program with the increased self-esteem that comes from tackling, and overcoming, tough educational programs. The current criteria address this by requiring that students perform above a certain percentile on the Raven Progressive Matrices or the CAT-6 exam.
Instead, the adults involved are dithering over the fact that the "diversity" is not what it should be, and have suggested criteria that may in fact be negatively related to the ability of any particular student to do this work or to benefit from it. Does that sound like a plan that's in the students' best interests to you?
The California Association for the Gifted, or CAG defines gifted students as:
...a child enrolled in a public elementary or secondary school of this state who is identified as possessing demonstrated or potential abilities that give evidence of high performance capability...
"Abilities that give evidence of high performance capability." Period. There is no research supporting the notion that overcoming learning disabilities, or learning English late in life, or coming from an impoverished background, are positively related to high academic ability. Therefore, lowering the academic standards for these students makes no sense. If a child shows that he or she is capable of doing the work, these factors should not be used to exclude them. I see no reason why they should be used to include them, either.
For schoolchildren in NYC, their number 2 pencils will now last twice as long:
To the relief of thousands of pencil-biting children and their parents, state and local education officials have reached an agreement that means that New York City's third, fifth and seventh graders will have to take only one round of standardized tests this school year.A conflict between the state and the city had raised the likelihood that children in those grades would have to take four tests over the course of the year - two reading and two mathematics exams - with one set of results for the city and the other set for the state.
Under the terms of the agreement reached yesterday, students will take only the two state tests.
As one educator was quoted as saying, "Common sense prevails."
New Hampshire wonders which came first - the extracurricular activities, or the high scores?
A questionnaire given to 10th grade students taking part in this past spring's New Hampshire Education Improvement and Assessment Program testing also shows that students who work limited hours while attending classes also have a firmer grasp on subjects...This year's 10th graders were offered a questionnaire that sheds lights on how activity outside the classroom might impact learning...a statewide analysis of results showed that student who took part in five or more extracurricular activities (i.e. sports, band, theater and more) had the highest mean scale score on this years test.
Students who took part in five or more such activities — of which Laconia had 9 percent responding — had a statewide mean scale score of 269 out of 300 points in reading and 270 out of 300 in math...The questionnaire showed that students who took part in no extracurricular activities...[had mean scores that] place [those] students in a category that identifies them as having a "basic" knowledge of those subjects.
The polling data showed in general that the more extracurricular activities a student took part in, the higher they scored....One difficult thing to determine is whether students are performing better because they are involved in such activities or whether students who are more committed to school are more likely to engage themselves in endeavors outside the classroom...
Yes, indeedy, it is a difficult thing to determine. It's not a surprise that students who work fewer hours, read more outside of class, and take part in more outside activities have higher test scores, but it isn't simple to disentangle these and draw firm conclusions about what causes what.
For example, students could be urged to work less outside of school, which would leave them more time for reading and other activities. But students from low-income families might be forced to work many hours, and it might be the genetics, family dynamics, home environment, and/or lack of parental education that drive the low test scores more than the time spent behind the counter at Burger King. For such a student, the familial incentive to read (or join the marching band) might be nil, while the time spent as a cashier might be helping them learn valuable job and cognitive skills as well as keeping them from unproductive extracurricular activities.
But there was one survey result that suggests a pretty clear cause-and-effect relationship (if I understand the tortured phrasing correctly):
Students who responded as receiving regular homework assignments also showed to perform considerably better on the math portion of the test than those who did not.
A long and informative article about the rise of technological inventions to meet the need of disabled examinees. There's also a nice discussion of the pitfalls inherent when technological modifications could cause a test to be measuring something other than its intended construct.
Do cheesy songs help raise SAT scores? (Free registration required.)
Renee Mazer is trying to help high school students get into good colleges — by teaching them silly songs and cheesy poems.Mazer is the creator of "Not Too Scary Vocabulary!: For the SAT and Other Standardized Tests and Success in Life," a boxed set of CDs (or audio tapes) aimed at beefing up students' semantic skills. Using playful mnemonic devices and slang-studded stories, the discs teach hundreds of words that often appear on the Scholastic Aptitude Test. By presenting the words in a manner that's easy to absorb and remember, Mazer said, she can help raise students' scores on the verbal section of the SAT. And that can help turn a hapless Ivy League reject into an ebullient Harvard freshman.
What's Mazer's secret? She's never dull.
I'll say, considering that one of the poems talks about emotions involved with "getting to first base."
If you feed them, they will improve:
A pilot program intended to ensure every child has the chance to eat a free and nutritious breakfast is being offered this year to students at Morris Elementary School in Rialto. The program is based on research that shows proper nutrition can positively influence academic achievement, Rialto Unified School District officials said.The free breakfast program paid for by reimbursable state and federal funds is being offered initially at Morris, but if successful, will be expanded to other school sites, said Syeda Jafri, district spokeswoman.
"We offered free breakfast during standardized testing at our schools during last year,' said Sharon Flores, director of nutrition services for the district. "What we found was that the free breakfast reduced tardies and that we had less students in the nurses' offices with stomach aches.'
This part, though, makes me think teachers will be driven nuts:
"The food is served in the classroom right when class starts so it's a family-type environment,' Flores said.
Which means teachers will have to monitor table manners and clean up all that jelly off the desks afterwards.
A thoughtful and quirky look at the testing craze, from a young (class of '98) reporter:
...I just took last year’s standardized English Language-Arts Test for the 11th grade - which anyone can sample at the state Department of Education’s Web site and which the state uses to monitor its districts’ progress - and while I scored well (a 95 percent, thank you very much), there were only 38 questions, nine of which were devoted to understanding a car rental agreement and the instructions of a food processor. Only one great literary work was examined - “Young Goodman Brown” by Nathaniel Hawthorne - and there were scant vocabulary questions, which, maybe, is a good thing....I can say that my foray into the world of standardized testing didn’t exactly fill me with the overwhelming confidence that the results of these tests will mean anything significant. Sure, the state will use them to decide which school districts are doing their job and how to parcel out an increasingly limited chunk of resources, but is it more than just a numbers game?
In speaking with the district’s curriculum director, Elizabeth Chapin-Pinotti, about the county’s California High School Exit Exam and Standardized Testing and Reporting results, which were released last week, it occurred to me that the above question doesn’t simply have a ‘yes’ or ‘no’ answer.
Chapin-Pinotti said that California schools tend to get a bad rap on the national stage when the test results come out, because what’s not factored in is that the state employs some of the strictest standards in the country. “The state set the bar very high and we’re not backing down,” Chapin-Pinotti said.
Reporter Raheem Hosseini goes on to discuss the Connecticut lawsuit before wondering why we care so much:
So what is it about tests we all love so much (and don’t kid yourself, we’re obsessed with them)? I think it has something to do with the simplicity of being able to quantify an amorphous concept such as intelligence. Color in a few bubbles, feed your sheet into the Scantron machine and find out how smart you are. And the strange thing is that even after high school and college and after having taken hundreds of scholastic tests through roughly 20 years of school, we’re not done with them.There are employee evaluations and credit assessments and loan applications and - even weirder - the tests we actually choose to take in our spare time: crossword puzzles, jumbles, online IQ and personality tests. It never stops.
But is it really so bad for people to crave a little disposable evaluation? Maybe what they’re really craving is intellectual stimulation. Heck, even Tommy Lee is back in school.
The Chicago public schools are abandoning the Iowa Test of Basic Skills, or ITBS, in favor of a revised Illinois Standards Achievement Tests, or ISAT:
It's welcome news for both Chicago teachers and students who had to spend hours of preparation for the tests and lost precious class time for other lessons. Now, the public schools will be able to focus on one high-stakes test, the revamped Illinois Standards Achievement Tests, which has become an important measure of Chicago schools' performance under the No Child Left Behind law. "It's fabulous," Nobel School Principal Mirna Diaz Ortiz says. "I'm very happy that we're going to be measured by one test and not have to take two tests"...In addition to the ISAT, there will be another new test, but it will not put the same burden on students and teachers that the Iowa test did. It will not be used to determine promotion. The Stanford Learning First measures students' strengths and weaknesses in reading. It will provide a diagnostic tool for teachers to see where their students need help. It will be offered three times a year through 40- minute exams and will cover the same kind of material that is in the ISATs. It won't require extensive preparation.
The Illinois Standards Achievement Test (ISAT) measures individual student achievement relative to the Illinois Learning Standards. The ITBS, on the other hand, gives results in relation to an Iowa norm, or a national norm. This means that promotional judgments will be made for scores based on comparisons within Illinois, not for the nation as a whole. This sounds consistent with NCLB regulations.
Interactive ISAT samples can be found here. I'd like to be objective, but I note that the eighth-grade Reading sample is a reading passage written by James Thurber, one of my all-time favorite authors. I started reading him about that time, too. So I'm predisposed to like this exam.
This Chicago Tribune article has more info:
The Iowas, used in Chicago since 1972, will be replaced with three short reading assessments that officials believe will prove more valuable in gauging the progress of individual students. Called Stanford Learning First, the new 40-minute exams will be given in October, January and May and will test the same kind of material covered by the new ISATs...Officials said they still expect to use the historic data as a basis for comparison by creating a new formula that can equate the results of the old Iowas with the new Stanfords.
This can work, if the new Stanford exams have the same content as the old Iowa exams. Seems like this is a new, low-stakes way to test reading, with the scores being equated back to the high-stakes ITBS.
The test change also will trigger changes in the district's controversial retention policy, the details of which will be announced in October. The get-tough retention policy was created in 1997 when Daley declared an end to social promotion and started requiring students to meet minimum test standards in reading and math.The policy has been softened over the years. Now the district only considers reading scores and bars schools from retaining students twice in the same grade regardless of how low they score on the tests. Last year, about a third of the 24,000 students required to attend summer school because of low Iowa scores had to repeat a grade.
The article also mentions that the ISAT-based promotion decisions will be based on the ISAT portion that is scored in comparison to national norms. Interesting.
If any of you have experience with the ISATs, or have information that hasn't made it into the Chicago papers, let me know.
One of the testing criticisms I often see is the claim that standardized tests can't tell you everything about a student's achievement:
Naysayers to the testing format say it reflects not what a student has grasped conceptually about a subject, but how well they take tests. These critics, most often teachers, point out that each student processes learning differently - some are better able to express verbally or in essay form their depth of knowledge on a subject..."Think of the driving test," said San Luis Obispo High School's English department chair, Ivan Simon. "If you just looked at how well someone answered the written part of the driver's test, then you'd assume the skill of the driver was represented by only that score. But that person wouldn't necessarily be a good driver."
It is true that the performance-based exam of driving skills tells you much more than the written exam, because the skill being measured is wholly performance. However, one could argue that someone who knows how to turn the key and step on the gas is not a good driver if they cannot read road signs or haven't memorized any of the rules of the road. Someone who passes the written exam is not necessarily a good driver, but we can argue that someone who flunks the written exams is necessarily a bad driver. Both components of the exam are important. The ability to understand signs could be folded into the performance assessment (and often is a part of it), but the reason for the written exam parallels the reason for many standardized multiple-choice assessments - it's a way to very quickly sample a broad domain and make a cheap, reliable assessment in order to flag those who just aren't getting it.
Many standardardized exams are, in fact, minimum competency exams, and the best precision is not in separating the brilliant from the good, but the terrible from everybody else. Simon's criticism that multiple-choice exams often don't tell the whole picture is correct. However, the critics gloss over that these exams can tell you quite a bit about how students have mastered basic skills.
And, as one principal points out, the basic skills are important:
We provide teachers with examples of multiple choice questions, but it's not the sole focus of the curriculum," said [Will Jones, principal of San Luis Obispo High School]. "The state releases sample questions, roughly 20 percent of the test, and if teachers want to use those questions they can, and they do, just like they do for advance placement tests. The misperception is that we spend all our time just teaching to the test and somehow the STAR exam is consuming education, but the truth is there are all kinds of assessments and we prepare kids for all of those as well."Jones said he believes good students will always excel regardless of test format. "If a student is capable of writing a good essay and answering short questions, then they will have success on the test," said Jones.
This is something that few testing critics are willing to admit. From what I can tell, the evidence that there are hordes of little geniuses out there who routinely flunk exams yet learn brilliantly through non-traditional methods is anecdotal, at best. The testing critics are right when they say that there are skills we aren't measuring with the one-size-fits-all exams, but I say they're missing the boat when they insist that students who can't master basic material are somehow ready for advanced performance assessments.
Devoted Reader Lori M sent a provocative column my way:
Tens of thousands of parents of schoolchildren and hundreds of thousands of other taxpayers learned from media reports last week that "the majority of Texas school districts and campuses in 2005 earned the rating of 'Academically Acceptable.'" Most of the moms, dads and school-tax payers breathed a sigh of relief and shrugged off the "bad" news that a small percentage of districts and campuses "received the lowest rating of 'Academically Unacceptable'"...To "earn" a rating of "Academically Acceptable" on the 2005 Texas Assessment of Knowledge and Skills test, the students in a school district or at an individual campus had to achieve ...
In reading and English language arts, a passing rate of 50 percent.
In writing ... 50 percent.
In social studies ... 50 percent.
In math ... 35 percent.
In science ... 25 percent.In case the educational horror of those numbers didn't sink in ...
If half of the youngsters in a district or on a campus failed tests in reading, writing and social studies ... and 65 percent failed arithmetic ... and 75 percent failed science — the Texas education establishment deemed that district/campus "Academically Acceptable"!
Lori comments: "I guess I'm part of that old-fashioned school of thought that believes that a passing grade consisted of mastering the majority (at least 65%) of the material. Looks like it's sufficient to learn 25-50%!"
What's happening here? A disconnect between how we, the educational consumers, think of "passing," and how Texas is ranking schools, which is with a minimum-competency standard. "Acceptable" here is not defined in the same way that we'd consider "acceptable" to be in an academic course.
The author isn't exaggerating when he quotes the low percentages above; those come straight out of the state's 2005 Accountability Manual. You can skip right to this table for the good stuff. Yes, it's true that this year, a school for which 26% of the students meet the standard in Science is acceptable in that content area. This is also the first year the Science standard was set at what the advisory panel actually recommended, as opposed to one (2004) or two (2003) SEMs beow it.
To interpret these numbers, you really have to have some idea of what the standards are, so that you know if a school in which only 26% meet those standards is a travesty, or just plain mediocre. You also have to realize that while the state set those standards very low, that doesn't necessarily mean most schools are squeaking in just over the bar. The 10th grade Science results, for instance, show that 54% of the overall student body in Texas met the standard. The raw score conversion table for that exam shows that a raw score of 34 out of 55 converts to the lowest possible passing score, and according to this document, that means a student would have to answer 62% of the items correctly to meet the standard. That's pretty much in line with what most people think of as a passing score.
I don't mean to suggest that parents don't have a right to wonder why the standards for Acceptable schools aren't set higher. And, given that we don't know how difficult the science items are, for example, we don't know how meaningful that 62%-correct standard is. But it might help in this debate to be sure to separate the standard for the exam from the standard for the schools.
Joanne Jacobs points out that our students may not be dumb - they just might not care:
You could conclude from these exams that American high-schoolers are ill-taught and ill-prepared for the competitive global economy. But what if you look at these tests like a capitalist rather than an educator? Nothing is at stake for kids when they take the international exams and the NAEP. Students don't even learn how they scored. And that probably affects their performance. American teenagers, in other words, may not be stupid. It could be that when they have nothing to gain (or lose), they're lazy...The dubiousness of these test results becomes clear when you compare them to the results of tests that actually do matter for teenagers: high-school exit exams and college boards...
Alexander Russo, for one, is suspicious of such a neat-and-easy conclusion:
if things are better now in secondary education than they were before, shouldn't kids today still outscore kids from 30 years ago? They were unmotivated to perform on the NAEP then. They're unmotivated now. They know more now, according to Starr. But the scores aren't much different.And what about elementary school NAEP scores, which are on the rise? If motivation is all, then shouldn't they stay flat?
Now I'm no economist or behavior expert, but it seems to me that if high school kids were actually learning more in school than they had before, the NAEP scores would show at least part of that change.
I'll play Devil's Advocate - could it be possible that kids know more, but care even less? After all, we're constantly told that kids are over-tested and are sick of exams, and perhaps there's truth to that. Could it be that years ago, even though kids knew the NAEP didn't count, they were more motivated, or less burnt-out?
I have no doubt motivation plays some part in test scores, which is one reason that I keep griping at states to stand behind the stakes for their exit exams. But I don't think the explanation is as easy as saying that our kids don't really care. Presumably, they care when they get to college, and yet the rise in remedial coursework and grade inflation belies the notion that today's students, as a group, have what it takes.
Update: Chris Correa notes:
...some research suggests high school students don’t perform significantly better on these tests when they feel more concerned in their own performance. For example, when researchers offered up to $10 per correct question on the TIMSS test, the paid participants did not perform significantly better than the control group. Students did report increased motivation when they were earning an average of $100 for their performance, but this did not make them any smarter.High school students seem to do about as well as they can on the test, even when they don’t have anything to gain from it.
Excellent point, Chris.
The merit pay discussion takes on a weird twist:
A San Joaquin Valley teacher hopes money will motivate California students to improve their standardized test scores.Jo Aldrich-Fallert of Porterville will soon start gathering signatures for an initiative that would give $1,000 to parents of children who score at the proficient level or higher on all sections of the California Standards Test. She says she wants to reward families who have made education a priority in their homes.
If the idea proves to be a successful motivator, the state could end up paying out more than $1 billion to families of students who do well on the test.
Critics of the idea say doing well in school should be its own reward. They also worry about the cost at a time when the state is strapped for money.
Seeing as how money doesn't grow on trees, I agree with the critics here. I hope this idea gets more publicity, though, just for the amusing debates it will generate. If we're willing to pay parents, but not teachers, more money for higher test scores, aren't we admitting that parents have more to do with a student's academic achievement than the teachers? Isn't that support for homeschooling? And what about the fabulous, motivated parents whose children get stuck with turkeys for teachers? They'll miss out on the dough because a hapless educator undid all the hard work of tutoring and motivation that was done at home. If this amount of money was on the line, wouldn't parents be even more observant - and critical - of bad teaching and bad schools?
Like I say, amusing.
It's an ironic world we live in:
UNCG professor Svi Shapiro has written articles critical of standardized testing. Now, he's become part of the test. One of four prompts used for June's essay section of the newly revamped SAT was adapted from Shapiro's writing."I became the test instead of criticizing the test," said Shapiro, who has taught at UNCG for about 20 years. "It is ironic."
Shapiro learned that an adaptation of his work had been used on the test while giving a lecture at the Governor's School of North Carolina. The prompt asked test takers to explore whether schools should help students "understand moral choices and social issues." "I was surprised, to say the least," Shapiro said. But he said the question captured the essence of his criticism of standardized testing...
Talk about your merit pay:
Indianapolis Public Schools may be among the first in Indiana to link their superintendent's pay to student success, but a growing number of urban districts across the country already have been taking that approach. School boards in Cincinnati, Dallas, Denver and Illinois have all negotiated contracts with their top administrators based on meeting tough academic goals.The Palm Beach County, Fla., superintendent can earn a 15 percent bonus on his annual $155,279 salary if test scores dramatically improve, suspensions drop and graduation rates go up.
That's...a lot to hang on one person, especially a person who isn't in the classroom. On the other hand, if he actually has the power to change things, well, this is a pretty good opportunity to do so.
Normally, when I read an newspaper article about a student who's been struggling with a standardized test and keeps coming in just under the wire, I'm pretty unemotional. Yes, that sucks, but a cutscore's a cutscore. While the setting of the cutscore is a delicate process, there will always be someone who just missed out no matter how it's set, and I'm usually willing to defend the process against that unlucky fellow who's just a point or two too low.
But I really don't want this guy mad at me:
All that stands between Garry Williams and a spot on the University of Kentucky's football team is a point on a standardized test. And it makes Williams downright mad. Not necessarily mad about the test or the NCAA's entrance requirements that call for student-athletes to achieve a minimum grade-point average and test score. Just ticked that he's not playing football at UK.Williams could be entering his sophomore season at UK and likely competing for a starting job on the Wildcats' offensive line. Based on the billing he earned as a first-team All-State performer at Seneca High School, Williams might even be a returning starter.
But the NCAA no longer allows "partial qualifiers" -- athletes who meet only one of the two minimums for grade-point average and standardized test score...
The article emphasizes that neither Williams nor his father used this opportunity to bash the test (in this case, the ACT), which is admirable. But Williams is mad, and who could blame him? Who's to say what might happen should he get more and more frustrated with the studying, and then one day happen to (gulp!) meet a psychometrician in the street?
Full disclosure, Williams - I don't work for ACT. Now put down the barbell. Please?
Knowing how badly he's needed only adds to the frustration...Garry Williams Sr. said. "To be honest, he's ready to strap on the pads and hit somebody."
Aaaaaah!
More good news from the elementary schools:
Black and Hispanic students are catching up with their white counterparts in reading and math at the elementary-school level, but there has been little closing of that achievement gap in higher grades, according to a study released yesterday.The Bush administration cited the data from the National Assessment of Educational Progress (NAEP) as evidence that its educational revisions are working. But the independent body that administers the tests urged caution, saying that many of the gains could have come from changes made before the 2002 passage of the No Child Left Behind Act.
The NAEP study of long-term educational trends showed a significant improvement among white, black and Hispanic 9-year-olds in the 2003-2004 school year in math and reading, compared with results from five years earlier. But blacks and Hispanics made greater gains than whites in both subjects.
Alfred Orsini spots a link between the "fat" society of the US and the unwillingness to get behind tests with real bite to them:
China is a country that promises to change the economic face of the globe in the years ahead. It has an incredibly lean, mean education machine. American policymakers, seeing embarrassing data such as the poor showing of U.S. students on recent international assessments in science and math, say they want to prepare our children to meet the global competition posed by countries like China.If the No Child Left Behind law is meant to do that—to help us compete with countries that have used big tests for a long time to scientifically weed and stratify their citizens—the plan will fail. This is not just because of the problems inherent in creating and enacting such tests, but also because of the differing social, economic, and cultural contexts that surround such tests...
U.S. business leaders, in their urgent push to whip American education into line, may be among the few in our country who are truly aware of how things are outside the United States. They have a concrete motivation to be aware: money...But if my perspective from China is valid, then a more general “leaning” of America may have to happen before big tests are widely tolerated by U.S. students and their families.
For now, we Americans can make all the tests we want. Kids will never be “lean and hungry” in a fat society. Big, consequential tests run up against a lot of obstacles in America. One of them is fundamental—what the historian Richard Hofstadter labeled in his influential book as Anti-Intellectualism in American Life. Others include the distraction posed by the cultural excesses that assault American kids every day and the fact that U.S. students know there is a college in America for just about anyone who can pay for it. But beyond such problems, I wonder if American kids sometimes don’t care about study because they sense that many adults don’t care about them—about the life of their minds and the enrichment of their souls.
On the one hand, I can understand why some critics claim that the US is test-obsessed; certainly, those on the front lines of education see it that way. On the other hand, how many of those tests have serious consquences attached? And by "serious," I don't mean "You'll have to go to Penn State instead of Harvard." The great hue and cry from the educrats over the fact that now, for the first time ever, schools are forced to justify their funding by showing student improvement suggests that Orsini is onto something with his theory. Americans are far more enamored of the idea of second chances and alternate pathways than they are of tests with truly high stakes.
From the Interested Participant comes a fascinating tale of teachers complaining about high test scores, and superintendents claiming that it's all about the money:
Thousands of Arizona students struggling to learn English are about to lose extra help because a new state test shows they can read and write in English, though educators fear many of them really can't. In some districts, students are passing a new state test that says they are proficient in English at nearly double the rate of last year. But educators say the test is just easier and these kids aren't likely to pass regular classes without help.Arizona Superintendent of Public Instruction Tom Horne said the criticism comes because some school districts don't want to lose the extra money, about $350 per English-language learner. He said some schools have kept students designated as English-language learners for years because it means more money.
What a mess. Certainly, though, if the teachers are correct in their suspicions about the test, the results should be pretty much immediately apparent; a doubling of the overall pass rate is certainly cause for concern. However, I'm not sure if they're comparing apples to apples here:
Arizona school districts used to have the choice of one of four language tests, but the federal government now requires states to select one test for consistency. Arizona chose a Harcourt Assessment test, along with several other states..."We're very concerned," said Cindy Segotta-Jones, director of language acquisition for Cartwright Elementary School District in Phoenix. She said about 2,200 Cartwright students passed the language test, nearly double the usual rate.
But on what test was the baseline set? If 1100 students passed last year, was that over all four tests? If so, is it possible that the other tests were too hard, and this one is closer to being just right? There's no way to tell from this information, of course, but it would help if I knew the pass rate on this exam from last year vs. this year. Now that everyone is using the same, it will be possible to compare results between schools in way that wasn't feasible before.
Sal Gabaldon, a language acquisition specialist for the Tucson Unified School District, said the new test is far different from previous ones used in Tucson schools.The previous test used in Tucson required students to pass all three parts - reading, writing and oral - to be declared proficient in English, he said. The new test uses a composite score so it's possible that a somewhat lower score in writing could be offset by a higher oral score.
Becuase the previous test was triply conjunctive, one could argue that it was more difficult to pass, even if the individual items were the same difficulty as on this exam. However, is it a bad thing if the test isn't conjunctive? I'm not a content specialist in the area of English proficiency, so I don't know (a) how possible it is to get wildly different scores in the areas of reading, writing, and speaking, or (b) whether having wildly differing scores is predictive of disaster. Is it highly unlikely that a student will be a disaster in one area and great in another? If a student is good enough to pass in any one of the three areas, will they have a decent chance at succeeding later on without additional remediative help? I'd have to know all of those things before I could judge whether the test is useful for this purpose.
Tucson's teachers apparently think these results presage disaster, but Interested Participant wonders if some of that is hurt feelings from having control over exam choice taken away.
In Michigan, sliding MEAP scores are scaring officials, who believe the state's chances of attracting more high-tech companies are in jeopardy:
...a shrinking percentage of high school students are exceeding the state's requirements in math and science. Results from Michigan Educational Assessment Program tests for the class of 2005, released this month, give the state a clearer picture of the challenge it faces in its effort to attract high-tech businesses with a better-educated work force...Over the past six years, the percentage of high school students scoring well below what the state considers failing on standardized math tests grew from about 20 percent to 29 percent, which translates into almost 32,000 students in the class of 2005. In science, it grew from 20 to 26 percent. Meanwhile, the percentage of students scoring well above state standards fell from 22 percent to 9 percent in math and from 7 percent to 5 percent in science.
Testing experts and school officials caution that there could be many reasons for the trend in math and science other than students' skills getting worse. They say more kids are taking the test to meet federal requirements of the No Child Left Behind Act. Under the law, schools must test 95 percent of their students. These students, who previously may have avoided the test because they felt they wouldn't do well, could be bringing down scores, they say.
Certainly, an influx of underqualified examinees can reduce the percentages - but I'd be interesting in knowing if the raw numbers have changed. If the actual number of students scoring at proficient has stayed the same while the percentage dropped, that would be evidence of the more-underqualified-examinees theory. However, if the number of students who are doing well is dropping along with the percentages, then something else is happening here.
I wouldn't feel comfortable assuming that the numbers above were signs of random, meaningless fluctuation. Either the examinee population is changing, the tests aren't equated very well, or it's some combination of the two (perhaps along with other factors).
Several Devoted Readers have sent in the news that the Virginia Board of Education is phasing out the Praxis I licensure exam for teachers. There is a new exam for these teachers that apparently doesn't contain any of the tricky math that was on the Praxis I:
Praxis I is a basic reading and math skills test that has been a hurdle for some teachers--especially the math portion.Starting Jan. 1, the state will require teachers to take a new test for licensure. It will eliminate the math material of the Praxis I. Instead, it will require teachers to analyze readings, write an essay, interpret tables and graphs, and demonstrate knowledge of grammar and vocabulary, all "on a college level," said Charles Pyle, spokesman for the Virginia Department of Education.
The Washington Post claims that VA teachers will now have to be more "literate and proficient in the subjects they teach," but those who don't intend to be teaching math need not worry about a standardized math test:
Kate Walsh, president of the Washington-based National Council on Teacher Quality, said she pushed for a new literacy test when she testified before a Virginia panel examining licensure requirements. Research has shown that the ability to read and speak effectively is the most reliable predictor of future success in the classroom, she said...
The Virginia board claims that the this new test actually represents an increase in standards, which makes me wonder - if the pass rate drops dramatically from the Praxis I rates, will Virginia stick by the new exam?
The Cranky Professor is skeptical:
What's not at all clear from the article is what the 'Instead, they will have to pass a new "literacy and communications skills" exam that will be introduced in January' is. Will it be a product of the ETS people, like Praxis? Will it be home-grown, in which case I dread the first 3 or 4 years of results and controversies.I took the Praxis I a long time ago, by the way, and it really IS at the 8th-10th grade level (I was half-heartedly pursuing certification to teach high school Latin). If people can't pass it (the Post's anecdote has a PE teacher passing on her 6th attempt) they probably aren't capable of figuring their own grades.
Why is no one asking why so many teachers - who are, after all, college graduates - are having so much trouble with basic math skills? The Praxis I math test has only 40 items and takes only an hour - but calculators are not permitted. The sample items mentioned in the ETS link are not tricky, and most of them, quite frankly, could be done in one's head, or with a bit of pencil scribbling.
Do English and music and PE teachers need to know this much math? Not necessarily. I just find it fascinating that schools of education are apparently churning out college graduates who can't do this.
Update: This can't not be related to the observed dumbing-down and politicization of math instruction in schools. Joanne's comments on this article are not to be missed.
Also, Right on the Left Coast has much more to say.. He also quotes the excellent book Innumeracy on the astounding phenomena of how people who would be ashamed to admit they can't read have no problem admitting that they don't understand basic math. It's all about being a "people person" instead of a cold, impersonal number cruncher, don't you know.
Update #2: As always, Joanne cuts right to the point:
Thirty-five years out of high school, I can do these problems in my head. It's hard to believe there are people smart enough to teach who can't pass a basic math test. How are they going to average students' grades?
Why does Joanne assume that these math-phobic teachers will be assigning objective, numerically-based grades? My guess is anyone this terrified of Praxis math items will be giving "holistic" and subjective letter grades.
That's much more "personal" and "caring" than those nasty ol' averages, you know.
An amusing (because it's positive) editorial on testing from the Arizona college newspaper:
They say the tests are ruining the educational landscape -- that they're causing students to study for nothing else...Many want to see them eliminated. These opponents point to statistics like the amount of money people spend on preparation, $310 million a year by one count, as signs that the standardized-test mania has gone too far and that a return to the core values of education is warranted.However, such logic fails to understand the purpose that they serve. Simply put, standardized tests serve one vital function – as providers of a means of comparison to admissions panels.
What makes them so valuable? Why can't admissions panels look at more important factors? Factors like grades, letters of recommendation, and extra-curricular activities? That's where the standardized part comes in. Without standardized tests, there is no way to compare students across the country. With GPA, for example, there's no way to tell whether the 3.6 from Northwestern is more impressive than the 3.6 from Virginia.
So standardized tests are good because they're standardized. But rather than complain, UA students should cheer this opportunity as they approach grad school rather than boo it. It's hard enough competing with students from big name schools like MIT and Princeton. With the GRE/MCAT/LSAT/GMAT to level the playing field, students can show that they're just as sharp as the Ivy-League trained type.
In fact, that's why tests of these kinds were developed. With all the anti-testing commentary in the press, it's easy to forget that. What's more, it's always driven me crazy that opponents produce the over-inflated and unproven expensive test prep course as evidence (somehow) that the tests themselves are a problem.
Cute photo of the author, by the way.
Also, this is the nth article I've seen that references the book, "Freakonomics," so I'm going to have to break down and buy a copy.
The portfolio option arises again, this time as an alternative to the Regents exam ($ubscrip required):
One of the items still on the table in this final week of the legislative session in Albany is a bill that would direct the state education commissioner to come up with an alternative to the Regents exams required for a high school diploma in New York...The bill has already passed the state Senate and is now before the assembly, where the speaker, Sheldon Silver, can either pass it or kill it. Our friends in the education policy community...are hoping the speaker will kill the bill, seeing it as a departure from the measurement, accountability, and standards that are essential to quality education. The New York Times and the Daily News are in agreement on the point.
I can't find this story anywhere else (the blurb from Google quoted a line about portfolios), and, unfortunately, that's all I can get without subscribing. Anyone out there got a subcription?
Update: A sharp-eyed reader found a more informative article from back when the bill had just passed the Senate:
Supporters say it's better than the state's current one-size-fits-all system of testing. Opponents, including state Education Commissioner Richard Mills, fear the measure could cripple their decade-long effort to use the Regents exams as a way of raising academic standards across the state...One of the bill's sponsors, Stephen Saland, R-Poughkeepsie, who heads the senate's Education Committee, said the measure stems from what he views as an unwillingness by Mills and the board to compromise on having all students take the exams. "It's their way or the highway," said Saland.
But Mills says setting up a system of portfolios would be too costly and difficult to police. "The bill would create an unworkable system," predicted an Education Department memo to lawmakers. "It would be virtually impossible to monitor all schools closely enough to ensure that schools followed the curriculum and assigned all projects, and that an A in one school equaled an A, not a C, in another school."
When it comes to school accountability, what exactly is wrong with "their way or the highway?" And Mills hits the nail on the head with his concern about the un-standardized nature of portfolio projects.
Update #2: The Instructivist has more, and links to a NYT editorial which is, astonishingly, crictical of the bill:
Before they jeopardize education reform, legislators should revisit a disturbing report issued a few years ago by a panel of education experts that evaluated the portfolio assessments used by the schools in the New York Performance Standard Consortium, a politically influential education group. The panel could find no evidence to support the claim that the consortium's schools were conforming to the state's learning standards or measuring student progress in any meaningful way.
A hullabaloo in Georgia over the dismal algebra end-of-course exam results:
More than two-thirds of the DeKalb County school system's eighth-graders failed the state's End of Course Test in Algebra I, according to preliminary results. The End of Course Test counts for 15 percent of students' final grades in the course. Teachers issued failing grades to 21 percent of all eighth-graders. This was the first year district policy required all eighth-graders to take high school algebra, a move the board approved in 2003 to beef up math instruction in all grades.Officials reviewed first-semester grades and anticipated a high failure rate. Originally, the policy called for eighth-graders to earn credit toward graduation. But the school board is expected to approve a policy change in July. Parents will get to choose whether their child gets a credit toward graduation, known as a Carnegie Unit, or a middle school credit that does not show up on the transcript college admissions officers see...
For many, it would jeopardize their chances for a lottery-funded HOPE scholarship. Other issues cited by board members: This year's eighth-graders did not have the foundation starting in kindergarten for algebra, and some teachers were not adequately trained to teach algebra.
All pertinent issues. The officials were wise to antipate a poor showing. The question is whether results will improve in the future.
Tough state standardized tests - how does your state measure up?
A newly published study...conducted by Paul E. Peterson and Frederick M. Hess, editors of the quarterly journal Education Next...compared how fourth and eighth graders performed on state tests compared with the 2003 National Assessment of Educational Progress test.State tests are used to rate school performance under the federal No Child Left Behind law. Each state creates its own test and sets the passing score. The result is states with easy tests can appear to be doing well, while those with challenging tests can appear to be doing poorly.
"Some states have risen to the challenge and set demanding proficiency levels for their students, while others have used lower standards to inflate reported performance," the report said. "Not only is the disparity confusing, but, perversely enough, the states with the highest expectations often stand accused of having the most schools said to be in need of improvementeven when their students are doing relatively well."
The top five states in the rankings were South Carolina, Maine, Missouri, Wyoming and Massachusetts.
In other words, if a state claims that fewer students are proficient than NAEP scores would indicate, that state gets a higher score for challenging its students; the opposite holds for states whose students do not seem to perform as well on NAEP as on state exams. The full list of 40 states (10 do not have state tests which allowed for NAEP comparisons) is here.
School administrators suspend a fourth-grader who balks at a WASL item:
Tyler Stoken is 9 years old and his mother says he's good at taking tests. But when it came to the recent Washington Assessment of Student Learning, one question stumped him. He was asked to write a short essay about a make-believe situation and his principal.Tyler paraphrases the question saying, "You look out one day at school and see your principal flying by a window. In several paragraphs write what happens next." He's asked, "So why didn't you answer that question?" He says, "I couldn't think of what to write the essay without making fun of the principal."
He refused to answer the question even after his mother was called to the school. Tyler's mother Amy Wolfe says, "And he said he didn't know the answer. He just didn't know what to write. And they were telling me to make him answer the question."
He still didn't, so Tyler was given a 5-day suspension. In the letter that went home to mother, the principal writes, "The fact that Tyler chose to simply refuse to work on the WASL after many reasonable requests is none other than blatant defiance and insubordination."
Amy and her son were shocked. Just then the phone rang. It was the superintendent calling to apologize. "Because I think a mistake was made and over reacting to Tyler's refusal to complete the test," said Aberdeen school superintendent Marty Kay. He says it points to the bigger issue of how much pressure is placed on students and staff to do well in the WASL.
The last I checked, no test in the world required every single examinee to get every answer correct, or even to answer every single item. Refusing to answer one item is not the same thing as refusing to take the test. The school's mistake was appalling, but equally appalling is the "excuse" that the WASL places too much pressure on the staff. This sort of nonsense plays right into the hands of testing critics, as well it should.
This shouldn't turn people against the WASL - it should turn parents against administrators who think badgering kids is the only way to raise WASL scores.
Via Morons.Org.
Devoted Reader Michael S. brought my attention to a story from his local paper:
Eighteen-year veteran Champaign Central High School mathematics teacher Kathleen Smith stunned board members by resigning as a protest, she said, against district and federal policies that force teachers to "teach to the test." During Monday's school board meeting, Smith said she's resigning because she's at odds with current standards in the district and with methods imposed on teachers by George Bush's landmark No Child Left Behind legislation."Each year students come to me with different skills, different strengths and different weaknesses," Smith said. "It's always a learning process. They learn about me and I learn about them. Now I find myself constrained by a mentality that says all students will learn the same material at the same pace and prove it by taking the same multiple-choice test within a given time frame. I do not believe a student's understanding of mathematical concepts can be assessed by a multiple-choice test, nor do I believe that such a test is fair for all learners," she said. "I'm resigning because I'm caught in a moral dilemma. "
Fascinating. She decided to resign immediately after Bush was re-elected this past November, but waited to make her announcement at the end of the school year.
New York City's eighth-graders aren't distinguishing themselves in social studies:
A stunning 81% of the city's eighth-graders flunked the state's basic social studies exam last year - and the scores have gone down annually since the test debuted in 2001. The troubling spiral was disclosed by Education Department officials yesterday at a hearing on civics and social studies instruction called by the City Council's Education Committee."Clearly we have a crisis on our hands," said City Councilwoman Eva Moskowitz (D-Manhattan), who was chairwoman of the hearing and blasted educrats for having a lack of urgency about how to adequately address the problem. The failure rate for eighth-graders on a test that measures students' knowledge of basic history and government has climbed steadily from 62% in the 2001-02 school year, to 76% in 2002-03 and 81% in 2003-04.
I think the conflicting reasons given for the decline - that teachers are focusing too much on reading, or that students can't read and thus don't understand the items - are priceless. But then, I don't have a child in this public school system.
So what are these basic history and government items like? I think this is the exam they're talking about (let me know if you know otherwise), and here's one of the June 2004 exams. Forty-five MCQ's and a few open-ended items. I found the scoring key, but I don't know the cutscore, so I don't know where the standard is set.
The MCQ's aren't gimmes. They do assume that students have memorized quite a few facts about Native Americans, American history, presidential elections, US geography, and the like. Take the item below, which is well-constructed and crystal clear - but if a student doesn't know the answer, guessing is their only hope:
The Supreme Court decision in Marbury v. Madison (1803) established the principle of(1) judicial review
(2) separation of powers
(3) habeas corpus
(4) nullification
Are these questions going to be tough for kids who can't read English? Absolutely! Does that mean the test is too tough for NYC eighth-graders, or that they shouldn't be able to read on this level or answer these types of items? I'd say no, but I'm not a content expert in social studies.
An amusing and thorough article about the rise and potential fall of standardized tests:
...the consensus is that standardized tests weren't created for such a sweeping, high-stakes purpose. Scores point out a child's strengths and weaknesses, but they don't paint a complete picture, say experts, known as psychometricians.They cite other problems with high-stakes tests: greater motivation to cheat and the possibility that results will be distorted by overpreparation.
"That's the position of our entire field," said Steve Dunbar, head of Iowa Testing Programs, developer of the Iowa Test of Basic Skills, taken by more than 350,000 Georgia kids in grades three, five and eight. "A test is just a snapshot of where that particular kid is on a particular day"..
Experts in the Lindquist Center, where test booklets are stacked high against the walls, expect No Child Left Behind to run its course. They're confident the politically driven pendulum will swing back to a more reasonable view of the value of testing. Dunbar predicts public support will wane because of results that don't seem to make sense — as when a highly regarded school like Cobb County's Walton High gets dinged for not making enough progress, which happened in 2003. (Like many Georgia high schools, Walton did not test 95 percent of its students, as the law requires.)
"The tests," Dunbar said, "will lose credibility."
Until they do, life is sweet for those schooled in test development. Testing companies, academic think tanks, public policy groups and state agencies compete for the great minds in testing, especially those that come out of the University of Iowa.
In fact, I'd say competition is fierce even for the not-so-great minds, as evidenced by the continuing employment of yours truly (heh). On the other hand, pretty much every testing company is understaffed and overworked, which is definitely "unsweet" in many ways.
The question of boycotting state exams is being posed in New York state:
Ryan Ficano, 10, and his fourth-grade classmates at Center Street Elementary School are scheduled to take a state-mandated math test next week. Ryan, who boycotted the state’s English language arts test in February, hasn’t decided whether to take the test, his mother, Carli Ficano, said Thursday night...Ficano was one of eight parents who met at the Center Street school to talk with administrators and teachers about required testing and the philosophy of mandated exams. Her discussion with Ryan would include teachers’ favorable and neutral opinions of the tests, she said, and that it would be OK not to take the test this year if he chose.
Center Street's principal has asked parents who oppose the testing to take their concerns to the state legislator instead of allowing their kids to boycott (which could certainly hurt the school standing.)
There's an abundance of standardized tests these days, and an equal abundance of confusion from test-wise, results-foolish districts that aren't quite sure how to use all that data. A town-gown collaboration aims to correct that, as Boston school district employees and Harvard faculty and students team up to write the book on using assessments wisely:
Data Wise: A Step-by-Step Guide to Using Assessment Results to Improve Teaching and Learning will be published by the Harvard Education Press in November. All royalties from the book will go to the education school to work with the 58,300-student Boston public schools. The book grew out of a yearlong workshop designed to help the district’s teachers and administrators learn ways of making more productive use of student test results and other data.Experts say the book’s partnership between researchers and educators could help bridge the gaps that exist between testing and instruction...
At the start of the school year, educators often describe feeling overwhelmed by the amount of data and where to dive in...So the workshop and the forthcoming book are structured around an “improvement cycle,” with the tools to use at each step along the way.
While the process might look different in different schools, the cycle helps educators: identify patterns in data; choose key issues to investigate; dig deeper into multiple data sources; agree on a problem and explore its causes; examine current classroom practices; draw up a plan to change those practices; carry out that plan; and then assess the results of those actions.
Schoolhouse Rock meets the SAT:
Pop culture references, jokes about dating and female empowerment messages - "Not Too Scary Vocabulary'' ($45.95) sure isn't your parents' SAT-prep program. Renee Mazer, a University of Pennsylvania Wharton School graduate and a standardized-test tutor for 17 years, uses poems and songs to help students study vocabulary in this seven-CD set. "I watch TV, what kids are watching, listen to the radio. Kids like 'That '70s Show,' and they are listening to the music I'm listening to. All those references are there.'' Some students have told Mazer their scores have jumped 70 to 150 points after using her prep kit.
Hot on the heels of the ridiculous cheating scam one flunking NY teacher tried to pull, Newsday runs an article entitled, "Don't Stop Now In Testing Teachers."
For years, the city employed hordes of teachers who had been in the classrooms for years, even though they never had been able to pass the teacher certification exams. One exam tests the teacher's general knowledge of basic skills such as reading, writing and math, while the other tests the teacher's knowledge of the subject he or she teaches...Wayne Brightly was able to remain in the classroom for 13 years because he was hired before the new rules took effect and because he got several extensions of his temporary teacher's license. But he hadn't passed the exam in his subject area. His last extension was due to expire in August.
The educators I talked to said the certification reforms have improved the quality of teachers over all, aided by programs such as Teach for America and NYC Teaching Fellows, which have drawn new recruits by making teaching seem a cooler and more professional job. The written exams matter because research shows that students perform better when their teachers have high verbal skills and a mastery of the subjects they teach.
I don't know which specialty exam covers Brightly's subject area, but here's the Liberal Arts and Science test (LAST) is here.
Despite recent criticism by UC officials, the College Board trustees voted overwhelmingly to continue using the PSAT as a scholarship qualifier. The arguments used by those opposing the PSAT are exemplified here, where one critic makes it clear that he doesn't consider it fair that students from wealthier homes do better on tests.
Apparently, recent NY DOE practice math items were a lesson in spotting errors. Unreadable graphics, misspellings, items with no right answers...not a pretty sight. (The Powerline guys are succinct: "I assume that pretty much all fourth-graders can spell 'fourth.' So who in the world writes this stuff?"
Let me know if you can make heads or tails out of this coverage of an education-related speech by Professor Joel Spring of Queens College. I can't, especially the part about how competition in schools and intellectual freedom are somehow mutually exclusive. (Free reg required.)
In Northview, MI, they're using cool cereal bowls as part of the breakfast of champions, in order to facilitate learning (and improve test scores).
Another op-ed sees a link between increased emphasis on test scores and the Red Lake School massacre. I agree, though, that schools need to help students feel safe emotionally as part of the academic environment.
In Colorado, the number of "conscientious objectors" against standardized testing is on the rise:
[Brentwood Middle School principal John Diebold] said that 10 of the school's 670 students opted out of this year's standardized tests, more than any previous year. He has to honor the parents' right to choose while dealing with the negative affects the decisions have on his school.When a student opts out of the test, the school receives a minus score for that test, which affects the "report card" the school receives from the Colorado Department of Education. Diebold said he uses the test results to gauge what they are doing well and what they need to improve.
"When parents have their kids opt out of the test it's hard to get the total picture of what your school's capabilities are because your whole school is not testing," Diebold said. "There are no legal ramifications, but it wouldn't be something I would advise."
Antoinette Medina is one student who is opting out; her mother was apparently influenced by testing critic Don Perl, who claims the Colorado State Assessment Program exams (CSAP) create a gap between the "haves" who do well, and the "have-nots" who don't. It's odd, isn't it, the way the test itself is seen as the stumbling block, as though if the test were removed, there would no longer be a gap between those who learn basic skills and those who don't. All that would vanish with the test's removal would be an objective way of assessing the amount of material students have learned.
As for whether or not the CSAP items measure learning, I've yet to see the test's critics present any evidence suggesting that the test items are not valid for the purpose for which they're being used. There are plenty of released items online if you'd like to check them out for yourselves.
Secretary of Education Margaret Spellings defends tests in the Hartford Courant:
Connecticut Education Commissioner Betty Sternberg has asked the U.S. Department of Education to exempt half of the state's students from annual testing under the No Child Left Behind Act. She said, "Adding tests in grades 3, 5 and 7 ... will tell us nothing that we do not already know about our students' achievement."I disagree. For one thing, it will tell you how well your third-, fifth- and seventh-graders are doing. Teachers cannot remedy weaknesses they don't see. The whole point of assessing students regularly is to catch problems early so they can be fixed before it's too late...
Connecticut has received more than $23 million to develop its assessments. Commissioner Sternberg claims the assessments for grades 3, 5 and 7 would cost Connecticut another $41 million. This estimate is off the mark. It includes costs either unrelated to testing, such as "curriculum adjustment" and school choice, or met by the federal government already, such as professional development. The testing mechanisms are in place - they simply need to be applied to the rest of Connecticut's schoolchildren...
Former Clinton administration official and state Education Commissioner Gerald Tirozzi, who now leads the National Association of Secondary School Principals, called it "two Connecticuts: separate and unequal." Students were misdiagnosed, victimized by low expectations and hidden behind district-wide averages - out of sight and left behind.
President Bush saw this for what it was: unacceptable. Today, nearly every state has reported improved academic performance, with minority students and urban schools posting some of the greatest gains. Thanks in part to No Child Left Behind, the pernicious achievement gap is finally beginning to close.
Not content with picking on the SAT, UC officials are now going after the PSAT as well:
...officials at the UC system have a new target: what they perceive to be the National Merit Scholarship Program’s overdependence on the the SAT’s cousin, the Preliminary SAT. A forthcoming article in National Crosstalk reports that a faculty committee at the university has recommended that the system’s campuses stop awarding National Merit Scholarships, and that a longtime UC administrator at is asking the College Board to break its ties to the program.At the core of the university’s objections is the belief that by using students’ scores on the PSAT examination as a strict cutoff for whether they qualify as National Merit Semifinalists, the merit scholarship program discriminates against black, Hispanic and American Indian students and students from low-income families who, on average, score significantly lower on standardized tests than do their white, Asian American and more-privileged peers.
Classic misunderstanding of test bias. The results of using PSAT score as a cutoff don't agree with the politically-correct version of how UC officials think the world should work; thus, the test is allegedly unfair.
There is nothing wrong with asking, as some UC officials and former officials are, if the PSAT is a valid test in this situation. It's true that it was not designed (to my knowledge) to be used as a means to obtaining a scholarship. However, it measures basically the same skills as the SAT, and thus it is an early measure of students who should be on the college prep track, and who could most likely use the money. (Full disclosure: I was a National Merit Scholar.)
College Board officials could not be reached for comment on Sunday. But Wayne Camara, the board’s vice president for research and development, told Crosstalk that the process by which the National Merit Scholarship Program and some individual colleges winnow the 16,000 semifinalists down to the 8,200 students who actually receive National Merit Scholarships each year takes factors other than PSAT scores into account. “The practice that National Merit is following is very consistent with the requirement that they use multiple sources of information in making a high-stakes decision,” Camara told Crosstalk.Camara also said in the interview that because the PSAT has been shown to be valid in predicting students’ SAT scores, the SAT’s validity in predicting student performance extends to the PSAT.
It's nice of the Colloge Board to point that out, but the heart of the complaint here is that not enough of the under-represented groups get scholarships when the PSAT is used for selection. It's disappointing to see the test being attacked in this situation, rather than a school system in which minority students are overwhelmingly short-changed. We can certainly discuss predictive validity and cutoff ranges, but those attacking the test would do well to step back and ask themselves just why so few minority students (and poor students, allegedly) are able to score at the high percentiles required for scholarship selection.
Touchy-feely educators, listen up!
Youth Focus - - Annchen KnodtAnnchen Knodt of Brenham is a senior at Brenham High School. She is the daughter of Tom and Diana Knodt and has a sister, Anya, 16. She has performed volunteer work at the hospital Wednesdays for the past two years. She is a member of Grace Lutheran Church, where she has played the handbells in the church choir for the past two years...
Q: What is your least favorite thing about high school? "Standardized tests."Q: What is your biggest accomplishment? "When I aced the SAT test."
Interesting. She doesn't like the standardized tests in schools, yet her biggest accomplishment - and this from someone who plays handbells, served as drum major, and does useful volunteer work - was acing a standardized test. I don't understand. Haven't we heard all along from the educrats that children and teenagers suffer irreparably when asked to do things they don't like, or things that might be too challenging? Haven't we been told that self-esteem comes from being told you're perfect just as you are, rather than from things you do? And yet here we have someone who acknowledges that the high point of their life, so far, was successfully tackling something they didn't like very much at all, but which was important, and needed to be done.
Here we have someone who gets it.
Testing of preschoolers for the Head Start program regularly comes under scrutiny; this Newsday article is the latest example. Thankfully, no tears are featured.
...many Head Start staffers on Long Island and elsewhere question whether children so young should take standardized tests at all. Scores now are rolling in from the U.S. government's first-ever tests of more than 400,000 preschoolers nationwide -- one of the biggest federal educational assessments in history. The new tests are the brainchild of the Bush administration, which seeks to measure whether the $6.8-billion Head Start program is doing its job in preparing impoverished youngsters for school. Nationwide, the 40-year-old antipoverty program enrolls more than 840,000 children, ages 3 to 5, including about 2,000 on the Island. So far, test results look mostly positive...Established in 1965, Head Start started out focusing on children's socialization skills, such as following directions. The program gradually has grown more academic. Skeptics in the education field wonder if this particular test is the best way to measure performance, though test sponsors say it serves as a useful reality check in combination with other assessments.
That's $6.8 billion, with a "b." I'd say a "useful reality check" is necessary to make sure all that dough isn't going down the drain.
The news today is all about the new SAT, which will be administered for the first time next Saturday, March 12. Many newspapers are carrying articles like this one (featured at Inside Bay Area) on the pros and cons of the SAT in admissions.
Spring's a big time in testing whether or not there's a new SAT. The FCATs start on Monday. The benchmark exams in Arkansas are next week. The Ohio Graduation Exam will be given on March 14, and will count for the class of 2007. I'm sure there are plenty more I'm not listing here.
One bit of news with an interesting twist - the University of North Dakota, and North Dakota State University, will not require the new essay portion of the ACT for admissions. Mainly, it's because they're interested more in developing their own essays - and not penalizing students who took the old ACT without the essay - but I found this comment from the student body representative to be amusing:
In November, the North Dakota State Board of Higher Education approved a new policy asking the state's 11 public universities to set the requirement. However, the board left the door open for schools to set exemptions. UND and NDSU's stop-gap approach was needed because the schools didn't have enough time to get formal, more permanent exemption policies in place. Many high school juniors have already taken the ACT without the essay...Jordan Schuetzle, UND student body president, said he's pleased with the UND administration's response. UND's Student Senate opposed the essay requirement because of its cost and because it added a layer of subjectivity to a standardized test. Schuetzle added that the requirement is unfair because it gives an advantage to larger, more affluent school districts with means to hire quality writing teachers and coaches. He said also it might impact enrollment.
If I read this right, Jordan believes it's unfair for people who write better to have the edge in college admissions, all those remediation classes be damned. He seems to think that the ND administrators agree with him, but it sounds like they're planning their own much more substantial writing exam.
In Tucson the natives are restless:
Legislators are wiggling worse than this year's high school juniors over the status of AIMS as a graduation requirement. The class of 2006 will be first to face passage of Arizona's Instrument to Measure Standards as a prerequisite to graduation.The kids get five tries. They already have taken the tests twice, and about half of them didn't pass all parts of AIMS. Then again, they took the tests as sophomores. They have until senior year to pass all parts. And they know that they must conquer AIMS to graduate.
Or do they? Some lawmakers insist the kids should get diplomas without passing AIMS. So instead of studying, many juniors may be crossing their fingers and biding their time.
Talk about mixed messages. Reminds us of the mom who tells her son he can't have the car till he finishes his homework, then tosses him the car keys. Please. Our kids deserve consistency - and not just at home...
The test series isn't perfect. No standardized test ever is. But we likely will spend eternity refining AIMS to better reflect evolving state standards. In the meantime, the least we can do is keep a strong and certain stance with our students. They know what to expect.
And our lawmakers have chosen precisely the wrong year to mess with those expectations.
What's gotten Tucson editorialists so steamed? My guess is, events like this:
Disabled and special-education students would be exempt from passing AIMS to get a high school diploma under a bill approved overwhelmingly yesterday by the House K-12 Education Committee. Students who meet other criteria would be exempt: those who had near-perfect attendance, completed all required courses with C grades or better, took AIMS every time it was offered and participated in senior-year AIMS tutoring...The legislation is a watered-down version of a bill (HB2294) that would have retained AIMS, a reading, writing and math test, but eliminated the graduation requirement.
The editorialists aren't the only ones aggravated by this:
State Senate President Ken Bennett said he would prevent his colleagues from debating a proposal to let many high school students graduate without passing the AIMS test...The lawmakers sponsoring the plan say the exam has taken away autonomy from local school boards and that they will continue pushing for their plan. They appear to have the support of about half of the Senate and a large majority of the House.
Bennett said he would halt any attempt to dismantle AIMS as a graduation requirement unless there are other legitimate alternatives in place, such as a specified score on college placement exams such as the Scholastic Aptitude Test...
Schools chief Tom Horne isn't kidding around, either:
The class of 2006 must pass the high school AIMS exam to graduate, and now Arizona schools chief Tom Horne wants each student's AIMS scores stamped on his or her diploma.Critics say the proposal would unfairly rank kids, and students would be less likely to show off their diplomas to families and friends on graduation day. Horne says it would motivate students to study harder and reach their potential. He will ask the state Board of Education to consider the plan when it meets Monday...
Under the proposal, districts would be required to place a sticker on each diploma, showing a student "met the standards" or "exceeded the standards" on the math, reading and writing sections of the exit exam. For example, kids who "met the standards" on last year's math section scored 70 percent to 83 percent; those who "exceeded the standards" scored 84 percent or above. A graduate who exceeded standards on all three sections and made A's and B's in college prep courses would get a "high honors" sticker.
...and you can just imagine what types of responses this plan got:
Linda Ronnebaum's stepdaughter is a junior at Deer Valley's Mountain Ridge High School, and Ronnebaum fears the stickers would become a competition and a stigma. "Once you start putting labels on diplomas, you're labeling children," said Ronnebaum, an administrator at a West Valley elementary district. "Let's make them feel good about getting their diploma, not comparing what stamp you got on it."Mary Silva and Louise Silva are not related, but both have children who passed AIMS and are juniors at Agua Fria High School in Avondale. They had very different reactions to adding the sticker to their children's diplomas. Mary Silva worried that such individual labels would make a child "feel better or less than someone else."
"I know there are some children who study and study and just pass," Mary Silva said. "How does that make them feel?"
Let's take that thought even further. What about those students who study and study but don't earn a diploma? Aren't we all making them feel bad by giving those who pass a diploma in the first place? Shouldn't we have a piece of paper to give the failers to put up on their walls - something along the lines of, "You're perfect, just as you are!"?
Seriously, people. We're not talking about children here. We're talking about young adults - some of whom might enter military service immediately after leaving high school - who, if they haven't already, will soon realize that the world out there is going to let them know just where they stand in comparison to everyone else.
Education author Bernard Chapin (who's been featured on here before) hits the nail on the head in a description of why touchy-feely educrats don't like standardized tests:
I had a chance to look at one of the Illinois books that involve “teaching to the test” the other day. It was quite informative. It contained exercises in phonological awareness that are intrinsic to any child’s learning how to read. I was rather impressed. The real thing that progressives hate about standardization is that it takes warm and fuzzy out of the process. All that is left is what you know and what you don’t know. Standardization merely compares the performance of an individual to that of a larger population. That’s it. Yet, comparisons of any kind are despised. It takes all the excuses and variables out. If a child cannot read as their peers do then is their manner of reading acceptable? No. Is it what we would call reading at all? No. That the testing of students is counter-productive is a widespread belief in our schools...
When I taught at the university...I had a student tell me that her brain did not think that way in regards to the multiple-choice examination that she got a C on. What processing deficit could she have had possibly had? All she had to do was select the right answer amid three wrong ones. It was child’s play, at least many years ago it was child’s play, now children, and some adults, are not expected to do it.
Emphasis mine. Standardized tests cut through a lot of the sheep dip in "progressive" educational techniques.
Maeghan Gibson is fed up with the state's standardized test. Focus on the Texas Assessment of Knowledge and Skills is encroaching on classroom learning, Gibson said. Instead of having high-level discussions, the Haltom High School junior honors student said she spends too much time taking practice tests and filling out work sheets.
So she and a few friends waged a silent protest Monday morning outside the school by handing out pre-sold green T-shirts with slogans including "Walking standardized test score," "I am not in the equation of my education" and "Total Annihilation of Knowledge and Skills."
I want one of those "Walking standardized test score" shirts. And, frankly, students should be free to wear them - even during test time.
The students say their protest was not aimed at Haltom High School, teachers or the Birdville district, but rather at state and national policies that require the standardized test.
"It's turned into a real 'Grab a work sheet, go sit down and you have to know this or you will fail' kind of thing. That's not good for long-term learning, in my opinion," junior Chase Robinson said. "We want our teachers to advance our knowledge, not a test."
And that's commendable. But long-term learning doesn't often happen in the absence of basic skills, which is what these types of tests are intended to measure. It's not surprising that some of the more bored students are fed up with all the tests, but it would be pretty tough to, as one student suggests, create a test that measures only whether a student improves, and not whether they fail.
Poptarts and pajamas help kids prepare for standardized tests:
At Sharpsburg Elementary, girls and boys giggled at each other's pajamas and slippers on the way to class Saturday morning. A couple of boys lugged boxes of Pop Tarts and fruit drinks to three classrooms, as about 30 fourth-graders settled into chairs or on sleeping bags and played "Survivor" and "Who Wants to be a Millionaire."
Teachers in pajamas were game show hosts, pulling questions out of pillowcases. The questions were from old Ohio proficiency tests and samples of the exams the students will take next month. Another class practiced for the math test by building geometric shapes with marshmallows and toothpicks.
Who says cribbing for standardized proficiency tests is dull?
Sharpsburg prepares its fourth-graders for next month's high-stakes, statewide tests by playing games on Saturday mornings.
It seems to be working. After three years of Saturday pajama parties, Sharpsburg went from not meeting statewide standards to meeting them all and earning an "excellent" designation last year, said Principal Brad Winterod.
Next up, "Truth or Dare" with math questions.
Devoted Reader Daryl sends along this article about the new SAT:
I've been a high school English teacher for 10 years, and if there's one thing I hate worse than the SAT, it's the idea of a new SAT...It's not that I'm against assessing kids. I give my own students eight to 10 assessments each marking period, though my assignments don't look anything like what students encounter on these high-stakes national exams...
The entire "writing" section of this new test is the kind of assessment that most teachers of writing would run away from. First of all, the idea that during the writing of this blitzkrieg essay..."You should take care to develop your point of view, present your ideas logically and clearly, and use language precisely" in under half an hour and under extreme pressure is ridiculous. We're not talking e-mail here. This article of mine you're reading now, for example, took several hours to compose - not to mention the fruitful give and take between the paper's editors and me. That's how real writing gets done...
Second, the slew of multiple-choice questions about grammar that the College Board calls "improving sentences and paragraphs" is not what Shakespeare had in mind when he dipped his quill in the inkwell before sitting down to edit a draft.
From the board's official Prep Booklet, here's the first example of what to expect [each letter is a point at which there is a possible error]:
"The students (a) have discovered that (b) they can address issues more effectively (c) through letter-writing campaigns (d) and not through public demonstrations. (e) No error."
This sentence appears OK to me, even if it is a little clunky. According to the College Board, however, the error occurs at (d) because: "When a comparison is introduced by the adverb 'more,' as in 'more effectively,' the second part of the comparison must be introduced by the conjunction 'than' rather than 'and not.' "
Got that?
But if I were to edit this sentence, I might make a few more changes: "The students discovered that they can address issues more effectively by writing letters than by demonstrating publicly." But, hey, I'd be wrong because this is not the portion of the writing section where I'm allowed to write anything.
What the bulk of the writing section of the new SAT is really measuring is acquired skills in managing style within the realm of standard written English...Students would be better served by consistently reading the commentary section of the local newspaper - and then periodically writing letters to the editor - than by sitting through the painfully boring lesson plans that these changes to the SAT are likely to inspire.
I agree. And all those testing critics out there should be taking notes - THIS is how effective test criticism should be done. No hyperbole. No hysterical rants about how the exams are completely biased or dependent on one's social standing. No childish arguments about how evil it is to hold students to objective standards.
It's completely legitimate to worry about the impact of the new SAT on writing curricula. It makes sense to put the horse before the cart and suggest changes that schools can make to improve writing skills. And portfolios, though expensive and laden with plenty of psychometric challenges, are not a bad idea for writing assessments.
We've all heard about the test score gap (aka, the achievement gap), and Andrew Coulson of the Mackinac Center has a provocative article about how ideology helps the gap persist:
As researchers know all too well, there is still a gulf of more than 220 points between the SAT scores of white and black students, and black children trail their white peers by significant margins on every subject tested by the National Assessment of Educational Progress. Many people are likewise aware that Michigan performs even worse in this regard. Across grades and subjects, Michigan’s racial achievement gap on the NAEP is four to nine points larger than the gap nationwide.[1]
But there is one aspect of the achievement gap that is almost universally unknown: how it differs between public and private schools.
Coulson uses NAEP data to reach a conclusion that isn't really surprising:
As the table shows, there is a sizeable achievement gap between black and white fourth-graders in both public and private schools. It is also clear that the private-sector racial achievement gap is narrower at the 12th grade than at the 4th grade in all of the core NAEP subjects. Public schools actually see a larger race gap in both writing and mathematics at the 12th grade than at the fourth.
Averaged across subjects, the public school racial achievement gap is virtually unchanged between fourth and 12th grades. By contrast, the gap in private schools is an average of 27.5 percentage points smaller at the 12th grade than at the fourth.
Note that the achievement gap does not close faster in private schools because white private school students lose ground with respect to white public school students as they move to higher grades. Rather, the gap closes because black private school students have learned at a substantially higher rate than black public school students.
I like his concluding remarks:
So, will the NAACP and other groups avowedly committed to reducing the racial achievement gap act on these findings? Will they compete with one another to discover the best way of bringing nongovernment schooling within reach of all children?
The answer, obviously, is no.
Why?
Because while these groups are committed, on some level, to the aims they profess, they are handcuffed by a self-destructive political ideology. Yes, they will say, we should do everything we can to close the racial achievement gap, as long as our efforts stay comfortably within the confines of a state-run education monopoly.
Given the choice between actually narrowing the racial achievement gap and remaining ideologically pure, they will chose ideological purity. Sooner or later, this position must surely crumble under the weight of its own immorality.
Emphasis his. He also notes that the dropout problems with public schools probably give a boost to the public school scores, because the worst 12th-graders aren't there to take NAEP. Yet, the private schools look better.
As for his belief that this position will crumble soon, let's just say I'm not holding my breath for that, any more than I'm holding my breath for the eradication of the racist belief that minority children cannot be expected to do well on standardized tests.
Test prep reaches the little ones:
Steve Gold and his son Austin studied an hour every school night for weeks to prepare for the big test. It wasn't a final exam or an SAT.
Austin is a fourth grader in Rockland County. They studied for the New York state assessment test on English Language Arts given this week.
"It was a lot of work. It wasn't easy," Steve Gold said. "But I'm glad that the school put the emphasis on it because I do believe it's going to make test taking easier as they get older."
Welcome to test prep - grade school style.
You can tell this article isn't appearing in the New York Times; if Winerip were reporting this, there would have been tears aplenty by this point in the article. The NY Newsday, however, sticks to the facts:
As standardized tests proliferate at the lower grades, study guides and programs have filled a growing market...The market is expected to expand when standardized tests for grade schoolers become the rule this fall under federal No Child Left Behind legislation...
New York school students take assessment tests beginning in fourth grade math and English Language Arts. In New York, assessments also help schools identify problem areas of specific students and in instruction, though they are not used to decide whether a student advances or is held back.
So is test prep at home even needed?
Good question. I say yes. The parents need reassurance, and the kids need practice. Good test prep books will be able to fill both sets of needs. The critics, of course, say that the test prep must inevitably be coming at the expense of "education," as though learning to read and learning how to take a reading test cannot possibly cover the same constructs.
Some parents won't buy the books, because they'll see their kids don't need them. But these books could be a boon to a parent who is unfamiliar with the tests and/or believes their child needs extra help.
The New York Post has the goods on the new but not improved Math A Regents exam standard :
High-school students taking the Math A Regents exam this week must correctly answer fewer than one-third of the questions to pass — the lowest benchmark in at least six years — because of a revised grading scale that critics charge is too generous. Students are required to earn just 26 out of a total 84 points — or 31 percent — to reach the minimum passing grade of 55.
To pass with "honors," students only need 34 points — or around 40 percent.
The state Department of Education insists the scoring is just as rigorous as previous exams and consistent with recommendations of the independent panel that devised the test. "There are somewhat more difficult questions on this exam, however, and so students need to get somewhat fewer questions right in order to pass," said state education spokesman Tom Dunn. "Anyone who looks closely at this exam will see that it is not easy."
I disagree (as does the Post), and for one simple reason. We heard a similar argument a while back, in the discussion of the UK maths GCSEs. If you recall, I said then that there might indeed be no problem with a very low passing standard, whereas here, I say there is.
The difference is that, while the UK exam is completely open-ended, the Math A Regents exam contains thirty multiple-choice items (or MCQ's, as we call them) that are worth a total of 60 out of the 84 total points - 71.4% of the exam. If the passing standard is set at a raw score of 26 points, or 13 MCQ's answered correctly. A student can pass this exam by getting 13 MCQ's and no open-ended items right, because all items are combined and scored; the conversion is done only on that final combined number-right score.
Mere guessing, at four options an item, gets a student up to 7.5 items by chance alone. It doesn't matter if the MCQ's are more difficult; no matter how difficult they are, students can still get the right answer by chance alone. So a student need get only an additional 6 items right in order to pass. They can completely bomb the open-ended part and pass. And calculators are allowed. And this represents an increase in the number of MCQ's on the exam compared to 2003. And we haven't even gotten into discussing the generous conversion table.
I'm sure you can guess the reason why the passing standard was lowered, by the way. They've been very concerned in NY over the miserable passing rates on this exam. This solution doesn't exactly address the root causes.
Others who have more math knowledge and understanding of these exams than I have been appalled by Regents math exams in the past. My guess is that they're still appalled. If you haven't visited the NYC Hold website, you should. It's a great read.
(Via Joanne Jacobs.)
A test-prep author makes the news:
...Because of the changes [in the SAT], previously used methods to taking the exam were cast aside, opening the door for Christopher Black, an independent education author in Greenwich, who has since revolutionized the way students approach the SATs.
In June, he published McGraw-Hill's SAT I, which is currently the No. 1 best-selling SAT I preparation book, surpassing the Barron's version...which takes the "buckshot" approach by trying to categorize questions on the test. Black, sole proprietor of College Hill Coaching, takes a different approach. He developed the College Hill Method, which "uses systematic lessons to reinforce the fundamental academic reasoning skills that lead to success in college as well as on the SAT I," he said.
"I did it because students, teachers and parents need a serious and smart guide to the SAT," said Black, 39...
It took Black more than a decade to develop the materials for the books. He received help from his business partner Mark Anestis and various high school teachers. "(The teachers) have told me that there is a real dearth of good materials to use in class to help students with the SAT," said Black of his SAT I book. Black feels that the loss of analogies from the SAT exam will render the "crack-the-test" approach, used by such companies as Princeton Review, "utterly worthless." The "crack-the-test" approach was created in the early 1980s by Ivy League business school graduates who viewed the SAT as somewhat of a joke that could be aced by anyone that knew the "insider's tricks," according to Black.
"Of course they were wrong," said Black, who used to substitute teach at Greenwich High School and Central Park East before receiving his master's in education from Columbia University. "But their approach sold a lot of books and prep tests"...
Yes, it certainly did. But Black's approach, so far, seems to be showing results, not just profits. Not surprising, considering that it focuses not on tricks and insider tips, but on "reinforc[ing] the fundamental academic reasoning skills" used on the exam. In other words, by actually teaching would-be test takers something useful.
The bar keeps getting set lower as the kids who make perfect SAT scores keep getting younger:
A 13-year-old boy has scored a perfect 1600 on the SAT, a standardized test usually given to college-bound high school seniors.
"I was pretty surprised and happy," said Lee Kennedy-Shaffer, an eighth-grader at Mechanicsburg Middle School. "I did not think I would score that high."
He got the perfect score for a test he took in December as part of a program for gifted children. He wasn't the first in his family to get a perfect score, but he was the first to do so at such a young age.
In June 2003 his brother Ross scored 1600 on the SAT as a junior at Mechanicsburg High School. The oldest brother, Alan, had 1520 on the exam.
Oh, I get it. This kid was just NOT going to let his older siblings show him up. Good for him. And why wasn't he chosen to play Harry Potter? (He already owns a magic wand - his pencil.)
On a related note, I wonder if anyone (other than ETS/College Board) has kept track of the number of students who achieve perfect scores each year? And what would it mean if that number has, for example, drastically increased as of late? The result of "dumbing down the test?" The result of those who might do poorly being more likely to bypass the exam for schools that no longer require it? Or the natural by-product of an across-the-board increased focus on standardized testing?
FairTest has been posting SAT score tables on its website which show that poor and minority students do worse on the exam than white, wealthier students, and the College Board isn't happy about it. It's a matter of debate as to whether this is an indictment of the tests (as FairTest maintains) or of our educational system (as I maintain), but, interestingly, that's not really the point of the latest controversy:
The nonprofit College Board, which owns the SAT college entrance exam, has demanded that its chief critic remove from its Web site data showing that minority and poor students scored lower than white and upper-class kids. The letter to the National Center for Fair and Open Testing, also called FairTest, claims the Cambridge-based nonprofit organization violated copyright law by posting the scores without permission.
"Unfortunately, your misuse overtly bypasses our ownership and significantly impacts the perceptions of students, parents, and educators regarding the services we provide," stated the letter, signed by College Board legal affairs assistant director Tasheem Lomax-Plaxico.
FairTest, which opposes an overreliance on standardized tests, posted the Oct. 27 letter on its Web site along with its refusal to comply with the College Board's demand. FairTest argues that the data is widely available in the public domain and therefore not subject to copyright protection.
Needless to say, this gives FairTest the opportunity to get up on their soapbox about how CB don't want any negative information about their test to be released, but the public is already quite aware of test score gaps.
Critics long have attacked the tests as unfair, chiefly because white students tend to do better than other groups. Many reasons are offered -- family income and education, school quality, courses taken, access to tutors and test-prep courses.
If they've "long attacked the test as unfair," it's hard to believe that CB is really trying to "hide" something from us now. This sounds like one for the lawyers to sort out. It's interesting, though, that the AP doesn't see fit to point out that three of the six "unfair" reasons above - courses taken, access to tutors and test-prep courses - are directly related to the amount of effort a student is willing to put into their education. Saying it's "unfair" for a student who takes more difficult courses, spends extra hours with tutors, and prepares for the SAT, to receive a higher score is ludicrous.
If anything, the list of reasons above suggests a great deal of validity evidence for inferences made with the SAT, because we'd expect those with more money, better schools, harder courses, and more preparation to do better. It's not the score gap that is seen as unfair so much as the disparate opportunities that lead to it.
Mobile Register reporter Rena Havner tries hard to make the scenario of "teaching to the test" seem disastrous, but it honestly doesn't seem to me like the kids are being disadvantaged by an increased focus on long division:
Two years ago, before the Mobile County school system began administering standardized quarterly tests, third-graders weren't taught long division until the last two weeks of school, [teacher Cindy] Naylor said. Now, with the federal No Child Left Behind Act and a greater emphasis on standardized testing, long division must be taught months earlier, before the children are mentally ready, Naylor said.
The kids may have been excited on this recent day, but they weren't fully grasping how to work the problems on their own, Naylor said. Some still hadn't passed timed tests showing that they knew how to multiply small numbers by 3, and many were still having to use their fingers to work one-digit multiplication problems.
Naylor, who has taught for 24 years, has told the Mobile County school board that too much emphasis is being placed on standardized tests and that children, and their grades, are suffering...She said she now has to "teach to the test," meaning that her lesson plans revolve around the subject matter covered on standardized exams that the system uses to evaluate teachers and that the state uses to rate schools.
Are we supposed to assume that a test for third-graders is bad if it requires they learn long division? I think so, but it sounds to me like many of Naylor's students are benefiting from this new structure:
She undertook plenty of tactics to try to make the long math lesson sink in: To prepare the children for long division, she read a story about ants that must find a number by which 25 can be evenly divided before they can march in a parade.
They played a math game: The students stood in front of the classroom, pretending that they were 17 cupcakes that must be divided evenly onto three platters. They soon realized, with a bit of confusion, that it couldn't be done without leaving two cupcakes off to the side.
The children helped Naylor work problems on the board using a funny story to help them remember the steps. She also gave them four problems for homework, and some clamored for more.
Some mornings and afternoons, Naylor has bus duty, meaning she has to leave her class at 3 p.m. to stand in front of the school to monitor things. On this day, though, Naylor stayed in her class until children could begin filing out at 3:30 p.m. First, the car-riders lined up, with Naylor asking each to solve a multiplication problem before leaving the room.
After she had walked them down the hall and out the door, she returned to occupy her bus-riders. Like the car-riders, each bus-rider had to solve a problem.
"What's three times seven?" Naylor asked.
"Ummmmmmm," a child responded.
"Go to the back of the line," she said.
"What's nine times eight?" she asked another.
"Seventy-two!" the boy said, bragging to his classmates. "That was a hard one, but I got it right."
Remember, this is a kid who wouldn't have been taught long division until sometime next year. And now he's thrilled about the fact that he can do multiplication in his head, on command. I realize Ms. Naylor does have a lot on her plate, and I'm sure the paperwork associated with NCLB can be overwhelming. But I don't get the feeling that her students are suffering because of it. Ms. Naylor feels she's not teaching as much "constructive" information any more, but the lesson plans above suggest she's doing a bang-up job in getting her students to understand the rules of mathematics. It's hard to see where the problem is with that.
Howard University's newspaper, The Hilltop, has an article about the "unfairly" high prices of test prep courses:
With many juniors and seniors preparing to take the Graduate Record Exam, there are many complaints about the high cost of preparatory classes. The GRE is a standardized test that provides graduate schools with a way to compare qualified applicants for admission and financial aid. Many students take these prep classes to help them get an idea of the format of the tests, to get practice and to help them maximize their scores. The prices of these classes can range from $400-$2,200 for 15- to 35-week courses.
These prices are out of the budget range of many college students who have to work and save or depend on family members to give them the money. This may force many students to opt to study for the exam on their own instead of forking out the several hundred dollars for a structured class.
Is it just me, or does that sound like we're supposed to believe that being "forced" to study for the GRE on one's own is a bad thing? It seems to me there is at least one erroneous assumption at work here, which is that there are special, hidden tricks to these standardized tests that only the test prep coursemakers know. What's more, these tricks must be very valuable, because Princeton Review is charging so much to reveal them.
I believe nothing could be further from the truth. The reason PR and their colleagues get away with charging this much is because students are willing to believe the anti-testing hysteria that's shoved down their throats daily. If you were convinced that psychometricians specifially write test items that female or minority students can't understand, you'd be rushing to sign up for test prep too, and hang the cost.
What's more, these aren't children we're talking about here, but young adults who are preparing for graduate school. Graduate school tends to require a great deal of self-discipline, logical thinking, concentration, and motivation. Although programs do what they can to support their students, when you come right down to it, the only person who can really shape your path towards a Ph.D is you. Anyone who can't get through the GRE without a lot of expensive outside help might find themselves floundering in the sink-or-swim world of a graduate program.
Hopeful college applicants aren't the only ones who should be prepared when it comes to high-stakes tests. This Globe and Mail reply to a letter-writer concerned about employment personality testing is quite interesting:
Strictly speaking, you can refuse or withdraw from testing at any time; no psychologist worth her salt can make you sit through an assessment against your will unless you're ordered to by the court. But refusing to be assessed in a job competition is declining to play the game by the employer's rules...
Instead of saying no, ask for more information....Here's a primer on questions to ask.
First, make sure that the person administering the battery of tests is a registered psychologist or counsellor....Interpreting the test results in a responsible way is also the duty of a psychologist. Psychologists can't make sweeping statements that aren't justified by the test results or they can be booted out of their professional association.
Even if this rarely happens, it's fair and prudent to ask the psychologist or the employer if you can have some general background about the testing instruments that will be used...To answer this question requires some sophistication about statistics and the role of chance and competing factors when human behaviour is measured.
If asked to recall how many statistics courses were required before they were handed a diploma, most psychologists are likely to respond with eye-rolling, grimacing, tongue protrusion and other signs of disgust.
Well, they got THAT part right. I've known too many school and clinical psychologists who hated statistics, and avoided statistics classes like the plague. They'd take the bare minimum and gripe about it the whole time. Now, I don't expect everyone to be as interested in psychometrics as I am - but still, I find troubling the anti-stats mindset that you see in abundance in psychology undergrads, and to a lesser extent in graduate students.
Not all employment testing is hogwash.
General intelligence, as measured by reputable standardized tests, was recently shown to predict job performance in a large study that analyzed much of the previous research on the topic. This study was published a few months ago by psychologists Nathan Kuncel, Sarah Hezlett and Deniz Ones, and puts to rest the notion that the standard school smarts that are linked to intelligence tests are completely different from street smarts or business smarts.
Intelligence tests are good predictors of job performance precisely because it's your level of intelligence that allows you to acquire job knowledge and skills.
Problem is, it's un-PC to say that nowadays.
In the past, some less-than-motivated students have been able to use a loophole, related to North Carolina's extensive end-of-grade and end-of-course testing, to avoid working consistently throughout the year. Students in Orange County were able to squeak by and pass classes with high test scores - but low homework scores - and now administrators are looking to close that loophole:
Occasionally, an Orange County student who's hasn't done his or her classwork during the school year has wound up passing the grade or class by acing a state-required standardized test.
Now, the county school board is trying to change that. Members are considering a policy that would require students to meet classwork and attendance standards before they could pass a class. The current practice allows students to move on just by showing proficiency on state tests.
"You potentially had students who would do nothing during the school year," said Superintendent Shirley Carraway. "It's just awful. If you think the end-of-grade or end-of-course test is a minimum standard, you could pass and simply eke by."
Board members are now aiming to make promotion dependent on any local requirements as well as test scores.
Newsday wonders whether the high-stakes exams for NYC's third-graders are working:
Nearing the end of a 45-minute intervention session in PS 100 in Queens, Amy Strauss looked into the faces of six third-grade and fourth-grade boys...The children, using a dozen block letters, had been asked to form three-letter words on their white magnetic journals.
"'Pat.' The next word is 'pat,'" the teacher said slowly. "P ... A ... T." Choosing from a selection of nine consonants and three vowels, five of the boys spelled the word correctly. The sixth boy spelled "p-i-t."
"OK, let's tap it out," Strauss said, holding up her fingers so that all the boys could see and rhythmically touching fingers to thumb in succession. "P ... A ... T."
The boy who had misspelled the word, mumbling to himself and following Strauss' example by tapping his own fingers, removed the "i" and inserted the "a" block in its place. "Very good, everyone," Strauss told the group approvingly.
Remember, these are third-graders, who aren't being asked to write out letters, or even know how words are spelled without being told. They're being asked to identify letters of the alphabet from a reduced set of block letters. Is there really any question about whether a kid who has trouble with this should be promoted to fourth grade?
The intervention sessions, held during the school day as part of the students' regular curriculum, are another crucial part of Mayor Michael Bloomberg's and Schools Chancellor Joel Klein's efforts to end so-called "social promotion," to keep failing students from being passed to higher grades.
It's not "so-called" social promotion. When failing students are promoted, it is social promotion.
Citywide, more than 3,600 third-graders were held back -- or "retained," in the Department of Education's words -- after posting Level 1 scores on a city standardized test in reading and math. Level 1 is the lowest of four levels on the test. Students who posted the low scores on the test last spring were required to attend six weeks of summer school and make a higher grade upon retaking the test to go on to fourth grade. In a small number of cases, students who had low scores successfully appealed and were allowed to advance...
City officials have touted the social-promotion policy, which caused an uproar among parents and education advocates for months last year, as a success. This year, the city has expanded the policy to fifth-graders and is offering intervention programs to struggling students on Saturdays.
Third-graders who were held back, those who barely made it into fourth grade and those who were promoted through the appeals process can qualify for help such as that given by Strauss...
Many education experts, however, say the jury remains out on the intervention programs until the citywide standardized exams are given to third-graders in the spring...Some third-grade teachers have had mixed feelings about the intervention programs, saying whatever improvement a student who is getting extra help shows is countered by the chaos of being moved around too much during school hours. One teacher, who asked not to be identified, said he found the interventions disruptive because, among other reasons, the push-in programs took away from his lesson.
I agree that the jury is still out until the test scores come in - even if the idea of intervention sounds good to us, that doesn't mean that what's being taught is effective. However, I think it's absurd to say that, if the intervention work is useful, it's negated by being moved around a great deal. And if the push-in programs take time away from lessons that aren't working, as evidenced by test scores, I don't see where that's a bad thing. Teachers should be kept more informed about how the intervention programs work, for sure, but if classroom lessons aren't working, there's no reason not to try something else.
They're cute. They're hip. They're awfully smart. And they both did pretty darn well on the SAT:
It seems like the kind of SAT question custom-made for Dillon and Jesse Smith of Long Beach: If one out of every 1,511 students taking the SAT will get a perfect score, what are the odds that twin brothers will both ace the test? nswer: No one knows for sure. Nevertheless, that's what the Smith twins have done.
Both Dillon and Jesse Smith, 16-year-old fraternal twins, achieved the elusive top score of 1600, a number most high school seniors dream about seeing on their SAT score report. "I was very, very happy," said Dillon, describing the moment he realized that both he and his brother received the top score on the aptitude test. "I've been hoping for it since we started."
It was a rare thing to hope for. Of the 1.4 million high school seniors who took the test in 2004, only 939 scored a 1600, according to the College Board, which administers the test. With those numbers, the odds of any two people getting that score would be almost 1 in 2.3 million -- and that doesn't even take into account whether those two people are related, never mind twins.
No, it doesn't (although, as my Devoted Reader Maureen points out, the Newsday reporter gets it wrong by implying that the probability would decrease if family relations were factored in). Dillon and Jesse don't have identical DNA, but they did share a tough mom with high expectations:
"I expected it," said Smith, 44, a physical education teacher at I.S. 143 in Washington Heights. "They have the potential to do even better -- maybe even write the tests ."
What? Momma Smith thinks they could become psychometricians one day? Be still, my beating heart. Oh sure, Jesse is quoted as saying that he doesn't think the test really measures anything, but I figure he's just being modest and humble. After all, I've yet to see the article where a perfect-scoring student is quoted as saying, "Yeah, this test really measures how smart you are, and this proves I'm the best!" I know some top scorers think that, but they'd never say it.
I'd always thought it would be a tad surreal to win an Academy Award, or an Olympic gold medal, or some such spectacular trophy for all-around splendiferousness. Surreal because of the rush of adrenaline that comes from all the admiration, knowing you are appreciated - but also knowing that you better get off your duff and do something else so that you don't seem like a flash in the pan.
It's even more surreal, I can now say, to receive a lovely award for which you didn't know you were being considered, and in fact didn't know existed. I fired up my computer this morning to read the following email from Daryl Cobranchi:
Kimberly,
Blog of the Century! Don't get a big head, now. Congrats.
Daryl
Wha? What? Through my blurry and uncaffeinated eyes I stared at the screen. I couldn't figure out what Daryl was talking about. There was an email immediately preceding his with the subject line, "WINNING NOTIFICATION!!!", but even an uncaffeinated hedgehog could've figured out that that was one of those Nigerian-lottery scams. Being not quite as sharp as a hedgehog this morning, I clicked on it, just to be sure:
ATTN;WINNER, We happily announce to you the draws of the Centenary Olympics Big Lottery International programs held on the 25th of OCTOBER 2004 here in Athens, Greece.This promotion takes place every Olympic year You were entered as dependent clients with: Reference Number:NM/BC921245/KY13, and Batch number NM/207161/KOP....
Okay, so it's not that. I went to Daryl's site, and discovered that Education News has declared N2P to be "The BLOG of The Century." They don't say which century; I hope they don't mean the last one:
In Defense of Testing Series
Number 2 Pencil: The BLOG of the Century
Tuesday, October 26, 2004
It seems hard to believe that Kimberly Swygert's BLOG, Number 2 Pencil, approaches its third anniversary. It still seems so new. For those of you unfamiliar, Kimberly is a card-carrying psychometrician who expresses in her BLOG her open, honest, and informed opinions on education policy in general and standardized testing in particular. Before she started, no psychometrician had been willing to do this, in the interest of protecting their careers. We all owe her a debt of gratitude for her courage.
Wow. How flattering. Especially given all of the other edublogs out there (see the list over on the right-hand side of this page if you don't know about them). And they got the timeline right, too - N2P will indeed celebrate its third anniversary in January of 2005. It sounds amazing even to me, especially considering that I thought no one would ever read this blog. Well, I thought maybe some other psychometricians might read it, but no one else.
What a nice way to start the day. A bit surreal, as I said, but nice. Had I known I was in the running for such praise, I would have tried a bit harder and posted a few more testing-related articles, at the expense of a few catblogging posts. But hey! - I made the Carnival of the Cats this week too. So I must be doing well all around.
Now would be a good time ("I'd like to thank the Academy...") to thank those who have provided a great deal of academic, moral, and technical support for N2P, almost since day 1: Joanne Jacobs, Dr. Greg Cizek, Dr. Richard Phelps, John Rosenberg, Dean Esmay, Daryl Cobranchi, and every single one of my Devoted Readers. Without you, this blog would be nothing.
This post will stay at the top of the page for the day; scroll down for new posts.
Thanks to the recent stampede of hurricanes through Florida, some schools may be able to obtain a "storm waiver":
State education officials said Tuesday that some hurricane-ravaged schools can appeal their grades next summer if students show unexpected declines on the Florida Comprehensive Assessment Test. But the same consideration doesn't extend to the students themselves.
The Department of Education's decision came in response to an unprecedented quartet of hurricanes that forced all of Florida's 67 school districts to shut down for at least one day, and shuttered 17 districts for 10 days or more.
Department officials said they didn't want to okay requests from hard-hit districts to simply exempt FCAT grades, because that would imply lowered expectations....But at the same time, he said, DOE wanted to acknowledge that the hurricanes may have an impact on student performance.
"I believe this is a fair approach," he said...
To be eligible for a grade waiver, schools must have been shut down more than five days by a hurricane, have shown good or improved grades over the last three years, and drop at least a letter grade next year. They must also show obvious effects of hurricane damage, such as a high number of dislocated students or classes on double sessions.
One other solution: Change the FCAT to measure how well students know to board up houses, purify drinking water, and plot evacuation routes. You know, the skills children need to best function in Florida.
Fellow blogger Liz Ditz emailed me to point out that students are skipping out on exams that are low-stakes to them, but high-stakes to the schools:
Torrey Pines High School [CA], the academic powerhouse proclaimed by Newsweek as among the top 100 campuses in the country, is in the academic doghouse. Nearly 300 students at Torrey Pines skipped the statewide standardized tests in the spring, and because of low turnout, one the highest-achieving high schools in California didn't receive a statewide ranking, known as the Academic Performance Index.
The index crunches test results into a single number between 200 and 1000, and Torrey Pines' base score of 855 last year was among the highest in the state. This designation doesn't have any consequences for students, but it is high stakes for schools.
Principal Rick Schmitt said the API score is a symbol of a school's academic standing. It affects property values and is used by real estate agents to sell homes..."Individuals feel there's nothing in it for them," said Schmitt of students who are expected to take the tests. "If the community is better informed and once people understand what it means on a bigger scale, they can appreciate why it's important."
Well, yes, but I can see why students skip. Trying to get good data without stakes for the test-taker has always been a thorny issue, and there's not any one solution that works for all tests. When students get fed up with tests to the point of drawing cool patterns on their bubble sheets, something's gotta change. The data are worthless at that point. Students must have an incentive.
They study hard for the SAT and sweat over the APs; they at least show up for the high school exit exam. But yet another test that doesn't matter at all, to them, might be too much.
Devoted Reader nicksmama sent this along with the headline, "Bad test-takers rejoice!":
Diane Smart has seen the stress and strain Praxis I puts on some teachers. For their sake, she's hoping the Virginia Board of Education lowers the scoring standards later this month for the general-knowledge teacher assessment test.
"If you know your job is on the line, you get test anxiety," Smart said in her fifth-grade classroom at Spotsylvania County's Riverview Elementary School. "I feel for people who have failed it repeatedly--and I know they're great teachers."
Smart is finishing a Master of Education degree at the University of Mary Washington. She breezed through Praxis I, missing just two questions on the math test.
Virginia has the highest minimum required scores of the 28 states that use Praxis I. While most teachers pass the state requirements, others struggle.
And...isn't that to be expected? Otherwise, why give the test? Oh wait, I get it - everyone who has a love of teaching should be allowed to be a teacher, right? No matter how much - or little - material they've mastered.
On Oct. 28, the state Board of Education will consider lowering the standards in one or more of the three assessment areas...The standardized test is similar to the SAT. Each section takes about an hour to complete.
The reading section tests comprehension of included passages. Math problems are at about a ninth-grade level. The writing section tests grammar and requires a writing sample.
Scores range from 150 to 190. Virginia demands 178 for math and reading, and 176 for writing.
The average requirements in other states are about 172 for math, 174 for reading and 172 for writing.
Emphasis mine. Care to tell me why men and women with bachelor's degrees shouldn't be expected to do well on a test of ninth-grade math?
Smart thinks the discrepancy is unfair and can force people to take the $130 test multiple times. She points to No Child Left Behind, which is the same in all states, and says Praxis I requirements also should be uniform.
"Unfair"? Really? Why? Because someone always fails it? And I thought teachers disliked NCLB because it imposed government standards on student performance. Now we hear that teachers want government standards on teacher performance?
Despite Virginia's allegedly-unreasonable high standards, it's not like massive numbers of would-be teachers are failing it. But the statistics are telling:
According to 2002-03 statewide data, about 92 percent passed the reading section, and 86 percent scored at least the minimum on math. People fared worst in writing, with 82 percent passing.
If you want to teach children, you should be required to pass any and all writing exams the state puts before you. End of story. Writing well is not an elective skill for a teacher of any subject, at any level. The ACT and SAT are alternate exams - and they both have writing components.
Cal State would like to reduce the numbers of incoming freshmen who require remedial classes, so they're doing early assessment on California juniors. The test used is an extended version of the state standardized exam, and the results are depressing:
Just more than half of California 11th- graders who took an early assessment test were college-ready in math, and only one- fifth were prepared in English, according to results from the California State University's new high school testing program.
Forty percent of California high school juniors this spring volunteered to take part in CSU's early assessment program. Fifty-five percent of math test-takers were deemed college-ready, as were 22 percent of students tested in English.
So these were the kids who were motivated and organized enough to volunteer. Presumably all of them are planning on attending college. And yet only 22% are ready in English? Good heavens. Cal State hoped to identify the problem areas, and they certainly have. What they do with this information now will be the crucial part.
A partnership between the Long Beach-based state university system and California's K-12 schools, the program aims to smooth the transition from high school to college by giving struggling eleventh-graders extra help during their senior year so that they can avoid remedial course work if they are admitted to a CSU...
To help struggling students during their senior year, CSU faculty have developed a new 12th-grade writing and reading course and have set up Web sites offering math and writing tutorials and feedback.
The diagnostic and tutorial services for reading and math that CSU is providing look to be substantial, and solid. I believe, though, that this project is essentially a tacit admission by CSU that college-bound students are no longer expected to acquire the necessary academic skills in high school. Some students have always needed extra help to reach the college level, and certainly high schools do not design every curriculum as college-prep worthy. But a program on this scale is obviously intended to remediate a large number of students who will earn a California high school diploma and expect to attend college, yet have not been adequately prepared.
Is it a better idea than just offering remedial classes in college? Yes. Should the high schools involved feel just a tiny bit of embarassment that this is so necessary? Yes to that, too.
Holy cow. It's the first-ever, "I want to be a psychometrician" article that I've ever seen:
Sara Monempour was 2 when her family moved from Tehran to L.A. Then she did what most new Americans do: learned English. Attending Los Angeles County public schools, Monempour excelled in class but scored "unbelievably low" on standardized reading tests, up to and including the SAT.
Then she noticed most of her bilingual classmates did poorly, too. "We were raised here ... and yet this pattern was always a factor," says Monempour, who spoke Farsi at home. "People who speak a different language at home or with their friends and family would have issues with testing."
Now 23 and a doctoral student at the University of California-Los Angeles, she hopes to become part of a small but growing group of elite researchers, known as psychometricians, who do little else but think about standardized tests.
"Do little else." I love it. I guess that's supposed to seem bizarre to the average reader...well, I guess it is bizarre.
No Child Left Behind, President Bush's education reform law, more than doubled the number of standardized tests schools must give each year, and it very likely will double it again in coming years.
Trained in both psychology and statistics, psychometricians work in school districts, education departments and private testing firms to make sure standardized tests actually test what kids know, quickly, fairly and accurately.
To meet the constant demand, the testing industry is expanding "far faster than the supply of competent people," says Bob Schaeffer of the non-profit Center for Fair and Open Testing (FairTest).
Only about a dozen universities have psychometrics programs. Most turn out only one or two graduates a year. But once they're on the job, psychometricians burn out quickly, observers say. For one thing, new federal requirements say tests given in the spring must be processed before students return in the fall, months earlier than in the past.
Several small quibbles, already:
1. It's an article about psychometricians and the first quote is not from a psychometrician, but from someone who is rabidly anti-testing? (Never mind that, in this case, his quote is correct.)
2. "Burn-out quickly?" Funny, but I associate burn-out with police officers and neonatal intensive care nurses. Every single psychometrician I know is employed as fully as they want to be, and we all work long hours for organizations that are often under-staffed. But I can't think of one psychometrician I know - and I know dozens - who would consider him- or herself to be "burned out."
The rest of the article is fairly informative, including the following:
It's worth pointing out that even psychometricians say such tests shouldn't be the sole criterion determining whether teachers can teach, whether schools pass muster, students graduate or colleges accept them.
"Test scores are limited in what they tell you about a person, and test users don't always keep that in mind," Walker says.
True. Those tossing about overheated statements like "reducing a person to a number" are always testing critics, and usually uninformed ones at that. No psychometricians claim that everything about a person can be measured, nor do any claim that all decisions should be made quantitatively. If your college does well with students who bomb the SAT but can put together a great artistic portfolio, so be it. Don't use the test. Use what works. But don't claim the test is flawed for everyone just because it doesn't work with your population.
On that note, back to Ms. Monempour. I'm thrilled that she has noticed an issue with the tests. I'm thrilled as heck that she even knows what a psychometrician is, much less is actively studying to be one. But I do have one question. Given that she's noticed a pattern with all of her bilingual classmates, is she willing to enter the field of psychometrics with an open mind, and consider the possibility that schools these days may not do a very good job of teaching bilingual students Standard English? Or that some other factor may be at work?
The problems can lie with tests, with the education, or some mixture of the two. I have the feeling that the more involved Ms. Monempour becomes with psychometrics - and I wish her all the luck in that regard - the more she will notice that tests sometimes uncover ugly truths that others do not wish revealed. If she decides the problems are with the tests, I hope she sticks to her guns, and actively researches ways to fix them.
Colleges don't need a new standardized writing exam, claims the Southeastern Missourian:
...is there a need to better test college students' writing skills?
The College Board, which administers the SAT tests, insists it's needed and that mandatory testing will prompt students to be better writers. A 2003 report by a national commission says poor writing skills are found at all levels of education, from elementary school to college.
But many of the schools that rely on the ACT exams, including Southeast Missouri State University, don't see a need for a new college-entrance writing test. Only 17 percent of some 2,000 four-year colleges and universities have told ACT officials that they plan to require applicants to their schools to submit writing test scores beginning in fall 2006.
Southeast admissions director Debbie Below said the Cape Girardeau school already has its own writing assessment program and Southeast won't make prospective students spend additional money to take another ACT exam.
It appears that schools are not so much relaxed about writing as they are more anxious about other subjects:
Despite the new emphasis on writing, a spokesman for the Iowa City, Iowa-based ACT says skill with words is not the biggest failing of incoming college freshmen.
"Science and math are the problems," said Ken Gullette of ACT.
A study of ACT scores shows that 68 percent of test-takers nationwide who graduated from high school this spring already have the academic skills to earn a C or higher in college freshman English classes.
In contrast, only 40 percent of them are academically prepared to earn a C or higher in algebra, and only 26 percent are prepared to earn a C or higher in college biology, Gullette said.
Emphasis mine. According the Missouri Department of Secondary Education home page, the MAP requires Missouri students to know Algebra. I think. The homepage is confusing, and the math framework page is even more so. Page 59 finally gets around to mentioning some concrete examples, but it's hard to tell just how much algebra is tested at the high school level.
Anyway, some Missouri colleges say they have their own writing assessments, but the matter-of-fact description of why just about curls my hair:
Bratton said some colleges like Southeast [Missouri State] require new students to take a writing test to determine what freshman English course students must take. Others determine course placement on the basis of the ACT score on the English part of the exam...
Jon Thrower, who also teaches English composition as a graduate student, said some Southeast students do have difficulty crafting a sentence. Sometimes students leave out verbs or subjects and resort to sentence fragments in their essays, he said.
Reinheimer said 400 to 450 students out of 1,600 to 1,800 new students tested at the start of a school year are found to have problems writing essays. That doesn't surprise him.
"There are always going to be people who don't write very well," he said.
Funny, but I don't remember there always being college students who could not write in complete sentences.
One very disadvantaged school district in Yonkers (NY) is doing well on the state standardized exams, and enquiring minds would like to know why:
Seventy-seven percent of Yonkers elementary school students are now meeting the state's standards. That's compared to 79 percent of all elementary schools statewide -- including other large urban districts such as Buffalo, Rochester and Syracuse.
And results released in June show fourth-grade English test scores climbed from less than 34 percent of students meeting the state standards in 1999 to 65 percent this year.
From the press release:
A sample of the most improved schools, when contacted by the State Education Department, gave these reasons for improvement: an all-out district or schoolwide effort to improve achievement, hiring of math instruction specialists, intensive staff development focusing on math instruction, an improved curriculum in line with the standards, teaching math more every day, setting targets for improvement, using the achievement data to help individual students, grouping students flexibly by achievement level to give them the help they need, and before- and after-school help.
Emphasis mine. Hoorah.
Hurricane FCAT is wreaking havoc in Florida's Pinellas school district - the nation's 22nd-largest school district with over 113,000 students:
In a major shift this year in Pinellas schools, teachers have been told to move much faster through lessons and to narrow their instruction to material most likely to be on the state's standardized test.
Elementary students already have taken two practice tests that mirror the content of the Florida Comprehensive Assessment Test and its fill-in-the-bubble format. Three more tests are scheduled over the next 20 weeks, designed to build students' stamina and to root out gaps in knowledge before the real FCAT in February and March...
The change has touched off a robust debate that is carving the nation's 22nd-largest school system into two camps. One side accepts standardized testing as the new way in education; the other views the FCAT's snowballing importance as a scourge that impedes genuine learning.
Some teachers say the changes give them much-needed tools to help their students succeed on the test...Others, accustomed to more freedom in the classroom, object to the new program as hand-holding and an affront to their professionalism...
School Board members also are complaining, upset that administrators did not brief them on such a large initiative until they rolled it out across the district.
Parents, too, are only now being informed. But the district administrators say they had no choice - almost half of Pinellas students are not performing at grade level. (No, this is not how is has to be - the FCAT grade levels are not norm-referenced, nor set at the average or the median, so there's no reason for half the population to always be reading below grade level. In this case, standards are set such that every kid, theoretically, could perform at grade level or above. There are norm-referenced portions to the FCAT, but the Sunshine State Standards are benchmarks.)
The district's mantra when presenting the new program has been: "If not this, then what? If not now, then when? If not you, then who? Our students can't wait"...
The initiative focuses on reading and math from kindergarten through high school. It has three components, though two of them have yet to be implemented in middle and high schools. The program to narrow instruction to FCAT knowledge is known as "essential learnings," which is in place across all grades.
To describe it, district officials use the example of a third-grade class learning word endings. Because the FCAT is likely to deal only with the endings "s" and "es," teachers should make sure enough students master them before plowing ahead to nontested endings such as "ing."
In math, the same class needs to master 45-, 90- and 180-degree angles before studying other angles that likely won't be on the test.
And so the passionate statements flow back and forth; one side insists that this is "teaching to the test" and the other side says that this is the best way to get all Florida's students up-to-speed on the standards.
In New Mexico, a school principal is fretting about the fact that his students didn't make the AYP (adequate yearly progress) targets this year:
Gonzales Elementary Principal Michael Lee knows exactly why his school didn't make "adequate yearly progress" this year under the federal No Child Left Behind Act: The fourth-graders didn't learn how to use rulers and measure quantities.
It is particularly painful to him, because his daughter was among the fourth-graders who took the test used to rate the school, and he thinks they're an exceptionally bright bunch of kids.
Dude. Fourth-grade is not too early to learn to use a ruler. It pains me to see these educators who insist that their students are smart - but just haven't gotten around to learning some basic facts and skills. If they really are that smart, heck, they should be converting inches to centimeters by now.
Still, Lee knows what his school has to do: Make sure this year's fourth-graders learn how to measure. Gonzales began a Cooking with Kids program this year, so the teaspoons and measuring cups they'll use in that program should help.
Well, that's something, but surely they could learn about measuring devices in math class as well, right? The article in general is good, but there are a few testing criticisms thrown in that, well, don't add up:
Sewing also sees problems with regard to students whose first language isn't English. Under the law, students have to start taking the standardized test in English three years after they enter the United States. Sewing said research shows language development takes five to seven years. "It's frustrating for these kids," she said. "It makes them feel like a failure."
"I would ask the adults out there, 'If you moved to China and lived there for three years, would it be fair to measure your education and your skills in Chinese?' " Sewing added.
If the entire time that I was in China, I was enrolled in a program that, for eight hours a day, was supposed to immerse me in Chinese and teach me the Chinese language, then yes, it would be fair. I might need a different standard than a native speaker - I can see the argument for that - but it's not unfair to test me to see if I'm where I should be after three years.
Gonzales Elementary's Lee said he'd rather "be sucker-punched" than repeat the experience of hearing his school had failed to meet AYP. "What we have to do at Gonzales is make darn sure we're teaching the standards. And because this test is based on standards, that should be easy."
"The pressure is going to build. And if principals and teachers are feeling pressure, then you can be sure that kids are feeling pressure. I have to wonder if this is the kind of world we want to bring them up in?"
If I had a fourth-grade daughter, and the choice was rearing her in (a) a stress-free environment, or (b) an environment with some stress in which she learned how to use a ruler, I know which one I'd pick.
Meanwhile, over in Boston, the educators insist that the "failing" labels are wrong:
...For the first time, a district could land on the federal watch list if just a single category of students fell below federal standards. Educators attributed the ballooning list to that new provision, which isolated the performance of groups including Hispanics, blacks, special education students, and low-income students...
Daniel Mayer, a school administrator in Maynard, where special education and low-income students fell short of federal standards, criticized the watch list as a punitive scare tactic.
"To me, it's sort of like the terrorist alerts that the federal government puts out and says, 'Everybody watch out, there's terrorists out there," he said. "No Child Left Behind is trying to motivate people from fear rather than well-thought-out initiatives."
So is it better for schools not to know if one group is doing more poorly than another? Is it better for parents not to know this? Mayer is apparently of the belief that any shortcoming within a school shouldn't be made public. But others don't agree:
Tom Loveless, a senior fellow at the Brookings Institution in Washington who has studied how states comply with the federal law, said bringing shortcomings to light is likely to spur progress.
"The real question is, is it better to know or not to know" how groups of students are doing, he said. "I think there is a growing public awareness it's better to know."
There's much test-score fretting in the North Star state:
Minnesota students are traditionally among the nation's top performers on key standardized tests. Unfortunately, the statewide averages mask an embarrassing reality. Students of color consistently score far below their white classmates.
This disparity in academic performance between groups of students is known as the achievement gap. It's a national problem. But Minnesota's gap is particularly wide.
A recent report from the Education Trust, Inc., highlighted the issue. Minnesota eighth graders ranked first in the nation in math on the 2003 National Assessment for Educational Progress. The average score among the state's white students (291) topped the list. The average score for African American students in Minnesota (251) ranked 22nd among the 50 states. Only Wisconsin had a wider gap between white and black scores.
I can't find the report on the Ed Trust site, although there's a lot of other good information on there. I particularly liked this article entitled "Good Teaching Matters," although that's another "duh" statement as far as I'm concerned.
But I digress:
The low test scores are a point of frustration to some; a source of anger for others. The Rev. Randolph Staten of the Minnesota Coalition of Black Churches says state officials have failed to adequately address the educational disparities.
"We wonder why it is with so many of our children being destroyed we have not declared an emergency in the state of Minnesota," Staten said.
Achievement gaps are often attributed to income level and home environment. Low-income families often have few educational resources at home. Recent immigrants don't always have the English language skills needed to keep pace in school. Some experts also point to low classroom expectations, peer pressure and teacher quality as key factors.
Nice to see that the tests aren't vilified here. And few reporters will touch upon the hot button of peer pressure and testing, even though at least one study suggests that peer pressure is more highly related to test score performance than is family income. It's more PC to blame the tests than to blame the negative peer pressure and low expectations that abound in poor schools.
Anyway, I tried to find out more about what's being discussed, and done, in Minnesota. (Note to self: Avoid future Google searches using "Minnesota score gap" as keywords, since this produces an avalanche of Packers articles.)
I found some 2003 NAEP data which suggests that the gap between fourth-grade boys and girls is increasing in reading; on the other hand, the black-white gap decreased slightly in fourth-grade math. Eighth-grade gaps between black and white students did not appear significantly changed from the previous year - which is good, because they're wide in both math and reading. If anyone knows of other articles that examine the Minnesota gap, let me know.
There's a new "new SAT" article making the rounds. Let's examine it, shall we?
The SAT is undergoing significant changes in 2005, including the elimination of those dreaded analogies...
Hey, I liked those!
...and the addition of a Writing section that includes an equally dreaded 25-minute essay...
Students who plan on attending college should NOT be afraid of having to write a short essay in 25 minutes. This is hardly setting the standard too high.
The changes are:
_The Verbal section will be renamed Critical Reading. Analogies will be eliminated. Short reading passages will be added.
_Quantitative comparisons will be eliminated from the Math section. Questions based on Algebra II skills will be added.
_A Writing section will be added, with questions on error identification, sentence improvement and paragraph improvement, plus a 25-minute essay. The writing test replaces the SAT II writing test previously taken by students applying to selective schools.
The revisions to the exam, the first in 10 years, make the test "better reflect what students are actually doing in classrooms," says Kristin Carnahan, associate director of public affairs for the College Board, the organization that designs and administers both the SAT and PSAT.
Though that might seem an obvious idea, it was not always the stated goal of the SAT. In the 1950s, 1960s and 1970s, the College Board widely touted the SAT as a measure of students' innate ability, and sections such as antonyms (eliminated in 1994) and those tricky analogies - brain teasers that were not directly related to schoolwork - were prized for that very reason.
But times changed. Students started studying lists of difficult words, and companies like the Princeton Review, which launched in 1981, began offering SAT-prep classes, all of which put the concept of the SAT as a pure measure of intellectual ability in question.
At the same time, some observers began saying cultural bias in questions' wording hurt the scores of minorities.
Not a bad timeline. Actually, the article in general is very good, and even-handed. The only "critics say" line is above, where it's actually appropriate. The reporter spends a lot of time on the topic that is most nerve-wracking - "that essay":
As for that essay - it will be read by two graders in a process that has been followed for years by the College Board in grading its SAT II writing test. Furthermore, the essay counts for only one-third of the Writing grade.
College Board representatives say the company conducted trials of the new test at 650 schools and found that a score of 600 on the old verbal test was equivalent to a score of 600 on the new critical reading test. Likewise, the scores from the old math test translated to equivalent scores on the new math test.
Of course, the Writing section is new and does change the balance of the test. Each student will now receive two language-related scores, which could concern some students who are significantly stronger in math than in language skills, such as students whose native language is not English.
Brian O'Reilly, executive director of SAT Information and Services, says, however, their research shows the addition of the writing test will be a boon for most English-as-a-second-language students.
"ESL students do not do as well on a writing test as non-ESL students, but with a writing test, that disadvantage is considerably less than with a reading test," O'Reilly explained.
And there's this tidbit, Lauren Schneider: "To some extent, it should help one group that right now scores lower than another group, that is women vs. men," O'Reilly said. "Women tend to do better on a writing test than men."
The bottom line on the new test, though, is that the vast majority of students will get a score that is comparable to what they would have received on the old test.
So students should relax.
Jason Karlawish, an assistant professor of medicine at the University of Pennsylvania, wonders if a standardized test would be useful in judging who is competent to vote in elections. Needless to say, this has some folks in a tizzy:
When should people with Alzheimer's or other cognitive impairments lose the right to vote? A new report suggests it's when they can't pass a standardized competency test.
A panel of doctors and attorneys, which floated the proposal this month, cautions that mental illness itself isn't good enough of a reason to deny access to the voting booth. But the caveat hasn't quieted critics who say a test spells trouble.
"Their proposed solution is misguided and would result in disaster," said Jennifer Mathis, senior staff attorney with the Bazelon Center for Mental Health Law in Washington D.C. "It essentially invites a new generation of Jim Crow practice."
The recommendations appear in a commentary in the current issue of the Journal of the American Medical Association. Jason Karlawish, an assistant professor of medicine at the University of Pennsylvania, said the panel decided to tackle the issue of voting rights for the mentally ill -- especially those suffering from senility -- after reading online posts from caregivers of Alzheimer's patients. The caregivers were discussing the 2000 presidential election and the voting habits -- if any -- of the people they cared for.
"Reading those postings got us interested in wanting to look at the ethical, legal and social issues of voting by those with cognitive impairment," Karlawish said.
On the one hand, it appears that some caregivers illegally cast the votes of people with cognitive difficulties, he said. (Assisting people with voting is legal; actually voting for them is not).
At the same time, many state laws disenfranchise people if they're under guardianship, or considered "insane."
In the report, the panel supported the use of a standard set down by a Maine court, which threw out a law that banned voting by mentally ill people under guardianship. The standard "is objective and it gets to the heart of the matter," Karlawish said. "Do you understand what is voting, do you understand the nature and effect of voting, and can you make a choice?"
A test to answer the questions would make sense in situations when someone is being put under guardianship, Karlawish said. Tests could also be appropriate in places such as nursing homes, he added.
Dr. Karlawish is active in research in geriatrics, aging, bioethics and Alzheimer's, so I'm betting his heart is in the right place. He - rightfully, I think - sees testing as a way to avoid wholesale bans, such as the one in Maine which grouped all "mentally ill" voters into one category (one wonders how they defined that group). I wasn't able to find the test in question here, although on other websites related to his research, I found information on other cognitive exams that he's recommended. He's a pro-psychometrics guy, from everything I can tell.
And what's the opposition's argument? The one lawyer quoted above resorts to hyperbole instead of reason - but then presents some sounder ideas:
Mathis, the legal advocate for the mentally ill, doesn't like the idea of a test, especially if it's administered by nursing home staff members. (The report suggests that election officials could take on the role.) "This type of test is extremely subjective, and it's fraught with the potential for abuse," she said.
A better approach, she said, would be to bring up the issue of voting competence during hearings to determine whether someone should be placed guardianship because of mental illness.
Agreed. No one other than disinterested and well-trained parties, such as psychometricians and clinical psychologists, should administer and score the exam. And certainly hearings would be a suitable place to discuss voting competence - although there's no reason that objective measures couldn't be introduced, and weighed, at such proceedings.
In the Lone Star State, parents are getting their hands dirty by taking on TAKS tutoring duty:
Learning the skills to master the Texas Assessment of Knowledge and Skills test does not have to begin and end with the classroom. Parents can play a major role in how successful their children are with the test by practicing at home, and the Ector County Independent School District Family Education Center is trying to help.
A mother of three, Myna Houghton, took advantage of one opportunity at Zavala Elementary’s Simply Science event. There the Family Education Center showed parents how they can create mini-science lessons at home.
“We came because of the science experiments. My son wanted to see more of what it was about,” Houghton said. “But I’m sure I’ll get a couple hints on how to do science at home.” In one experiment, Houghton and other parents who attended the science workshop learned that a drop of dishwashing soap could send a paper boat across a pan of water. The point was to show how the soap disrupted the water molecules and caused the boat to move.
Barbara Villaloboz, ECISD parent involvement specialist, said the education center wants parents to encourage their children to ask questions and to reason...Villaloboz said parents can help their children to do well on the test by teaching them how to read labels on products and asking questions about the weather and grass.
A useful message to send to parents. However, others are ready to give up, claiming the test is too difficult - at least when used as an exit exam:
A revised scoring plan designed to boost passing rates on the new high school graduation test was pitched Thursday by State Board of Education members worried about the possibility of massive failures on the exam.
The proposal, outlined to board members Thursday, would allow students to pass the exit-level Texas Assessment of Knowledge and Skills by averaging their scores from the four areas of the test – English, math, science and social studies. It has not been scheduled for formal action.
That would allow students to offset a failing score in one area with high scores in one or more of the other subjects so they could pass the exam.
Not all the board members liked this proposal, though. The Class of 2005 will be the first to be required to pass the new exit exam for graduation. They have five chances to pass, and if this composite plan is approved, Students take the test in spring of their junior year and have five chances to pass before the end of their senior year. Yet some are concerned about passing rates - and increased standards:
Results published this week by the Texas Education Agency indicate that 78 percent of high school seniors have passed after two rounds of the exam. That leaves almost 50,000 students who haven't passed. They have three more opportunities in their senior year. Students have done best on the social studies section, with 98 percent passing.
By ethnic group, the overall passing rates are 87 percent for white students, 68 percent for Hispanics and 66 percent for black students.
Some people are concerned that failure rates – particularly among minority and low-income students – may jump as the exam's passing standard is increased. This year's seniors had to correctly answer fewer than half the questions to pass the test.
Juniors this school year will have to get more answers correct to pass, and the cutoff score will increase again for juniors in the 2005-06 school year. The test replaced by the TAKS was much easier, measuring only eighth-grade skills.
So people are concerned that 12th-graders now need to display above-8th-grade skills. Hmm. Take a look at the new exam for yourself - here are the exit exams for the English, math, science, and social studies.
The 98% passing rate on social studies intrigues me. The Spring 2004 results are decent, with 85-87% passing overall on the other three components. and the Social Studies rate is high acros the board. Is the passing standard set lower for the social studies segment? Is the passing rate on English lower than social studies because the English portion has a writing exam? Interesting.
BTW, kudos to Texas for the testing site, which is easy to navigate, easy to read, and stuffed with information about all of the state exams.
I can only hope that the principal of Monroe Area Comprehensive High School is misquoted in this article about using the SAT to assess educational progress in Georgia:
While Walton’s system-wide Scholastic Aptitude Test (SAT) scores showed a drop-off from the previous year in most instances, local officials are questioning the state’s use of the SAT as a barometer for education.
System-wide, the 327 juniors and seniors who took the 2004 test averaged a 931 total score, 56 points lower than the state average and 18 points lower than last year’s system results. On the verbal portion of the test, Walton students scored a 463, 31 points below the state average and 16 points lower than last year’s results. On the math portion, Walton students scored a 468, 25 points below the state average and two points below last year’s results.
Officials partially attribute the drop in scores to more students taking the test in 2004.
“Statistics dictate that the more folks that take the test, the more you’re closer you’re going to get your mean score,” said Dr. Jimmy Stokes, principal of Monroe Area Comprehensive High School.
Whaa? What? Even if you try to figure out where some words must have been accidentally inserted, that doesn't make sense. Yes, if more lower-ability students took the test, the mean might drop. If a larger number of students who might not otherwise be considering college took the test, that might do it. But it could happen that more students whose abilities range from high to low were added this year, and they aren't going to shift the mean that much. "Statistics dictate," my ass.
Stokes follows this mush-mouth comment with one that is crystal clear, and wrong.
The mean score for the SAT is 800, Stokes said.
The top possible score per section is 800. The national mean for 2004 on the two sections combined is 1049. Reporter Joe Dennis should learn to use Google.
And the Monroe Area Comprehensive High School should find a better spokesperson. Especially when the article immediately follows his claim with an example of a system which has an increase in SAT-takers and an increase in mean score:
Monroe Area saw a 19 percent jump in the number of students who took the SAT. Despite the jump, the school actually saw its math test scores improve, from 447 in 2003 to 452 in 2004. It’s verbal score dropped from 466 in 2003 to 445 in 2004 for a total score of 897.
That would be "its," not "it's." Grrr.
The article also includes this interesting, and unchallenged, statement:
While the state places a heavy emphasis on students taking the SAT for college admission, the Walton system is starting to promote the ACT to its juniors and seniors.
“The SAT is not for everyone,” said Dr. Karen Rutter, technology and career coordinator for Walton County Schools. “The ACT is geared more towards what you know, but unfortunately Georgia has pigeon-holed everyone into taking the SAT.”
And, the SAT is about....who you know? Yes, it's true that the ACT differs from the SAT, and the ACT is most definitely based on what kids know:
The ACT is curriculum-based. The ACT is not an aptitude or an IQ test. Instead, the questions on the ACT are directly related to what students have learned in high school courses in English, mathematics, and science. Because the ACT tests are based on what is taught in the high school curriculum, students are generally more comfortable with the ACT than they are with traditional aptitude tests or tests with narrower content.
But the correlation between ACT composite and combined SAT scores has been found to be a cool .92, which is very high for the social sciences. This suggests that (a) the SAT and ACT are tapping into the same skills, and (b) it is unlikely that there are vast numbers of kids who bomb the SAT but would do just fine on the ACT.
Luckily for this school district, someone with sense is employed there :
Superintendent Dr. Tim Lull said the way the educational system is set up in the state doesn’t adequately prepare students for the SAT. “If the push in Georgia is going to be the SAT, then the state has to modify its testing and curriculum,” Lull said.
Lull noted that the Georgia Department of Education uses state-produced tests throughout the year — such as the Criterion Referenced Competency Test and the Georgia High School Graduation Test — to track student progress, but then emphasizes the results of the SAT, a nationally standardized test. “You can’t hold kids to the state standard and then expect them to hold to a national standard,” he said.
He has a point. If the state cares about SAT scores, then it should focus on those skills. It's not a given that the CRCT and GHSGT measure the same skills as the SAT/ACT. It would certainly provide some criterion validity evidence for those tests if they did, though; if in fact large numbers of students are passing the GHSGT but bombing the SAT, that's something for the admins to think about.
There's nothing like a group of students doing well to change testing criticism to testing praise, as evidenced by this article that's just chock-full of educators singing the praises of the SAT:
Ohio's class of 2004 achieved scores 30 and 24 points above the national average on the verbal and math Scholastic Aptitude Tests (SATs), according to results released today by the College Board's Advanced Placement Program. Ohio students had an average score of 538 on the verbal test, compared to the national average of 508. They also averaged 542 on the math test, compared to 518 nationally...
The SAT results demonstrate that Ohio students continue to outperform the national average on national tests, including the American College Testing entrance and placement exam and the National Assessment of Educational Progress...
"It's encouraging that Ohio students are continuing to increase their state test scores and stay above average on national tests," said Susan Tave Zelman, superintendent of public instruction. "Ohio's educational system is working. These improvements are a trend we expect to see as new assessments are aligned to the state's academic content standards"...
"Today's students need higher order thinking skills to compete in a 21st century global economy," Zelman said. "Challenging academic standards and coursework will prepare all of our students for success in college and careers."
Why, there's not a "Critics say" line in sight. Imagine that. From now on, if I see anyone bashing the SAT as measuring nothing but "test-taking skills," I'll just send them along to Zelman, and she'll take care of 'em.
And do a Google search for "national SAT scores" to read all the latest articles about schools that are either celebrating or fretting over their comparisons to the SAT and ACT averages. However, none of the articles link to the College Board site that has a plethora of SAT information - for shame, mainstream reporters.
From Pennsylvania comes this tale of students who have learned facts but don't really understand the material - or so the teachers claim:
Wolfe, an eighth grade reading teacher at Big Spring Middle School, helped design the Reading Increases Students' Excellence (RISE) class to prepare students for the ninth-grade standardized writing test...
When she started incorporating the relay race into the class, students who thought they knew the words but didn't know them well enough to write their own examples were "devastated," Wolfe says.
They had memorized the definitions for a test in other classes, but "they didn't actually know that they had to know it for knowledge, for life."
This, of course, is the lead-in for a crop of test criticism. But I see this as a criticism of their former teachers. What did they do in their language arts classes? Say to the kids, 'Okay, just memorize this list, but you'll never use these words again - they aren't important"? How bad do teachers have to be for kids to get the idea that the English language is not knowledge they'll need for their lives? Don't go blaming tests for this, nuh-uh.
Getting the right answers on a state standardized test is a "game," says Donna Benson. It's a game many of her students refuse to play.
Benson teaches gifted students at Cumberland Valley High School, and says highly intelligent students tend to ignore test-taking conventions, especially when writing essays. Instead, many write creatively, and as a result score low on the Pennsylvania System of School Assessment (PSSA).
I hear this a lot, but I've never seen any evidence to back it up. Anecdotal evidence, sure. But there isn't a scoring rubric out there that doesn't give examinees a boost for using correct spelling, grammar, and vocabulary. Is it really true that there are vast numbers of very smart kids whose writing is so creative that, despite proper vocabulary and spelling, their essays recieve very low, or flunking, grades?
"It's done kind of as formula writing," Benson says of how the writing portion of the PSSA is scored. "I want the kids to know what the formula is, but ... I want the kids to go beyond that," she says.
And they can. But they should understand why the formula is there, and why it would be silly to protest the formula on the grounds that it's too "dumb" for them. For a lot of kids, it would be a big step forward just to be able to write well, period, even if the writing came out formulaic.
Highly intelligent students often have trouble with multiple choice and true-or-false questions because they "over-analyze" the question, Benson says. She worries gifted and bright children get left behind when schools emphasize remediation...
When schools make proficiency their goal, they miss chances to enrich bright students further, she says. "Proficient isn't good enough for the progress we're facing in the future."
"I'm glad my children aren't in school anymore," says Candy Shively, who works for Cumberland Valley School District and used to teach special interest classes. "I think we're really skimping on the enrichment things and the higher level of thinking."
And I can understand her way of thinking. Unfortunately for the bright kids, the act is called Leave No Child Behind, not Push The Smart Ones Ahead. Proficiency isn't enough for a subset of kids, true. But when so many students are failing miserably, schools are often forced to focus on them.
From Carnegie Mellon comes this advancement in education:
Carnegie Mellon has developed a Web-based computer tutoring system to help middle-school students prepare for standardized mathematics tests like those required under the federal No Child Left Behind Act. The "Assistment" system aims to solve a dilemma for teachers: how to prepare students for tests without sacrificing quality instruction time. The system is designed to quickly predict a student's score on a standardized test, provide feedback to teachers about how they can adapt their lessons to address students' problems and provide individualized tutoring to suit each student's needs. The system is being tested in Massachusetts with a grant from the U.S. Department of Education, but it can be easily adapted for use in other states. In developing Assistment, researchers have drawn upon the proven success of Carnegie Mellon's popular Cognitive Tutor®, a comprehensive secondary mathematics curricula and computer-based tutoring program that is in use in 1,500 schools nationwide. Contact: Jonathan Potts at 412-268-6094 or Anne Watzman at 412-268-3830.
I found more info at the Pittsburgh Advanced Cognitive Tutor site. Sounds interesting.
The Contra Costa Times has an amusing round-up of info for parents whose eyes glaze over when confronted with stacks of percentile-ranking charts:
There's no nip in the air and the leaves are a steadfast green, but the shift of seasons is palpable.
Freshly scrubbed minivan windows no longer scream, "Swim fast, Sharks!" Dime store lines are filled with parents clutching three-ring binders and graph paper. The first day of school is coming. For some kids, it's here already, along with new teachers, new friends and fresh starts.
Teachers, on the other hand, are still smarting over last week's release of STAR test scores. "English language learners didn't do well on an English language test? Could it be because they don't speak English yet?" they holler down the phone lines.
August is not just back-to-school, it's STAR, CAHSEE, API and AYP month, too. Last week brought scores from the Standardized Testing and Reporting program and the high school exit exam. Aug. 30 brings new Academic Performance Index rankings and "adequate yearly progress" data.
Frankly, there's enough school assessment data flowing from Sacramento this month to make any parent's eyes glaze over. Cucamonga kids performed how well? Inyo County fifth-graders posted what percentages? School assessment data has been available online for some time, but in a format only an education policy wonk or closet statistician could love.
I never knew there were statisticians in the closet. Maybe that's because I'm surrounded by people like myself who are the statistics-geek equivalents of the drag queens parading down 5th Avenue during the Gay Pride parades. We love numbers, and aren't ashamed of it.
Miriam K. Freedman, whose work I've covered in N2P before, has a great website and a new article at the Hoover Digest about the fight for high standards. It's phenomenal, and you should all read the whole thing, but if you don't have time, here are a few of my favorite spots:
High-stakes tests, which affect a student’s ability to earn a high school diploma, are now in place or on the drawing boards in about half the states. Often controversial, they have spawned “test boycotts” and lawsuits...But a strange thing is happening: As we get closer to having the graduation tests “count,” many leaders have blinked, with the result that standards are compromised and test results invalidated...
Where is the outrage over the need for valid tests? Inconsistency in test administration has real consequences. As is becoming increasingly obvious, confusion and inconsistency are leading to a loss of credibility in the standards movement. But why is it happening—why are some blinking?
Word choice is telling. It used to be that a student “earned” a diploma. Now many speak of a student being “denied” a diploma. The first is about standards; the second, about rights and lawsuits. Our evolving language—unfolding daily in the press—tells the tale.
...consider the 2001 settlement of a lawsuit against the state of Oregon by Advocates for Special Kids, represented by Disability Rights Advocates (DRA). The suit alleged that Oregon’s test for the Certificate of Initial Mastery (CIM) discriminated against students with learning disabilities when it tested them on basic skills such as reading, writing, and math. Note that the test was not a diploma test. When I first heard of this allegation several years ago, I dismissed it. The basics are discriminatory? They can’t be serious! Common sense and the law say otherwise. But life is full of surprises: Oregon settled the case...
Recently, I asked a friend whose learning-disabled child had a very hard time learning to read, “What do you think of state policies that allow a child to take a reading test by having the test read to him?” Her quick response was, “A cop-out. My son would never have learned to read if that was the law then.” Indeed. Yet some states allow students to use any accommodation on the state test that they use in classes. Thus, a reader (someone who reads material to a student) can be provided on the reading test, a calculator for the math calculation test, and so on...
Yogi Berra is reputed to have said, “If you come to a fork in the road, take it.” We’re at that fork in education reform. Many are going in the front door, loudly supporting high standards—until someone threatens to sue or someone might actually be denied a diploma. At that moment, the back door of retreat and erosion flings open and we quietly sidle out. Sometimes, we even call something “reading” when it’s really “listening,” and we report scores with questionable meaning without so indicating. We’re on a very slippery slope. Although intentions may be good, the response is misguided...
Blinking at standards fails the public when, as the going gets rough, we quietly alter tests and standards. Such compromises have a crippling ripple effect on education reform, leading to cynicism and loss of faith in the entire venture. Such compromises are not legally warranted. For the sake of our children and for America, we must not blink.
For those who may have been hoping that laptops were magic wands when it came to test scores, these results are surely a disappointment:
Middle school students who used laptop computers for two years performed about the same on a standardized test as students in the past who did not have access to computers. Critics of laptops say the scores are the first real evidence the program, which has cost the state more than $15 million, is an expensive fad. Proponents say it is too early to expect dramatic changes in test scores...
Scores for reading, writing, math and science in the Maine Educational Assessment were essentially unchanged in the past school year among eighth grade students compared with the previous two years. But there was a measurable improvement in writing scores among students who took the online version of the test at 60 schools...
Florida's students are improving in history, as measured by standardized test scores:
About 86 percent of students taking U.S. history passed a new districtwide standardized test this spring. About 74 percent of world history students passed a similar exam. Two years ago, when the 100-question tests were first implemented, so many students were in danger of failing that passing grades were curved to the extreme — a student needed to answer just 23 percent of the questions correctly.
Administrators and teachers acknowledge that the high passing rates from the most recent exams were because they are getting better at "testing what they teach" — or teaching to the test. Detailed class guides show teachers exactly what points they must hit, and when.
"It's been met with some resistance," Palm Beach Central High School teacher Don Meyers said. "But you have to adjust. That's the reality of education in the age of accountability."
Yes. Yes, it is. It's a reality in a day where people are starting to acknowledge that it's a travesty when students who have taken US History can't answer 23 questions out of 100 correctly. And the details provided in the rest of the article provides clues as to why things might have gotten so bad.
In American history, teachers have one year to cover from 12,000 years ago, or prehistory, to the current Iraq war. It was easy for teachers to focus on one era they were particularly interested in and gloss over others.
12,000 years? I know they had to cover the Native Americans, but surely no one thought that any substantial percentage (more than 10% or so) of American History class time should be spent on the many, many years before America was America.
School board member Debra Robinson provided the impetus for the standardized history test because of her concerns that African and black history were not being taught. The test allows administrators to see whether teachers are touching on key areas.
For example, in American history, students did the worst on questions on the effect of immigrant groups and questions on political events in contemporary America. They did the best on multicultural questions and black history questions, getting an average of 77 percent and 79 percent, respectively, of those correct.
Well, then now we know what students need to focus on - immigrant groups and contemporary politics.
In world history, students struggled with the Age of Reason and the Industrial Revolution. They did the best on the Age of Discovery and, again, on African and black studies.
I'd say the Age of Reason and the Industrial Revolution are fairly important topics.
Still, Robinson said she continues to hear anecdotally that the black curriculum is not being taught in the detail it deserves.
Yes, but the data suggest that's not true. Wasn't the point of this test to see where students need extra help?
She's also concerned about the average scores on the tests. Although 86 percent of students passed the American history exam, the average score was 75, a solid C. In world history, the average score was 67, a D-plus...
"It's sad when you say 70 is good," Robinson said. "I think that shows what a pitiful state of affairs we're dealing with in general. The definition of pass in my house is a B. I won't kill you if you bring home a C, but I'm not happy either."
I agree. They may be passing now, but just barely.
Things aren't looking great in New Mexico:
Barely half of New Mexico’s high school juniors were proficient in reading and less than half were proficient in math, according to results of a standardized test released Friday by the state Public Education Department. The first-time assessment test, taken by more than 19,000 students in November, will be used to rate public high schools. Results show 56 percent of students are proficient in reading and 47 percent are proficient in math...
In Clovis, 51 percent of the 512 students tested were proficient in math while 52 percent of 513 students tested were proficient in reading.
Clovis schools Superintendent Neil Nuttall praised the test as a fair assessment.
“We feel OK. I don’t think we can say we feel good, but what makes us feel more encouraged more than anything is what we have put into place,” Nuttall said. “We have beefed up our curriculum. We have beefed up courses. We have increased the number of requirements for graduation.”
Is there some course that superintendents take in college that teaches them how to put a positive spin on anything, including the news that half their students are not proficient in reading and math? If so, it looks like a few administrators were absent the day that spin was taught in class:
Portales schools Superintendent Jim Holloway did not return phone calls placed to his home and office. Portales High School Principal Melvin Nusser did not return calls placed to his home Saturday.
But one educator is a master of the "spinning by redefinition" technique:
Fort Sumner High School ranked above state average in both subjects. Last month, Superintendent Lecil Richards apologized to state officials for failing to administer a standardized test to fourth and eighth graders in March. He said he will leave it up to the state to decide if Friday’s results vindicate him and prove his schools are excelling.
“Test scores are just one indication and not the only indication of how our children are doing,” he said. “I really look at how they do in life, how they handle themselves and how they do after high school.”
Because, as we all know, students who can't read and don't have mathematical skills are likely to succeed after high school, thanks to how well they "handle themselves."
An article about political response to Illinois' recent cutback of standardized tests has some telling comments. Some legislators are horrified about the cuts, and it's worth noting why they fear this will have a negative impact on education:
A decision by state legislators to scrap social science and writing exams in Illinois grammar schools is drawing mixed reactions from local educators.
Norridge District 80 Superintendent Sue Knight is dismayed. "I was absolutely speechless," she said. "I couldn't believe this was happening. It's a blow to all of us who are dedicated to doing the very best for our children."
In the late 1970s, Knight was part of a group of teachers who developed the standards upon which the state's writing program are based. She brought those standards back to the school where she was teaching English. Teachers were aghast, she remembers.
"I remember one teacher asking me, 'How can they write paragraphs in third grade? We're just teaching them to write complete sentences,'" Knight recalled. "I told her we'd just have to shift our priorities, and we did.
"Now, our kindergarten students know what a complete sentence is, and our students know how good writing is done."
The teachers were aghast that third-graders would have learn to write paragraphs? How did things get that bad in Illinois? If the teachers weren't expecting third-graders to write in complete paragraphs, I bet you dollars to doughnuts that the teachers didn't expect these kids to read complete paragraphs, either. How on earth did they manage to take three full years of school just to get a kid up to the point of being able to write a complete sentence? No wonder Ms. Knight is afraid that removing the test will have a negative impact.
The fear of standardized tests is apparently so strong that references to testing bugaboos have crept out of news reports - and into the movies:
When Winter Garden novelist Edward Bloor set out to write a chiller for young adults, he decided to include a murderous poltergeist. Then he added a really scary touch. The poltergeist lives in a magnet school where the only thing that the students do, day after day, class after class, is take standardized tests.
It's the spookiest fictional school setting since Carrie made a mess out of the prom.
Hee hee hee. I think it makes for a nice surreal (and frightening) touch.
"I just thought it would be humorous to take things to the nth degree -- but then, I have an extreme sense of humor," [Bloor] says. Perhaps. But he's not alone. The hopes and fears of many students, from grade-schoolers on up, are so intertwined with standardized testing that references to it have crept out of headlines and PTA meetings and into the province of popular culture. Novels, movies and television shows are full of young characters grappling with issues related to test scores...
Not even the reigning prince of young-adult literature is exempt from testing-related tremors: The title character of Harry Potter and the Order of the Phoenix gets the shivers over his upcoming exam, the O.W.L. -- Ordinary Wizarding Level.
Oh, rock on. That's too awesome. Not even the soon-to-be most powerful wizard in the world will be able to escape the horror of --- STANDARDIZED TESTS. Tee hee hee.
I have no problems with testing critics who have a sense of humor, and are willing to show us how absurd testing (like anything) can be when taken too far. Kudos to Bloor. But I bet you some humorless educrat will watch this movie and try to convince the public that some schools, and tests, really ARE this bad.
Illinois' new budget cuts mean the state standardized exams in writing and social studies are history. Some educators have no problem with this, claiming the tests were "inessential":
"Standardized tests are important, but they're nonessential in 85 percent of the school district. In the state of Illinois we are in deficit spending right now. We're in a nationwide crisis and non-essential programs are going to have to go," says Guilford High School history teacher Walter Hoshaw...
Hoshaw says results of tests don't always reflect how the student is performing on a daily basis. "A standardized test is just a snapshot of how that kid is doing that day," says Hoshaw.
Hoshaw says there are plenty of other ways to test a student's ability. "We have pretty good measures through the SAT and ACT tests that we do (with) our high school kids. That gives us a much better measure of how our kids are doing," says Hoshaw.
Others, though, aren't so sure:
The decision was as much about the state's testing philosophy as saving money. It is not expected to change even if the money were restored.
"I don't think anyone would argue that those tests don't have value ... but what it really boils down to is whether or not we can afford to expand an assessment system beyond what the federal government requires," said state Rep. Roger Eddy(R-Hutsonville). Eddy is a former social studies teacher who serves on the Education Appropriations Committee and also runs a rural school district Downstate.
"I have heard the philosophical argument that if we stop testing it, schools will stop teaching it. But I believe in professional educators more than that. We have to remember that, long before this standardized student movement, teachers were teaching their students how to write."
Yes, they were. But are they still?
The announcement already has educators rethinking their priorities for next year. Becky McCabe, a principal in Urbana, said she originally planned to make writing part of her school's improvement plan. But the testing change means her staff likely will refocus on improving reading skills.
Writing instruction will continue, she said, but probably will shift to the type of writing with which students and teachers are most comfortable--creative and narrative essays. The other kinds of writing now tested by the state--persuasive and expository essays--will likely lose favor, she said.
She also expects fewer districts will invest in training teachers on better writing now that they know their schools will no longer be judged on this subject.
"I hate to say this, but you treasure what you measure," McCabe said. "When it comes down to money, you're going to focus on things that are on the bubble. Writing is just not going to be the same, and that's a shame."
Have we really reached a point where teachers - teachers, for heaven's sakes - won't teach kids how to write in any non-tested fashion? Where do the teachers think kids will pick this up, if not in school? Comments like this make me wonder if Representative Eddy really understands the lack of focus on writing skills these days.
Update: Devoted Reader JW has this to add:
A liitle more background for your post about dropping the Social Studies test here. The most important thing to understand is that Illinois is undergoing a profound, severe, and probably long lasting budget crisis. There are a lot of reasons for this that don't matter much for this discussion. Every state program is getting chopped to some degree. The Social Studies test doesn't seem to measure very much, or to provide useful info for parents. Districts that do well on the reading and math tests tend to do well on the SS, so it gets the chop. Sure, there are a lot of teachers and administrators who aren't nuts about NCLB, and who don't like testing anyway; it's part of the territory. There are also a lot who just deal with it without being obsessed one way or another-like, I'm happy to say...
The dance between soon-to-be Democratic presidential candidate John Kerry and the educational unions ought to be fun to watch over the next 100 days...
...while education experts agree that Kerry has mapped out a more liberal education agenda, he has surprised some educators with more moderate proposals.
Kerry caused some consternation with the National Education Association in May when he proposed spending $30 billion over the next 10 years to hire 500,000 teachers, but to reward teachers with higher pay when their students' performance improved.
The NEA opposes the measure. It would use students' standardized test scores to determine whether teachers should receive bonuses, as opposed to traditional factors such as tenure.
"I believe we need to offer teachers more pay. More training, more career choices, and more options for education. And we must ask for more in return, that's the bargain," Kerry said.
Emphasis mine. Think this was because Kerry sensed the parental approval of get-tough acts like NCLB? Whatever the case, he now appears to be backpedaling. So he's for "pay for performance," until he's against it. And even though Kerry's not yet at the convention, the teachers are, in full force:
Milton Bond Jr., a high school teacher from Milwaukee, Wis., and a first-time delegate from that state, said he wanted to come to the Boston convention because of his concern about the direction the country is heading under President Bush...
Mr. Bond echoed many complaints that his union has leveled at the law, which holds schools accountable for showing yearly academic gains by their students.
“I feel like you’re punishing schools, and you’re punishing students. It’s an empty policy,” he said.
But we are punishing schools. That's the point. We're trying to punish the schools that are punishing students by depriving them of a quality education. It's pretty hard to hold schools accountable for anything if there's never any punitive action for the schools who fail their students. Perhaps Kerry, with his talent for "nuance," can think of some way to get accountability with only praise, never punishment.
And then there's this:
Convention delegates flipping through copies of the Boston Globe on Monday might have stumbled upon a provocative quarter-page advertisement with the headline: "No Child Left Behind?" The ad, signed by more than 100 classroom teachers, parents, noted education advocates, and others, suggests the federal law is part of a plan by President Bush "to privatize America's public schools," and that it threatens thousands of schools with closure. The law, the ad argues, encourages "lying about the facts" and "uses blacklists to banish professionals, institutions, methods, and books."
Addressing John Kerry, John Edwards and the Democratic Party, the ad declares, "Teachers need your support to save our schools from the punitive law misleadingly labeled No Child Left Behind ... " Sens. Kerry and Edwards, along with most other Democratic congressional delegates here, voted for the No Child Left Behind Act in 2001. As candidates for the White House, both have suggested the law needs some changes, but the ad calls for stronger medicine.
"Will the Democratic Party commit to getting rid of NCLB?" it asks.
The ad quickly drew fire.
"It's outrageous," said Andrew J. Rotherham, the director of education policy at the Progressive Policy Institute, a Washington-based think tank aligned with the centrist Democratic Leadership Council. "There are legitimate criticisms of No Child Left Behind, but that ad seems to go out of its way to avoid them."
Yes, it does, and in doing so overplays its hand. I can't improve upon Mr. Rotherham's summary of the effect of the ad:
...Mr. Rotherham suggests that the ad's rhetoric may well undermine its mission.
"Hysterical paranoia went out of style after the primaries, when John Kerry [prevailed]," Mr. Rotherham said.
"Ads like this hurt the cause of people seeking changes in No Child Left Behind, rather than help it," he added. "Your average person sees an ad like that and is going to smell weirdness, not reasoned debate."
Whee! Let's have fun! Let's get kids wondering whether the outfit they wear on the day of the SAT will effect their score! That's so much more fun than drilling them on vocabulary!
Test-prep company Peterson's says it plans go beyond drilling students in the three Rs, starting what it's calling a testing laboratory to see whether students gain any edge on the SAT from the little things - the choice of pre-exam meal, the hue of their clothes, the music they hear on the drive to the test...
In the end, Peterson's says, it's mainly aiming to inject some fun into the stressful standardized test process - and if scores creep up a few points, so much the better.
"We don't want people to think they really will raise their scores 100 points if they wear the right color," said Jessica Rohm, vice president of communications for Thomson Learning, Peterson's parent company. But "just taking the edge off by bringing in some fun things associated with testing I think will raise their scores a little."
"Fun things associated with testing"? C'mon. I'm in testing, and "fun" is not one of the first five words I'd associate with the field, or with the image the public has of it. I'm all for taking the edge off for examinees - test anxiety is through the roof these days, in no small part because of inaccurate press which claims that tests are biased, unfair, useless, or all three. But I don't think everything should have to be "fun" to get students involved, and that includes a demanding admissions tests.
I'll eagerly await the outcome of Peterson's experiments, but I'm betting the top scores will continue to be delivered by students who see the SAT as a challenge for which one has to methodically and intensely prepare, and are able to motivate themselves for it without "fun" - or wardrobe considerations (how classist! What about those kids who don't have that many clothes to wear? And doesn't that just add more stress for obsessive clotheshorses?) - being in the picture.
Well, well. There appears to be a clash between teachers and SAU administrators in New Hampshire's North Hampton school district:
The School Board voted not to test North Hampton School students next year using a standardized test recommended by SAU 21 administrators. The NorthWest Educational Association tests were used this year, but teachers at NHS unanimously said they believe the tests are not useful and should not be continued. School Administrative Unit administrators suggested that teachers at NHS were not trained to properly use the tests and so cannot judge their effectiveness.
Ouch! But it's worth asking - who was in charge of making sure the teachers were properly trained? The NWEA is a computer-adaptive exam, so the students would have needed a bit of training as well.
NHS Principal Peter Sweet said teachers "tried to make (the test) meaningful," adding, "They used and shared the data, but they don’t want to do it again." Sweet said he would prefer that the school focus on grade-level assessments developed by the teachers to monitor students’ learning.
Teachers have complained that because of the structure of the test, students might end up being asked questions usually meant for much-older kids, and this kind of data is not helpful for how they want to teach.
SAU administrators pointed out that other districts in which more teacher training took place seem happier with the test, and more able to use the scores.
So far, we're not seeing a lot of good results from Florida's summer camps intended to boost FCAT passing rates:
Low grades and failing FCAT scores sent thousands of students back to classrooms throughout Southwest Florida this summer. For third- and 10th-graders who had to pass the FCAT or the alternative SAT-9 to advance, few were successful.
District officials in all three counties said they have not compiled scores yet, but among a handful of schools contacted, all reported few passing students.
Among third-graders attending reading camp at Sallie Jones Elementary in Punta Gorda, only one of six passed the SAT-9. At Taylor Ranch Elementary in Venice, three of nine passed...
Teachers said the test scores don't reflect everything.
"There's not one person that we didn't see growth in their skills and that's what's important," said Michele Markstahler, lead teacher for the summer program at Sallie Jones.
In all the hubbub of commenting and contradictions on last week's post on the infamous geometry question posed to Governor Bush, I completely forgot to go find what Charles of Reform K-12 had to say on the topic. An unforgiveable mistake, on my part.
Charles is not only a superb geometry teacher, but is more willing to go out on a limb than I. While I tactfully said that I didn't think the item mentioned by Ms. Marques was on the FCAT, Charles said, "We have no doubt that there was some question involving either a 3-4-5 right triangle or a 30-60-90 right triangle, but not both. We will publicly eat a copy of the FCAT if anyone can prove us wrong."
Like me, he also found it appalling that it was "news" that the Governor got the item wrong, but not that the girl herself got it wrong:
No one expects grownups in non-scientific fields to remember high-school geometry, so the Governor's ignorance is understandable. But here is not only one student, but her friends as well, who are confoundingly ignorant on basic facts from 10th grade geometry, while being convinced they know their stuff.
In other FCAT-related news, I received an email from Devoted Reader, and concerned parent, V.G., who is an FCAT supporter but has some questions about a "research" FCAT remediation class. It seems that schools receive funding to place students in this remediation class and may be, um, overeager to place students (even those who are performing at grade level) in the class. Has anyone else had an experience like this?
California has decided upon a traffic-light theme to make test scores more interpretable to parents - probably not a bad choice in a state that's as car-crazy as California:
Parents of California's public school students can expect easier explanations this summer of how their children are performing on state standardized tests. Education officials have scrapped complicated old report cards that they said confused parents and replaced them with easier charts and color-coded guides.
In three weeks, parents of nearly 5 million students from grades two through 11 will receive the mailed Standardized Testing and Reporting (STAR) Student Report, showing how each performed on last spring's tests.
The new two-page report will use the three colors of traffic lights to show how students are faring - green for proficient, yellow for basic and red for performance below basic, state Schools Superintendent Jack O'Connell said.
Students will also gets lists of their strengths and areas that need more focus. For the first time, the reports also will include translation guides in Spanish, Chinese, Hmong, Korean, Tagalog and Vietnamese.
First it was a secret. Then it wasn't. Now it is, again. Georgia's DOE needs to make up their minds:
The state will delay going public with any more "cut" scores — the number of correct answers needed to pass a standardized test — until the Board of Education adopts a formal process for releasing the figures.
The state board wants to make sure the public understands what the cut scores mean, Chairwoman Wanda Barrs said Wednesday.
Last month, the state released cut scores for the Criterion-Referenced Competency Test following an open records request from The Atlanta Journal-Constitution and a unanimous board vote that the cut scores should be released. The newspaper also requested cut scores for two other standardized tests: End of Course Tests, taken by high school and some middle school students, and the Georgia High School Graduation Test.
Basically, they're not revealing any more cutscores until the public has the information they need to interpret those scores correctly (and not make disparaging comments about how low the cutscore is, as some other journalists - and I - have done). It's certainly not a bad idea to make sure a definition of cutscore, along with other test information, gets released when the cutscores do. So let's see what Georgia ultimately releases, if anything.
What's going on with test scores in Delaware? And where's Dave Huber on this?
Buoyed last month by an apparent bounce in the state's standardized test scores, elected officials rushed to put themselves in the way of reflected glory...
But what looked like rising test scores could be, in part, a fluke, a one-time bounce that next year could burst like a bubble and this year could be obscuring academic decline in some student groups. The inflation may be related to a state law that took effect two years ago to hold back students with low test scores, meaning this year's testing pool may have contained more higher-achieving students than in past years.
Nonetheless, both Minner and Education Secretary Valerie Woodruff said the improvement in test results is genuine.
The rest of the article, which is long but worth reading, goes on to explain that when the repeaters eventually reach the 10th-grade exam (next year), the scores will most likely drop down a bit. It's true that it's hard to disentangle the real progress from the examinee pool anomalies in Delaware's startling test score improvements (53 percent met the math standard, compared with last year's 45 percent; 71 percent met the reading standard, compared with 66 percent last year). The retention rate for eighth-graders in 2002 was 11.5 percent (as compared to 3.8 percent in 200), so that's a sizeable little chunk that didn't take the 10th-grade exams this year.
Of course, if retaining those kids did in fact help them to be better prepared for when they do take the test (results so far suggest otherwise), the test scores won't drop as much. So I think next year's scores are going to be very informative.
Me, I'm just savoring the opportunity to read a news article in which it is assumed that a state's standardized exam scores are accurate indicators of true student ability (we're seeing high scores because the examinees are smarter, for whatever reason). Funny, but don't we almost always read the opposite - tests are biased, educators shouldn't assume that student ability is being accurately portrayed - when low test scores are observed? Why isn't this reporter rushing to remind us that "critics say" such tests are biased? Most times, we see quote after quote in articles trying to convince us that the tests are meaningless and that students are really doing much better than scores would suggest; here, we're being cautioned to think that students are actually doing worse than scores would suggest.
This reminds me of a time-honored technique of testing critics, in which they damn all tests as biased and unfair - unless those tests support a political point they want to make. Witness the groups who claim SAT scores are biased, but then use SAT scores to make a point about a different test.
You know, Governor Bush should have expected something like this:
During a speech to high school students who mentor younger children in reading, a teenager asked the governor a basic geometry question taken from the Florida Comprehensive Assessment Test, which Bush has championed.
"Me and a couple of my friends ... we know that the FCAT is a very important part of schooling in Florida and we were wondering if you could answer one of the questions we remember from the FCAT?" said Luana Marques, 18, who just graduated from Freedom High School in Orange County and is heading to Flagler College in the fall.
The luncheon crowd at an Orlando hotel, gathered to honor 200 students who take part in the Teen Trendsetters Reading Mentor program, laughed and Marques posed the question: "What are the angles on a three-four-five-triangle?"
The governor gave a steely grin and then stalled a bit. "The angles would be ... If I was going to guess ... Three-four-five. Three-four-five. I don't know, 125, 90 and whatever remains on 180?"
Marques had an answer, although it wasn't the right one: "It's 30-60-90."
The correct answer was 90 degrees, 53.1 degrees and 36.9 degrees, said Michelle Taylor, a graduate student in mathematics at the University of Florida, when told about the governor's pop quiz...
"If the point is, I haven't been in school for the last 30 years, that's true. But if I'm going to be graduating from high school and I can't pass a 10th-grade aptitude test, then I'm fooling myself," Bush said. "The fact that a 51-year-old man can't answer a question, is really not relevant. You're still going to have to take the FCAT and you're still going to have to pass it in order to get a high school degree."
Marques thinks Governor Bush didn't have much of a sense of humor about the whole thing, and he should have, but to be fair, he could have pointed out the grammatical atrocity in her opening statement, "Me and a couple of my friends...were wondering..." And what if he had known her answer was incorrect and pointed that out to her as well? Why did she ask a question to which she mistakenly thought she knew the answer?
BestOftheWeb points out:
Now first of all, does anyone who isn't a graduate student in mathematics know the answer to this? We certainly didn't. Besides, geometry is the most useless branch of mathematics, at least in our experience. We occasionally make use of algebra, trigonometry and calculus, but we dropped our high school geometry class after a couple of months, and we've never missed it.
I certainly wouldn't argue that geometry was useless, but I wouldn't have known the answer to that item off the top of my head, either, and I aced AP Geometry in high school, and I'm younger than Governor Bush. While I may not remember the proofs, the mental discipline that came from doing them has helped me in many ways unrelated to math, and to suggest that students shouldn't learn that material just because Bush doesn't know the answer to one item off the top of his head is just plain silly.
Update: Given the discussion in my comments section, I think the theory floated by commenter Josh - that the young lady misremembered a 1-2-sqrt(3) item as a 3-4-5 item - is quite possible. According to the FCAT math item specifications, the only trigonometry tested on the 10th-grade FCAT is right-angle trigonometry (the famous SOH-CAH-TOA). It's hard to believe that the uses of inverse trig functions (necessary to solve a 3-4-5) were tested on this exam. Perhaps she misremembered a Pythagorean theorem item that used a 3-4-5 triangle.
Given that WFTV went to the trouble to contact a mathematics graduate student for the correct answer (and note that they didn't ask a Florida high school math teacher), why didn't they ask her what skills were required to solve the item, and compare those skills to the FCAT item specs listed online? Why did WFTV (and I, for that matter) miss the possibility that the student misstated the item, especially given that she gave the wrong answer for the item she posed?
Or was the point here just to have a hook on which to hang two "critics say" claims and convince the audience that the FCAT is bad because Governor Bush was stumped by one item?
From Business Week Online, a nice profile of ETS founder Henry Chauncey:
In the early 1930s, Harvard University's graduating classes were made up of young men and (a few) women who had spent their teen years reveling in the heady 1920s. It seemed to one young Harvard assistant dean, Henry Chauncey, that these sons and daughters of the elite were simply expecting that they would rightfully inherit top positions in business and society, as if the Great Depression couldn't touch the ruling class.
Chauncey, a Harvard grad himself, was distressed to watch as class after left the august institution, and failed, in his view, to meet the titanic civic challenges of the times. Together with then-Harvard president James Bryant Conant, Chauncey initiated an experiment to bring Harvard a new type of student, based not on the connections they or their parents had, but solely on what the students knew and their potential for further learning.
Many people would like to forget (if indeed they ever knew) the proud principles upon which ETS was based. Yes, there are still inequalities in our educational system, and students with more wealth and health and good fortune still tend to do better. But it's a far cry from the days where, if you were very smart but from a poor family in Nowhere, Kansas, you were completely out of luck, and tests like these have a lot to do with that.
One elementary school in Vermont isn't interested in the state standardized exams, and believes that surveys - yes, surveys - are the way to see if they're really being effective:
In an attempt to measure Marlboro Elementary School's quality without federally mandated exams, surveys will be sent out to graduates from the past eight years. A draft of the survey has been written by Marlboro Elementary alumnus Ariel Poster, who is a sophomore at Barnard College in New York City, a women's college affiliated with Columbia University. She was hired by the Marlboro Elementary School Action Plan Committee to evaluate the school's ability to educate.
"The School Board does not believe that proposed federally mandated testing is an accurate or useful form of evaluation," Poster wrote. The effort comes two months after the Marlboro School Board announced they would say no to federally mandated standardized tests unless they are educationally beneficial.
"I think that'll gather us some very good information," school board member Andy Reischman said at Tuesday's meeting.
The draft featured eight questions, but is expected to be expanded, said Marlboro Elementary Principal Francie Marbury. The draft only asked positive questions, and negative ones are expected to be added.
It started off in a flattering fashion, reading, "Congratulations, you are an incredible person! You had the unique opportunity to spend a huge portion of your childhood learning and growing at Marlboro Elementary School and the Marlboro community is extremely proud of your accomplishments."
Before anything is sent out, the school needs to come up with a list of alumni. The school currently has about 77 students.
The tricky part will be finding all the graduates, said board member Lauren Poster, Ariel's mother. The survey will go to high school students and graduates old enough to be in college.
Oh, this I gotta see. A survey. Are these folks aware that a response rate of 25% is considered normal for surveys? Are they aware that respondents self-select, and that only those who were extremely satisifed/dissatisfied may bother with responding? Are they going to account for the fact that a respondent's recollections and impressions of their time in elementary school, so far after the fact, may not only be incorrect, but also have nothing to do with how well the school did in educating students as a whole? Why is it more important, or more meaningful, to ask former students what they remember of the experience, than to see how current students are doing now?
And how 'bout that "you are an incredible person!" opener? This survey is going to high school and college students. You'd think Marlboro wouldn't feel the need to use the grade-school touchy-feely tone with more mature students. And what if one of those alumni is going through a rough time? If I'd just gotten fired, dumped, or rejected by the college of my choice, some goofy letter that told me how proud my elementary school was of my "accomplishments" would go straight into the ol' circular file.
In May, the school board announced that the school would no longer administer any tests which the principal finds void of educational value, participate in Adequate Yearly Progress as determined by the act, or forward any information to the Windham Central Supervisory Union that can be connected to a specific student's name...
The school will not be in jeopardy until they refuse to hand out tests. The state will be forced to act if the school refuses to administer state-mandated standardized tests.
The draft said that the surveys are due back at the end of the month, but it was unclear Tuesday whether that date would be pushed.
A CD-ROM will accompany the survey, with the top 10 reasons the graduates should fill them out.
A CD-ROM? Well, that'll weed out any respondents who don't have easy access to a computer. And, face it, there's no pressure for students to respond, much less respond honestly. I eagerly await any data that might be produced from this little experiment.
Update: This interview with Windham Central Supervisory Union assistant superintendent Jim Peters contains a very odd quote on this survey topic:
The federal government, he said, has a bigger need to quantify everything in order to assess a school's quality. The method of standardized testing is something one school in his district has taken a blunt stand against.
Marlboro Elementary's school board decided in May to just say to no to federally mandated standardized tests unless they provide educational benefits. Surveys will go out to the elementary school's alumni from the past eight years, in an attempt to assess its quality.
"It's not surprising that Vermont has embraced portfolio assessment, because it speaks to parents' needs to see examples and samples of what their child knows and is able to do, instead of an annual achievement test," he said.
Stop right there. A survey is not the same thing as a portfolio. The article from Wednesday did not indicate that anything other than surveys were replacing the state standardized exams. A survey is most definitely not an assessment and it does not fall under a collection of work produced by a student. I still think the survey idea is ridiculous, but if Vermont is indeed using some type of portfolio assessment in addition to the survey - something that wasn't made clear in either article - that makes the whole thing seem not quite so ridiculous.
John Jay Ray at Dissecting Leftism discovered two good education-related posts today. The first, from Dangerous Liberty, is from someone who's been writing about education in Florida for 10 years, and isn't happy with the anti-testing movement:
....Apparently, it is impossible to criticize public education without hating it. I don't hate public education. I want it to be better than it is.
And so, here we are today, and educators and Democrats are using the Florida Comprehensive Assessment Test as their whipping boy as if the fact that this remedy was deemed necessary comes as a total shock to them; as if there were no red flags, no warning bells, nothing whatsoever to alert them to the fact that our kids have, for a long time now, been in serious academic trouble and that the policies of government schools have, for a very long time, been the agency of this trouble.
Let's talk a little bit about FCAT, this test that has surprised and dismayed so many people.
Educators should have foreseen something like the FCAT. The National Assessment of Educational Progress has been showing us a more and more dire picture over the last decade or two. More kids are dropping out. Fewer of them can write or read well enough to hope to prosper in the real world. More people have begun to notice and to complain.
And yet, when we get the wonderful news that of the 230 kids who repeated third grade after failing the FCAT once, 147 passed it, the entire focus is on the 83 who failed the test again and hand-wringing abounds. The concern is for these children's sense of self-esteem, but what self-esteem can they have if they get out of school unable to read?
Why, they can have the kind of self-esteem that some educators believe in, that comes not from accomplishments but from constantly being told you are fine just as you are, and that you always know best, even if you don't know anything. Other people might term that "narcissism," but hey, why quibble over the fine details?
And then we get the story that the “FCAT is frustrating seniors' plans” (May 10). We learn that a student with a B/C average in high school “has yet” to pass the test.
“It's like you went to school for twelve years of nothing,” says this high school senior.
The observation hits ya right between the eyes, doesn't it? If she has a B/C average and still cannot pass the FCAT after two years and four more opportunities to take the test, she evidently did go to school for 12 years of nothing!
But blaming the test is easier, and it makes for snappier quotes.
I understand that any test is going to be imperfect and that there are bugs to be worked out of the system, but we're seeing some real gains happening that moves me to wonder what the true problem is here. Could it be simply that teachers and schools are not used to being held accountable?
Mmmm, I think this writer is on to something.
The second article is by a guest commentator, Tina Blue, on the Irascible Professor's site. After reading it, boy, do I see why she's Blue:
Those of us who teach college English classes are always overwhelmed by the astonishing deficits in our students' ability to get their facts straight, to think clearly and logically, and to express their ideas in language that actually makes sense and that follows the most basic rules of grammar.
Even our best students write incoherent essays and make grammar and usage errors that would have failed a third grader in the 1950s, when I was in grade school. They get to college with such writing deficits, of course, because no one has ever required them to learn how to write any better, and no one has ever penalized them for making such errors...
Here is an absolutely true story.
When I was in third grade (1958), our teacher, Mrs. Colona, would come in twice a week and present us with an essay topic. We had no prior notice of when we would write our essay or what the topic would be. What we had was 45 minutes to write a 500-word essay on that topic, and we were required to do it right. Mrs. Colona took off points for everything we did wrong. We had to follow her formatting instructions to a T, and if we put our names in the wrong place, or didn't leave appropriate margins, if we forgot to number our pages, or if we wrote in pencil rather than ink, we lost points.
We also lost points for errors in grammar and usage, for structural flaws, for logical lapses, and for stylistic weakness (e.g., writing short, choppy, repetitious "Dick and Jane" sentences or using vague or inappropriate words).
Now, Mrs. Colona did not give us "deep" topics to write about. One I remember was "Write about your favorite holiday memory, and explain what makes that memory so special to you." But you know what? Almost none of the college students I have taught since 1972 could write a 500-word essay on that topic in 45 minutes without that essay's being marred by numerous errors or infelicities in one or more of the following areas: grammar and usage, diction, style, formatting, structure, and logic. I know this for a fact, because we often do give such simple topics as the first essay assignment in English 101.
Why, I see the problem right there. They aren't giving "deep" enough topics these days, for those "deep thinkers" with "deep minds" who can't be expected to do well on standardized tests, much less essays. After all, what is self-discipline, structure, dedication, and instant, objective feedback when compared to the merits of having a "deep" mind?
For more essays by Ms. Blue, visit her Teacher, Teacher website. I've read through some of them, and let me tell you, Ms. Blue has NO problems with expressing exactly what's on her mind in language that is grammatically correct and to the point. And I agree with everything I've read so far.
Devoted Reader Erin T. sent along an email in which she expressed her astonishment about an anti-testing site on the web. Why, I never knew such things existed, did you?The author of this website, Chris Carter, cites Wacky Alfie Kohn's books approvingly, and also posted this "press release" to a teacher's e-board, which is how Erin found it:
I have created a website that summarizes the critical literature regarding standardized admissions tests such as the SAT, GMAT, and so forth.
Because, Lord knows, we need more criticism of testing out there, and it needs to be easier to find. I mean, doesn't it get to you how reporters fall all over themselves to print only the good things about tests?
With few exceptions, students who wish to attend professional schools anywhere in the world, or to attend almost any university in the US, are forced to write these tests.
"Write these tests"? "Forced"? I love it when someone restates a requirement as being "forced" to do something, especially when schools vary widely as to how much emphasis they place on test scores.
I think it is our duty as educators to be familiar with the case against these tests.
I would think it was your duty as educators to be familiar with the research and hard data surrounding these tests, and be familiar with the cases for and against testing. Is there any particular reason that educators should have only this one viewpoint?
If you read my article, you will see that these tests have no validity as predictors of actual accomplishment in any field.
That's because SAT scores predict college grades, which aren't necessarily linked to later performance. Thus, the flurry of low to negative correlations between scores and later accomplishment that the author cites in this article are beside the point. The SAT has never claimed to predict success in life, so criticizing it for failing to do so is incorrect. What's more, for someone who goes to a lot of trouble to explain what a correlation is, Carter leaves out a discussion of restriction of range, possibly because to do so would leave the door open for contradiction of his theories.
An extended discussion of restriction of range wouldn't be appropriate here, but to sum it up quickly, a correlation is a measure of how multiple variables co-relate, or co-vary. If one of those variables has restricted variance, the correlation of that variable with any other variable will be "restricted" or lowered (closer to zero). If a variable does not vary, it cannot co-vary.
SAT scores for college students are restricted, because, for the most part, if you have a low SAT score, you don't get in. So, as a hypothetical example, let's say that most everyone who goes to Harvard has an SAT of higher than 1200. That leaves us with scores of between 1200 to 1600 to correlate with some measure of college success, or later success in life. Given that even smart people will screw up, fail to be "successful," or simply choose to stay out of the rat race, it's very possible for a Harvard grad with an SAT of 1200 will do fine, while one with an SAT of 1500 may drop out, or go bankrupt years later. That results in lowered correlations, but it doesn't necessarily follow from this that the SAT is not useful in college admissions. SAT scores tend to correlate with other measures of intelligence, and as long as we believe intelligence affects college performance, then colleges will have more success with high-SAT admittees than low-SAT scorers.
To continue on with Carter's criticisms:
They have extremely limited validity as predictors of first-year college grades;
For what school? Every schools weights the SAT and ACT differently, because those tests hold different predictive validity for different populations. To average over all schools is to again mislead the reader. And the author reports that, "The SAT has the most predictive validity of the tests1, with correlation coefficients ranging from .2 to .5 at most (R-squared ranging from .04 to .25)," as though this were a bad thing. Obviously, he's unaware of just how rare a correlation of .5 is in the social sciences, especially for one test, taken on one day, with a limited number of items. It's such a tiny snapshot of performance that the correlations of .2 to .5 are just amazing.
Does Carter know of any other snapshot that is this cheap, standardized, and quick for schools to use that will have that high of a correlation with college grades?
...and they are biased against women, minorities, and the poor.
Oh, again with the bias. As OpinionJournal likes to point out, if the world were to come to an end, the NYT would print the headline, "Armageddon arrives; women, minorities hit hardest." I have to give Carter credit for trying to define bias in his article, but then he wedges the idea of bias in where it shouldn't belong, here:
Are the tests biased against the poor? Well, it depends on what you mean by "bias." The poor certainly do not score as highly on average as wealthy students. Over the last forty years SAT scores have been positively correlated with family income...So the SAT appears biased against the poor in the sense that the poor tend to score lower and therefore will be less likely to be admitted to the college of their choice.
Sorry, but that's a definition of "differential impact," not bias. And while differential impact IS a topic that should be discussed thoughtfully, to lump this kind of effect in under bias is misleading. If the "poor" simply do not learn as much - quite possible given the likelihood of deprived homes and poorly-funded schools - then the test is actually performing correctly in assigning them lower scores.
Perhaps most surprisingly, there is evidence that these tests are biased against students with deep minds.
Pardon me while I snicker uncontrollably. "Deep minds" sounds like a concept you think about while passing a bong around. The SAT is test of basic skills which are very likely to come in handy for college classes. Will very smart - forgive me, "deep" - students find the test boring? Probably. Will it be less than useful for predicting how those students at the very high end of ability do in school? Most likely. But the only way that "deep" students will bomb the SAT in large numbers is if they fail to learn basic geometry and algebra, or how to discern the main point of a paragraph.
Anyway, that's my take. I also just have to point out that, although I started N2P as a way of rebutting anti-testing articles, I don't believe all test criticism is bad. There's plenty of room for debate on issues of standard setting, high stakes for young kids, differential impact as mentioned above, and so forth, and it would be great to see a website that discusses these issues without resorting to that old journalistic standby, "Critics say".
Unfortunately, Carter's site looks like it will just be rehashing the myths and bashing tests unconditionally, while not providing much of an alternative for states that want all their students to meet certain standards, or universities that are flooded with thousands of applicants each year. If testing is so bad, what's the best alternative? Carter believes it is, "Samples of work, references, statements of purpose, and extra-curricular activities," all of which are fine, but not necessarily verifiable, or comparable across students, or shown to be predictive of college success. Never fails to amaze me how people who will nitpick to three decimal places the predictive validity of the SAT will offer up, as an alternative, things like "statements of purpose" for which no predictive reliability data exist.
Oh, wait, before I go, I just have to quote this portion of his article, if only because I am amazed that he believes it is correct, or to the point:
Incidentally, despite having a mailing address in Princeton, New Jersey, ETS has no connection with Princeton University. Its luxurious headquarters, including tennis courts, a swimming pool and a private hotel, are in Lawrence Township, not Princeton. The Princeton mailing address is merely for public relations.
How EVIL! Actually, the Princeton link is very simple. In the 1920's, Carl C. Brigham, the Princeton professor who published A Study of American Intelligence, came up with his own version of the Army Intelligence exam to use as an admissions test for Princeton freshmen. Brigham was hired by the College Board (which is in NYC) to lead a committee to develop the test that eventually became the SAT, which was administered for the first time in 1926 - 22 years before ETS officially opened its doors.
The two founders of ETS, Henry Chauncey and James Conant, met Brigham in 1933 when they traveled to Princeton and decided the SAT could be of use for Harvard students, and the rest is history. Why they incorporated in Princeton rather than Boston, I'm not sure, but the first ETS office was in fact in Princeton proper, in Professor Brigham's original space on Nassau Street:
By the middle '50s, ETS was cranking out not only the SAT, but the Graduate Record Exam, Law School Admission Test and a host of Foreign Service and military exams - each exam swelling in importance with the size of the American educational establishment. The company was badly outgrowing its Princeton headquarters. Often, tests had to be prepared at the nearby firehouse on Chambers Street. When the fire alarm rang, employees had to push their work tables out of the way and make room for the engines.
The need to move was obvious. In 1954, Chauncey had a vision of its future when he took a hike along Stony Brook and saw a stretch of open farmland that looked perfect for ETS headquarters. ETS' move to its 360-acre campus in Lawrence Township was complete by 1958. It was a complex of low-rise, modern, brick buildings, but for the first few years it also shared space with a working dairy farm.
Therefore, the claim that ETS has no connection to Princeton University, or that ETS chose Princeton purely for PR, is both laughable and easily disproved through a bit of Googling. But I suppose the truth didn't fit with Carter's meta-theory about how all psychometricians are eeeevil capitalists, though.
The good news: After much dithering, the cutpoint for Georgia's CRCT reading exam for third-graders has been released (or, at the very least, leaked to the press).
The bad news: It's a pretty low cutpoint, which puts into perspective Georgia's triumphant announcement that 91% of their students passed.
To meet expectations, third-grade students this year needed to answer 17 of 40 questions correctly — just 42.5 percent. In Glynn County, 89 percent of students met that requirement on the reading test and will matriculate to fourth grade. The remaining 11 percent were targeted for summer school and took the test again Tuesday, according to a new state mandate that requires third-grade students to pass the reading test before moving on to fourth grade.
Generally, cut scores fall between 40 and 60 percent, said Kirk Englehardt, spokesperson for the Georgia Department of Education.
"What it's supposed to do is help us identify when students meet the minimum competencies on a test," he said. "That says you're on level. You're where you're supposed to be."
It's a minimum-competency exam. Fine. Now give us some measures of variability. What does the distribution of test scores look like? What's the standard deviation? How many students are within a few items of the cutpoint? Parents need this information to judge the claims of the state that 91% of its students are really ready to move forward.
Administrators at some South Carolina schools won't waste this summer at the beach; they're already planning the upcoming school year, with an eye towards closing the racial achievement gap, which continues to widen in the state:
It's summertime at South Elementary School. The halls are silent and empty. But behind closed doors, teachers, maintenance workers and school secretaries are busy, setting up classrooms and thinking about the coming year weeks before school begins in August...
South Elementary is one of 107 schools in South Carolina that are "closing the gap" between the test scores of black and white students and the haves and have-nots, according to a study released Thursday by the state agency that oversees the implementation of education legislation. The Education Oversight Committee report said black students and students who receive free and reduced-price lunches are likely to fall farther behind their peers in math and reading in the next 10 years unless "dramatic" corrective action is taken...
Some of the report's recommendations include increasing instruction time for students who are at risk of falling behind, developing academic assistance plans for each child and improving programs for 4-year-olds to prepare them for school.
At some SC schools, students in the disadvantaged categories perform well on standardized tests, and other school hope to follow their leads. More on some of the successful schools here:
The biggest local winner was Stono Park Elementary in West Ashley, which has a school population that is 84 percent black.
It posted high levels of performance across grade levels and subjects for both black and poor students.
"It is our passion that our kids will do well," said Principal Stephanie Strous, one of three principals statewide selected to speak to EOC members Thursday. "All kids can achieve. As principal, it is my goal to prove it."
She summed up her philosophy in two words: "We believe." Order and structure are also important, she said. "These seven (school) hours, we control. We don't wring our hands. We make use of it."
You say you want to know what the passing score is on Georgia's high-stakes reading exam for third-graders? Well, nyah-nyah, they're not going to tell you:
For weeks, the state Department of Education has refused to say how many questions Georgia's third-graders needed to answer correctly in order to pass a high-stakes reading test required for promotion to the fourth grade. Without knowing where the state set the standard, many observers say it is unclear what Georgia's surprisingly good performance on the test means.
Georgia officials were overjoyed that 91 percent of the state's 115,000 third-graders passed the proficiency exam. The state had expected as many as 26,000 students to fail the test, which was given last spring...
State Superintendent of Schools Kathy Cox says the state did not manipulate the test to improve performance. The cut score — the number of correct answers a student must have in order to pass — was the same this year as in 2002, the last time the third-grade reading test was given, she said.
Yes, but while that tells us the standard wasn't lowered from 2002, that still doesn't tell us what it is. Keeping the cutscore in place doesn't guarantee that everyone actually improved, because the items could have gotten easier.
Standardized tests are not graded the same as classroom tests, which typically have a scale of 100 with a score of 70 being the minimum required to pass. Tests like the state's curriculum exam have determined cut scores based on how much a group of educators decides students need to know in order to prove they have mastered the subject.
On Gwinnett County's standardized Gateway exam, used for promotion in grades four and seven, a passing score is less than 50 percent of correct answers. The cut score is low because the questions are difficult, said Linda Mitchell, head of testing in Gwinnett.
It's quite possible that there's no reason for the hush-hush other than to satisfy the lawyers. I'm not going to jump on the conspiracy bandwagon. But knowing the placement of the cutscore does allow for more argument about how useful the test is, so naturally some have jumped to the conclusion that the DOE wants to forstall those conversations.
Say it's revealed that the cutscore is 40% of the items. This could mean the items were very difficult, or it could mean the cutscore is so low that almost anyone can pass. Since everyone's seen the test, the public has a shot at gauging the difficulty of the test, and can use the cutscore knowledge to get an estimate of what "proficient" really means. Texas reveals their cutscore, and it isn't any sort of infringement on copyrights or test security to do so.
On the heels of Philadelphia's crackdown on misbehaving students comes some good news - Philly's test scores are up:
The TerraNova test results, says school district CEO Paul Vallas, find that student scores continue to improve across the board:
"In eight of ten grades, we saw improvement in reading scores. In nine of ten grades we saw improvement in language arts scores. And in seven of nine grades we saw improvement in math scores.
"While we saw solid growth in our reading scores, the growth in math was very strong. And let me point out that the strongest growth was in the middle grades -- 6th, 7th, 8th, and 9th grades."
This Times-Record (AR) article is about cultural biases on tests, but no evidence is provided to support the charges:
In remarks in Van Buren last week, President Bush said he is committed to narrowing an “achievement gap” demonstrated in standardized test scores among the nation’s students. To some extent, the gap is the legacy of decades of segregation, some area educators said...
Jim Hattabaugh, Mansfield superintendent, said his experience as a guidance counselor and administrator has shown him that most standardized tests have cultural and racial biases. Minorities tend to perform worse than whites on the test because the tests are authored by whites, he said.
And does he have proof of that? Does he know for a fact that each and every single item writer is white? Does he have any knowledge about the extensive item bias review that all standardized test items undergo? Is he aware that item writers, test developers, and psychometricians come in all sizes, shapes, sexes, and ethnicities? Does he have any evidence whatsoever to prove the causal relationship between the skin color of the item writer and performance of examinees of different colors on the exam?
I doubt it. He's saying this because he knows the reporter will not challenge him to provide proof.
“If you look at how an inner-city student is raised and what they’re exposed to, you ask those questions on a test and they don’t have a clue,” he said.
And why is the only legitimate explanation for that cultural bias? Couldn't it also be because those inner-city schools are not teaching children to read, they're failing to introducing them to concepts outside their narrow environment, they're not expanding their vocabularies, and they're teaching these kids that you have to "be white" to do well on tests?
“There could be some possibility of a cultural bias,” said Lavaca Superintendent Harvie Nichols. “I know, for example, when I look at textbooks, it’s difficult to understand, even with a college degree, what the textbook wants that child to do.”
What the heck does this mean? It could be that it's a bad textbook. It could be that Nichols college degree wasn't too useful. But why should we assume this anecdote proves that textbooks are culturally biased? Textbooks undergo even more review for bias than do test items. If Nichols can't make heads or tails of the textbook, then he should order new ones, instead of insisting that cultural bias must be to blame.
This is an example of shoddy reporting. The claims of cultural bias against tests in general - including the outrageous and inaccurate charge that only white people write test items - go unchallenged by the reporter, as does the unspoken assumption that inner-city students should never be expected to understand material that is not specific to their very narrow sphere of experience.
St. Petersburg Times columnist Howard Troxler takes a look at all the recent FCAT numbers:
The most important news was that for the first time, more than half of Florida's kids in grades 3-10 are reading at or above their grade level. To be exact, it is 51 percent.
A naysayer might look at the same number and say: That's terrible! You mean that 49 percent of Florida's kids are NOT reading at their grade level? Indeed, as if on cue, Florida's Democrats issued a statement finding fault. "Victory!" sneered the Democrats' sarcastic headline. "Half of Florida Kids Can Read!!!"
Does this mean the Democrats agree that the FCAT Reading test does in fact test genuine reading skills? Hee hee.
Really, I thought Bush was fairly frank and realistic about the numbers he presented. For the most part they represent slow and incremental improvement...The best news in Bush's numbers came in the lower grades, where the most emphasis has been placed on reading.
But that is a nice way of saying that the latter grades ... well, stink.
There also still is an enormous performance gap between white and minority students. Only 32 percent of black kids are reading at grade level and 42 percent of Hispanics, compared to 63 percent of white kids. The numbers for math are similar.
In other words, a black kid in Florida is still only half as likely as a white kid to be reading at grade level. Find all the economic, cultural or educational excuses you want - this is a bedrock problem for Florida.
Do the Democrats have a solution to that, other than tossing the tests which spotlight the reading gap?
Still trying to get caught up on my "blogligations" here...
-----
Power to the people, man! Modesto High students protest standardized testing policies. You'd think I'd be mocking them, but the kids have their heads on straight. Their complaints - that the tests they must score "proficient" or above on are not in sync with their International Baccalaureate program - are pretty sensible. Their black t-shirts are pretty spiffy. And they're wearing these t-shirts as part of a silent protest. Teenagers who use common sense, wear black, and understand the power of silence - they've converted me to their cause already.
-----
High school GPAs are up, nationwide; test scores aren't. This editorial in the Las Vegas Review Journal wonders why.
-----
An Arizona State University survey reveals that parents like the idea of ranking schools with test scores and imposing exit exams. Ethnicity didn't matter, and the survey was conducted in both Spanish and English. It's a telephone survey, though, which means that people with only cell phones, or no phones, were left out.
-----
If you're an NYC third-grader and you missed the first reading test, the makeup is on May 12th. Be there, or be in third grade again next year.
-----
North Carolina is inching closer to dropping a test for teachers. But the Governor is fighting the change in plans, which was apparently pushed through to help battle a teacher shortage.
-----
Finally, in Louisiana's St. Charles parish, three teachers have flunked a different, very important kind of test:
...three teachers became cheaters themselves, betraying their profession, their school district and, worst of all, the children in their charge.
A teacher at Norco Elementary School gave answers to 15 fifth-graders who were taking the Iowa Test of Basic Skills last month, according to state education officials. A teacher at Ethel Schoeffner Elementary School in Destrehan looked at questions on the Iowa test and gave 15 sixth-graders examples of the types of questions they would be asked, officials said...
These teachers set a terrible example for their students. Ironically, it was the children who brought the incidents to light by telling another teacher what had happened. They showed courage and honesty in doing so, but they shouldn't have been put in that position.
From this article on the STAR in California comes an odd, unquestioned testing criticism:
Sergio Miguel remembers all too well what it was like taking tests in a foreign language, something nearly 30 percent of Tulare County students do. Miguel moved to Lindsay from Mexico when he was a high school freshman.
"It was extremely difficult to learn English," he said.
Nevertheless, Miguel learned English at Lindsay High School, went on to College of the Sequoias and got his bachelor's degree at California State University, Fresno.
Now back at Lindsay High, where he teaches math, Miguel watched last week as his students who speak limited English struggled on standardized tests, re-quired by California law to be in English, the very same tests that determine statewide school rankings.
He watched, not able to help.
"There is a huge frustration for me," he said. "If they were given a test in Spanish, without doubt their scores would be way ahead of their scores now."
Does Miguel not see that the reason he was able to succeed in our society was because Lindsay High School insisted on teaching him English? His students might very well do fine on the test if it were given in Spanish, but so what? Does that mean his students would have the tools necessary to succeed in our society? No.
That much said, I'm aware the tests aren't perfect, and testing does not in and of itself improve education. But I wonder if so many teachers would be "throwing their hands in the air" if there wasn't the constant, not-quite-accurate refrain about how these tests only measure affluence. Schools, and educators, who insist that accountability measures reflect only the home environment make me wonder why we should be paying their salaries.