Two more blogs to read every day
If you're already short of time, too bad, because there are two newbie blogs that you must check out.
The first is the international blog Giants and Dwarfs, which is about "high politics, low culture, and everything in between". I particularly like the commentary on anti-Semitism on campus and the shortcomings of the vaunted Oxford tutorial system (their archives and perma-links don't seem to be working yet).
Next up is The Irascible Professor (no, not The Cranky Professor, this is someone else entirely), who provides "Irreverent Commentary on the State of Education in America Today". I found a nifty SAT-related article on his site; it's from September, but it's interesting enough that I want to comment on it now.
According to the College Board web site, there are three major changes planned for the New SAT...The SAT Verbal Exam will become the SAT Critical Reading Exam which will no longer include analogies. In their place, short reading sections will be added to existing long reading passages. As it states on the web site, "While analogical reasoning is important, analogies are rarely taught in schools and these questions have become rather artificial." I was considering writing to programs like the Word Masters Challenge, a national competition for students in grades 3-8 which encourages growth in vocabulary and verbal reasoning...According to WordMastersChallenge.com, "unlike other language-arts contests, which emphasize punctuation and grammar, this competition addresses word comprehension and logical abilities such as those measured for high school students by the verbal SAT I." Not anymore!
I agree with IP's frustration, but I also see this decision as a "damned if you do, damned if you don't" for ETS. After all, one of the main criticisms of standardized tests is of their artificiality, and if in fact analogies are not taught in high school - or college - and ETS finds it difficult to defend the use of them in college admissions, then they're going to drop that item type. I don't know of any data showing the correlation of that particular item type to college GPA, but I assume that ETS no longer felt the item was statistically defensible.
The New SAT will also have a new section called the SAT Writing Exam. This section will contain multiple-choice grammar questions as well as a written essay...In spite of their best efforts, I finished studying the College Board pages about the New SAT, and I was still filled with questions. They provided an email address for comments and questions, so I took advantage of it...The first prompt and professional reply I received was from an executive at the PSAT/NMSQT Programs at the College Board...For me, a discouraging part of her reply was about the essay. She said that on the PSAT, they would be offering an essay for in-school scoring with all the rubrics and support to enable teachers to both learn how these essays will be scored nationally and then to provide the scoring for their students. She added that it was a good professional development exercise as well as practice for the students. My interpretation of this is that the College Board will not be grading the essay on the PSAT; that will be left up to the teachers at the individual schools.
That's very interesting - I wasn't aware of that wrinkle in the grading scheme. I had assumed that the College Board and ETS were going to be using the new automated-essay scoring program that ETS has spent a great deal of time and money developing. Perhaps they aren't going to be using this system on the PSAT - only on the SAT. But there is another possibility, which is that ETS is going to be training teachers to be essay judges, and those results are then going to be used to improve the performance of the automated essay scoring system. I'm currently involved with the development of such a system, and it seems to be standard practice to assess the performance of the automated system by comparing the automated results to a set of human rater results for the same essays. Perhaps the first year(s) of the PSAT essays will be double-scored, by teachers and by an automated system.
The second response was from an employee at the SAT Program...She closed by saying that graduates of 2006 may be required by universities to take the New SAT, that it will be up to the individual admissions office. She said Admission Offices will be able to equate the math and verbal portions of the old SAT with the new SAT because the math, critical reading, and writing scores will still be scored on a 200-800 scale....With that reply, some more of my lovely tresses turned gray, but I have a reputation of overreacting to change. I suppose the psychometricians will work very hard to make sure a 650 on the old SAT Verbal test will be equal to a 650 on the new SAT Verbal test. I am still uncomfortable about the writing sample and I wonder how many seasons it will take their staff to read and rate the essays. How many snow days will pass until the results are mailed, and how accurate is a test that requires subjective grading?
IP has reason to be nervous about the subjective grading, but not so much about any equating of multiple-choice items. Equating is a very well-researched topic in testing, and multiple-choice items tend to be the easiest to equate, as they tend to have the highest reliability. Essays, on the other hand, are harder to equate, from both a prompt-difficulty standpoint and a grading stringency standpoint. It's going to take some work to ensure that every student receives an essay prompt that is of equal difficulty, or as equal as possible. Even if the prompts are relatively equal, differences in grading stringency will certainly affect the resulting scores. The purpose of an automated essay scoring system is to provide comparable scoring stringency across essays in the shortest amount of time, and I hope that ETS has plans to employ their system as soon as possible, rather than relying heavily on teachers as raters.