Criterion-related tests now a requirement?
The NYTimes columnist Richard Rothstein informs us that a group of Democratic senators are "insisting that the new federal education law be interpreted as requiring the use of criteria-referenced tests." Those of you who have visited my site before know that I have posted a few times explaining the difference between norm-referenced and criterion-referenced tests. I won't explain it again here, because Mr. Rothstein does a good job of it.
I can't say I'm as impressed with the rest of what he says, though. He's basically stating an argument for not using criterion-referenced tests because of their supposed "flawed proficiency definitions and limited ability to detect progress." However, much of what he has to say this is critical of criterion-related tests has nothing to do with the inherent design of such tests. Yes, administrators can fudge the categories afterwards (these are the complaints I'm hearing from parents in Texas). Yes, categorizing studies is a much less distinct way of assessing them than giving percentile rankings. Yes, if test norms are held constant from year to year, a norm-referenced scale can help in assessing student progress (this is why we can compare current SAT performance to previous performance, at least back until the scale was tweaked).
However, the only requirement with criterion-based tests is that the test-takers are compared to how well they perform on a set of standards. The categorization can be set as broadly or as fine as the administrators wish. He gives an example that supposedly defies common sense: " [Suppose] 'proficient' scores are achieved by 40 percent of third and fifth graders alike but by only 30 percent of fourth graders. If the criteria were valid, such results would suggest (implausibly) that fourth-grade teachers were awful but fifth-grade teachers terrific." It could also suggest that the fourth-grade standards are set too high, either erroneously or just because fourth grade is a hard grade. It could suggest that the fourth-grade teachers are using a method that is not as effective. Nothing in that scenario is implausible, much less in defiance of "common sense".
I can understand his wish to remove any disadvantages or negative ramifications suffered by kids who improve from year to year but never manage to make it into the "proficient" category. The problems, though, are with the way categories are set and the ramifications of placing kids in categories, not the criterion-based nature of the test itself.