When the stakes are high, do the results make sense? Jay Greene, Marcus Winters, and Greg Forster address the issue:
Several objections have been raised against using standardized testing for accountability purposes. Most concerns about high-stakes testing revolve around the adverse incentives created by the tests. Some have worried that pressures to produce gains in test scores have led to poor test designs or questionable revisions in test designs that exaggerate student achievement...Others have written that instead of teaching generally useful skills, teachers are teaching skills that are unique only to a particular test...Still others have directly questioned the integrity of those administering and scoring the high-stakes tests, suggesting that cheating has produced much of the claimed rise in student achievement on such exams..Most of these criticisms fail to withstand scrutiny...This study differs from other analyses in that it focuses on the comparison of school-level results on high-stakes tests and commercially designed low-stakes tests. By focusing on school-level results we are comparing test results from the same or similar students, reducing the danger that population differences may hinder the comparison. Examining school-level results also allows for a more precise correlation of the different kinds of test results than is possible by looking only at state-level results, which provide fewer observations for analysis...
The conclusion? That, within a school, correlations between high- and low-stakes tests tend to be large and postive:
The finding that high- and low-stakes tests produce very similar score level results tells us that the stakes of the tests do not distort information about the general level at which students are performing. If high-stakes testing is only being used to assure that students can perform at certain academic levels, then the results of those high-stakes tests appear to be reliable policy tools. The generally strong correlations between score levels on high- and low-stakes tests in all the school systems we examined suggest that teaching to the test, cheating, or other manipulations are not causing high-stakes tests to produce results that look very different from tests where there are no incentives for distortion.Posted by kswygert at March 20, 2006 10:41 AM