May 16, 2002

News from the Lone Star

News from the Lone Star state

Alert reader Dudley Crawford sent me the link for this article, ostensibly about the lowering of standards on the TAAS. However, it seems that the reason students now have to answer fewer question correctly in order to pass is because the test is harder, not because the standards have been lowered.

I sometimes encounter potential test-takers for the standardized test that my company produces, and they often ask, "So, do you set the curve [their term, not ours] for the test before or after you give it?" The answer is "Both", meaning that we do our best to assemble tests that are all of the same difficulty level, but we adjust the conversion of number-right to final score afterwards to take into account any fluctuations of test difficulty. Therefore, if we were to give a test this year that was more difficult than last year's test, we would adjust the conversion so that you could get the same score with fewer items correct.

That much said, it's not a good idea to just toss off a more difficult test and assume you can correct it afterwards, because the impact on the test taker is different. I hate to use the phrase "It's not fair" here, because I hear that bogus, unsupported complaint so often in regards to testing, but it really isn't fair to give tests so different in difficulty that students can answer 10 fewer right out of 56 in order to pass. Yes, the process of equating is standard for these kinds of tests, but 10 points is a huge difference on a 56-item test. A more difficult test can be more nerve-wracking to the test taker, especially if they're expecting a test similar to those they've seen in the past. The test I work on has almost twice than many items, and our number-right conversions don't fluctuate by more than three or four points. A difference of over 17% of the items is an indication that the test is not well-assembled - perhaps there were not enough low-difficulty items in the item pool. In that situation, I would do some investigation to make sure that the more difficult test is in fact measuring the same thing as the previous tests.

Also surprising - the TAAS is being phased out next year in order to make way for a new test. First I've heard of it (although I don't have any psychometric informants in Texas right now).

Posted by kswygert at May 16, 2002 11:16 AM
Sitemeter