Edu-bloggers Daryl Cobranchi and Bas Braams are all over the TAKS, now that a defective item was just recalled on the 10th-grade math portion of the exam. The official Texas Education Agency press release is here.
What went wrong in the Lone Star State? Seems those dadgum octagons were confusing everyone. The incorrect item (number 8 in this document) asks students to find the perimeter (to the nearest centimeter) of the regular octagon shown. If you knew the length of one side, this would be easy, but students were provided only with the inscribed radius (distance center to flat edge) and circumscribed radius (distance center to angle between edges).
Problem is, the data as provided are, as Bas will be happy to explain to you, contradictory. For the inscribed radius of a regular octagon to be 4 cm, the circumscribed radius would have to be 4.33 cm, not 4.6 as shown:
Taking the 4.0cm and 4.6cm at face value a student might reason that the perimeter of the octagon is somewhere between 2*pi*4.0cm and 2*pi*4.6cm, and this leads to the answer 27cm in the multiple choice format. Or the student could apply trigonometry and obtain perimeter 26.5cm by starting from the given inscribed radius or 28.2cm by starting from the given circumscribed radius. A fourth approach is to use Pythagoras's theorem on a right triangle that has hypothenuse 4.6cm and one right side 4.0cm; then one finds that the circumference of the octagon must be 36.3cm. That (or rather, 36cm) was the intended answer.
According to the TEA press release, "item eight on the 10th grade math test could have been read in such a way that the question had more than one correct answer". That is putting a very kind spin on their blunder - there is in fact no reading of the question under which it has just one correct answer.
Bas doesn't miss a trick - he also quotes the portion of the press release which states that, "Each test item goes through a rigorous review process..", including review sessions by "professional educators who have subject-area and grade-level expertise." He's right to say that this reflects badly on the professional educators in question, but if this item was in fact field-tested, the psychometricians should have caught it as well.
One of the nifty things you learn in psychometrics training is that it's quite easy to check statistics that will tell you what the key (correct answer) to an item is, and quite easy to see if an item has been miskeyed or has two correct answers. One way to do this is to see how performance on the item correlates with test performance as a whole, and to see what percentage of the high scorers are choosing which item options. In the field test data, this item should have been flagged, because the smarter kids, instead of choosing just one answer, were probably choosing both correct answers in equal numbers.
It's possible that the smarter kids were the only ones using the Pythagorean theorem method (although the teacher who caught the mistake thought the smarter kids would be using the trigonometry method). But if the answer of "27" wasn't supposed to be an attractive distractor (an incorrect answer that an examinee would reach through incomplete reasoning), then that answer shouldn't have been chosen more than any other of the incorrect options. I wonder if any of the field test item stats looked peculiar, and just weren't caught.
The really scary number here? Almost 4,700 kids who failed will now pass because of this one item.
Posted by kswygert at August 7, 2003 08:21 PM