A word to the wise, and the not-so-wise
I'm beginning to notice a trend in the email I've been receiving. Well, several trends, really, some welcome, and some not. I figure now's the time to set a few ground rules:
Emails that are not welcome and will not receive more than a perfunctory, polite answer, if that much:
1. Emails in which the writer bitches non-stop about the existence of psychometrics, and standardized tests, and any sort of objective assessments. Some folks have well-thought-out and logical approaches to this viewpoint, and when I find their writings on the web, I try to link to them (and debunk them, of course). But the email I get along these lines tends to be from "trolls" - people who crank out a highly-emotionally-charged email that they know I'm going to disagree with every word of, and they're just trying to get a rise out of me. Not going to happen.
2. Emails in which the writer expresses his/her intent to sue a testing organization. I can understand that test takers, especially ones who get low and/or failing scores, want to to look for misbehavior (intentional or otherwise) on the part of the testing organization as a reason for their score. I can even understand that they might want to get the legal system involved to right the great wrong that has been done to them.
However, writing every psychometrician on the web isn't a productive means toward that end. The reason is not because we're all involved in some great conspiracy against the test takers, but because every test, and every testing organization, is founded on vastly differing rules when it comes to test scoring, quality analysis, test content, and protocol for dealing with challenges to the test. If you write me and ask me to help you sue, say, ETS because you think your score on the PRAXIS exam is flawed, I'm not going to be able to help you, because I've never worked on the PRAXIS. I don't know their scoring scheme, their test content specifications, or their rules for dealing with test center irregularities. I don't know what policy they follow if they accidentally administer a flawed item. Anyone who is an expert on PRAXIS either works on the exam now, or used to work on it, and they're not going to help you out, because all that material is either copyrighted, or confidential, or both.
This is not to say that no one can challenge a testing organization, because it happens. It just means that you're probably not going to get anyone who is an expert in the particular test you are challenging to help you, and you're certainly not going to be able to enlist random psychometricians in your quest. Proceed with caution.
Emails that are welcome, but that I may never get around to answering:
3. Emails in which the author names a particular new, relatively-unknown, or state-specific standardized exam, and then proceeds to ask me every possible question about that exam. I appreciate your curiosity, I really do, but if you toss an exam at me that I'm not familiar with, it will take me a lot of time to help you find, on the web or in print, public information such as test taker performance on the exam, or the number of years the exam has been operational. Some information I can't help you find, due to confidentiality reasons. I am an "expert" on exactly one test, which no one has ever asked me about, and am becoming an expert on four others, but even for those all I can do is help you find what's been published on these exams. Don't ask me to tell you the exact item content specifications, or the decisions that led to a certain scoring scheme, because I either won't know or can't tell you. I might be willing to forward your email to someone more informed than I, but there's no guarantee that they can or will reply.
Emails that are always welcome and that I will do my best to respond to:
4. Emails that are asking for clarification of general psychometric terms. Anytime you want someone to give you definitions and clarifications related to item response theory, test validity, test reliability, test equating, bias, differential item functioning, ipsative scoring, compensatory scoring, percentiles, norm-referenced or criterion-referenced scoring, computer-adaptive testing, or performance assessments, I'm your girl. I'll do my best to explain things to you.
5. Emails from concerned parents who just want someone to listen while they tear their hair out over the new testing requirements. My shoulder is always here for you to cry on, and I'll agree with you when you say it's idiotic that you weren't given a score report for your daughter's 6th-grade reading exam until after she graduated from Harvard. I may not be able to rectify any wrongs, but I can certainly sympathize with you about them.
6. Emails in which the author tips me off to a testing scandal in their state that didn't make the national news. Oooh, scoops and scandals. Always a good thing - I appreciate you tipsters out there.
7. Emails that tell me I rock, or that my blog is great. Enough said.