March 14, 2005
Statistics Term of the Day: Experimental Method
The experimental method is, basically, a method for manipulating variables in order to observe change in other variables. Let's say we have two methods of teaching a cat to roll over (yes, I realize that in real life, this will only happen when and if the cat feels like it, but work with me here). What's the simplest way that we could see if one method works better than the other?
Let's take that random sample of 1000 kitties that we obtained last week. First, we create our independent variable, or the variable that we are going to manipulate. In this case, or independent variable (or IV) is method of teaching. The IV needs to have at least two values, but it can be continuous (where an infinite number of possible values may fall in between observed values) or discrete (where values are separate and indivisible). In this case, we have two levels of a discrete variable (method #1 and method #2).
The IV is usually pretty easy to figure out, but the dependent variable - the variable in which you hope to see change after manipulating the IV - is trickier. The reason for this is that it all depends on how you define success. In our case, we're going to manipulate cat roll teaching method, but how do we show that one is "better" than the other? As measured by cat satisfaction with the roll? Owner satisfaction? Time to complete the roll? Total number of rolls completed in a minute? Flair and style in rolling over? Not scratching innocent bystanders while rolling? Deciding upon a specific DV takes some thought, because your research question should determine your DV, and what you measure with your DV will limit the research questions you can answer.
Let's say our DV is time to complete the roll. We'd want to measure it before we try any method, so that we can measure the change in time at the end; this is a repeated measure.
So what's the simplest experiment we could do? We could take that random sample of 1000 kitties, assign one random half to method #1, the other random half to method #2, and then try to hold all the other variables constant while we teach them. This means, perhaps, that we have the same woman teaching both sets of cats, and she teaches them at at the same time of day on alternate days, using the same room, the same treats for rewards, and the same tone of voice for commands. In real life we usually can't control all the confounding variables that we like, but it's best to limit confounds as much as possible. If a woman teaches one group of cats and a man teaches the other, the group that does better might be responding to the lower voice instead of to the teaching method.
We could also divide the cats randomly into three groups (with an extra kitty in one to balance out the total) and have a control group for comparison. This is a group to which you do nothing, but you still measure them before and after to see if they change on the DV. We all know that untrained cats aren't going to waste much time rolling over on command, so we don't expect to see much change here. Control groups are essential in the health sciences, though, when it's useful to compare treatment methods to subjects who either haven't been treated, or who have received placebos.
So good luck with the experiment. If you get any result other than your cat sleepily ignoring you, let me know.
