Section 5.2: Experimental Designs

Remember that scientists are in the business of trying to figure out why things happen.  Frequently, the question becomes one of cause. That is, the scientist observes something in the world and wonders what causes it.  For example, a criminologist may observe a particular crime and wonder what caused the perpetrator to commit it. For answering cause and effect questions, nothing is better than the experimental method.  Correlational methods help us identify relationships between variables but don’t provide information as to which variable caused the other.

Experiments are better at providing evidence of causal hypotheses because of temporal precedence.  That is just a fancy way of saying that B could not have caused A because A came before B in time.  It would be silly indeed to stipulate that a baseball flying out of the ballpark caused the bat to strike it.  Since the bat hitting the ball came first, it doesn’t make sense.

The simplest design of an experiment would be where a researcher draws a sample randomly from a population and then randomly assigns the people in the sample to two groups (be sure to keep the difference between random selection and random assignment clear in your mind!).  One group, called the experimental group, gets a treatment of some kind.  The other group, called the control group, gets no treatment.  At the end of the experiment, both groups are tested to see if there is a difference between the groups on whatever the researcher is studying.  Assuming that the two groups were the same at the start of the experiment (this assumption is why we make such a big deal about random selection and assignment), the researcher can conclude that any differences between the groups after the experiment were due to the treatment.

There are many different variations on the above theme.  The complexity of the variables being studied, the time and financial limitations faced by the researcher, and a host of other issues force alterations to this basic design.

Two famous researchers named Donald Campbell and Julian Stanley established a system of grouping these variations in experimental designs that the vast majority of the social science community has adopted.  They identified three categories of experimental designs: pre-experimental, quasi-experimental (a.k.a. causal comparative designs), and true experimental.

The primary difference between these different categories of designs is the degree of control they place on the variables being studied.  That is, the degree of control over extraneous variables. The pre-experimental designs have the least amount of control, the quasi-experimental designs are somewhere in the middle, and true experimental designs have the most control.  Remember that in the sciences, the idea of control means that experimental results are more trustworthy. Thus, when all other things are equal, true experimental designs are superior because they offer the most control.

The primary difference between these three general classifications of experimental designs is the degree to which randomization is a part of the design.  The most control (such as in a true experiment) involves both random selection and random assignment.  Random selection means that the sample of people was drawn at random from the population of people that the researcher is interested in.  Random assignment, on the other hand, means that the random sample of people was assigned to two or more groups randomly. Remember that bias can creep in when a sample is drawn from a population and when the sample is assigned to groups.

In practice, random selection is far less common than random assignment.  Often, it is too difficult to obtain a list of every member in a population.  Not so with assigning a sample to groups—the researcher knows who is in the sample and there is no good reason not to assign them randomly.  The exception to this is when the grouping variable is a static characteristic of the person, such as gender. Obviously, a researcher cannot assign people randomly to the groups male and female.  Another element of random assignment is randomly assigning the treatment to groups.  In true experiments, this step is generally considered necessary and prudent.  

          

Categories of Experimental Designs
	True Experimental	Quasi-experimental	Pre-experimental
Control Group?	Always	Often	Rarely
Random Selection?	Yes	No	No
Random Assignment?	Yes	No	No
Control Provided?	High	Some	None

Pre-experimental Designs

Pre-experimental designs do not involve random selection or random assignment.  Because of this, the power of the design to uncover the nature of relationships between variables is greatly reduced.  In short, pre-experimental designs are bad designs and should be used only when there is no other choice.  The most obvious problem is the complete lack of control over extraneous variables.  For example, let us say that I decided that drinking Guinness Stout can cure the common cold.  The next time I get a cold, I begin drinking copious amounts of beer. Behold! In only seven days the cold is gone—the beer worked!  The fatal flaw in my design is that I am not accounting for an important fact—a cold will usually run its course in about a week. Had I used a better design that utilized control groups, I might well have found different results. (I refuse to stipulate that Guinness Stout doesn’t cure the common cold without rigorous experimental validation).

Perhaps the worst of the worst is a design called the one-shot case study.  I include this type of design merely to point out that it is bad and that you should never use it.  In addition, other people do use it; be very wary of their results!

The basic design of the one-shot case study can be broken down into three easy steps.  First, you get a group of people. Most commonly, this is a captive group, such as a class of students or all freshmen in a university.  Second, you administer some sort of treatment. Third, you measure something about the participants. This will often take the form of a test.  Since this test comes after the treatment, it is usually referred to as a posttest.  That’s it.  You’re done.  You have absolutely no control over extraneous variables.  Anything at all could have caused the results observed by your measurements and you don’t know it.

Regardless of my hardline stance on not using such designs, you may be forced into using such a design.  This can happen when you are brought in to evaluate something after the treatment has been administered.  In such a case, your only option is to make the best of a bad situation.

Another pre-experimental design is called the one-group pretest-posttest design.  This design differs from the one-shot case study approach in that a pretest is given to the participants before the treatment is administered.  It can be summarized as follows:

1. 1. 1.  Participants are assigned to one group.
      2.  A pretest is administered.
      3.  A treatment is administered.
      4.  A posttest is administered.

While still not an ideal method, this design is far superior to the one-shot case study because it provides a baseline for comparison.  That is, the pretest scores can be compared to the posttest scores to see if there is a difference. If there is a difference, we can assume that something happened between the pretest and the posttest and that something could be our treatment.  The primary weakness of this design is the uncertainty about whether the observed difference was, in fact, our treatment.  Because of the lack of control, this is still a weak design.

Modification History

File Created:  07/25/2018

Last Modified:  11/01/2023

---

[ Back | Content | Next]

---

This work is licensed under an Open Educational Resource-Quality Master Source (OER-QMS) License.