The Multi-group Experiment
Source: Laboratories of Gary Lewandowski, Dave Strohmetz, and Natalie Ciarocco—Monmouth University
A multi-group design is an experimental design that…
1. Introduction of topic/research question
- Research question: People are unique and different, yet mostly the same. When it comes to interacting with other ethnicities, which perspective is better? We could focus on and appreciate our differences (a multicultural perspective), or focus on the many similarities we share (a color blind perspective). If one is more effective, does it have to be explicit (i.e., does the person need to realize it)? The researcher then forms a hypothesis based on educated guesses about potential answers.
- Research Hypothesis: Those who are exposed to the multicultural perspective will display more favorable attitudes toward an outgroup member than those who are exposed to the colorblind perspective.
2. Key variables
- Variable: Anything that changes in a study
- Independent variable: The cause or what the researcher manipulates/changes in order to detect changes in the participant
- Based on the hypothesis, interethnic ideology is the independent variable.
- Dependent variable: The effect or the outcome that the researcher measures in the participant
- Based on the hypothesis, attitudes toward an outgroup member is the dependent variable.
3. Defining the variables
- Interethnic Ideology: To manipulate the independent variable of interethnic ideology, have participants perform a word search (so that participants will not explicitly know they are being exposed to the different ideologies).
- Attitudes toward an outgroup member: To measure the dependent variable of perceived attractiveness, have the participant choose a seat (near or far from the outgroup member).
4. Establishing conditions
- Levels: The independent variable’s number of conditions or groups
- In simple experiment, there are two levels (experimental group and control group).
- In a multi-group experiment, there are more than two levels.
- Potential conditions/groups
- Two or more types of treatment (such as this study)
- Placebo
- Empty-control groups (included in this study)
- Experimental Conditions: Groups who receive different types of the key ingredient
- Multicultural Group: Word search with multicultural terms, as well as 5 distractor words to minimize hypothesis guessing
- Hypothesis guessing: When a participant actively tries to figure out what the study is about, which can lead to unnatural responses
- Color Blind Group: Word search with color blind terms, as well as 5 distractor words (same ones as the multicultural group) to minimize hypothesis guessing
- Multicultural Group: Word search with multicultural terms, as well as 5 distractor words to minimize hypothesis guessing
- Placebo Condition: A condition that doesn’t receive any treatment, but participant’s believe they are
- Though we do not include one in this study because participants likely won’t realize that they are in an experimental group, a placebo condition is often useful if we want to see how participants act if they believe they are receiving a treatment, but actually aren’t. Placebo groups generally help us get a handle on how participants’ expectations influence outcomes in our dependent variable.
- Control Condition: This group does the same thing as experimental groups without the key ingredient.
- They will do a word search with common everyday words that aren’t associated with multicultural or colorblind perspectives.
- Empty Control Condition: A group that does not receive any type of treatment or “key ingredient.” This provides a baseline that shows how participants act without any treatment.
- They will not do any word search.
5. Confounds
- What it is: Anything the researcher accidentally changes along with manipulation
- Its importance: The presence of a confound makes it impossible to know if the treatment or “key ingredient” is responsible for the changes in the dependent variable.
- Application to study: In the present study, the researcher must be careful that the word searches are all as similar as possible. If one had 10 words to find, while another had 5, that would be a confound. If one was on white paper, while another was on yellow, that would be a confound.
6. Measuring the dependent variable (attitudes toward an outgroup member)
- Seat choice
- Key measurement considerations: Need to set up 8 chairs in a room with the outgroup member’s belongings on the far left chair. This gives the participant 7 seats to choose from.
- Higher score indicates more positive attitude toward outgroup member (Table 1).
| Chair with Belongings | Empty Chair | Empty Chair | Empty Chair | Empty Chair | Empty Chair | Empty Chair | Empty Chair |
| 7 points | 6 points | 5 points | 4 points | 3 points | 2 points | 1 point |
Table 1. Point distribution based on seat choice. More points are given to the chair chosen near the outgroup member’s belongings.
7. Conducting the study
- Setting = Research lab and adjoining room with 8 chairs
- Informed consent
- In a research lab, meet the participant for study on “Word Searches.”
- Go through informed consent: “Here is the informed consent, which outlines what the study is basically about, any risks/benefits of participation, and lets you know that you are free to quit at any time.”
- Random assignment to condition
- Randomly order the packets so that the participant’s condition (multicultural, colorblind, control, or empty control) is not based on anything other than chance. Otherwise, the researcher may subconsciously be more likely to assign certain participants (e.g., those who look physically fit) to certain conditions (e.g., running).
- Running the study
- Give participant randomly assigned word search (need to highlight the words they are searching for to show the ones associated with multicultural and colorblind, as well as the distractor words—in italics. The word search for the control group contains all mundane words.
- Multicultural: culture, variety, difference, diversity, multi, flowers, stars, artistic, world, music
- Color-blind: equality, unity, sameness, similarity, blind, flowers, stars, artistic, world, music
- Control: practical, relaxed, logic, creativity, friendship, flowers, stars, artistic, world, music
- Empty Control: nothing; Say you ran out of word searches for the first part.
- Measure of attitudes toward an outgroup member
- Escort participant to adjoining room for Part 2.
- “For the next part of this study, you are going to work together with the following partner (hands participant a picture) on additional word search tasks. They’ve already arrived, but had to run out to their car to get something. Please have a seat.”
- Walk to a different part of the room and note the participant’s seat selection.
- Give participant randomly assigned word search (need to highlight the words they are searching for to show the ones associated with multicultural and colorblind, as well as the distractor words—in italics. The word search for the control group contains all mundane words.
- Debriefing
- Explain the purpose of the study to the participant: “Thank you for participating. In this study I was trying to determine if different interethnic ideologies influenced attitudes toward someone different than you. Specifically, I focused on a multicultural perspective, which is the idea that we should celebrate differences and the colorblind perspective, which is the idea that we should ignore differences and focus on shared similarities. I manipulated those based on the words participants searched for in Part 1. There were also control groups, one who searched for neutral words, and one who didn’t search for any. Do you have any questions?”
- Address deception.
- Explain: “It is important that we get a natural performance, not one that the participant feels is expected. If participants were to know the true reasoning and hypothesis behind the study, they may perform in an unnatural way by trying to live up to the experimenters’ perceived expectations. To eliminate this problem, it is necessary for me to mislead you about the true nature of the word search and the fact that there was an outgroup member who you were paired with in Part 2. In actuality, the word searches were part of my manipulation and there wasn’t anyone in Part 2. Rather, we used the same picture of another participant for everyone. Because of the nature of how we did the study, it is quite natural for participants to have believed there was another person, but rest assured there wasn’t.”
Choosing the correct experimental design is essential to answer the specific scientific question at hand.
In an experiment, researchers are concerned with variables?what changes. For instance, the researcher manipulates the independent variable to detect possible differences amongst participants.
The independent variable can have different?levels, also known as conditions, which may result in different outcomes?what the researcher measures as the dependent variable.
If an independent variable has three or more conditions, the experiment consists of a multiple-group design. This is in contrast to a simple experimental design, which contains two levels, the experimental and control groups.
Each design is used to answer different questions; a two-group design tells you whether the independent variable has any effect, whereas a multiple-group design tells you how much of an effect each condition has.
Using a multi-group approach, this video demonstrates how to design levels of variables and conduct the study, as well as how to analyze data and interpret participants? attitudes towards complex ethnic interactions.
In this multi-group experiment, participants are randomly assigned to one of four conditions: Colorblind, Multicultural, Control, or an Empty-Control group.
Two of the groups, Colorblind and Multicultural, are considered main experimental conditions, in which participants are unknowingly exposed to diverse perspectives through a word search task that focuses on shared similarities or appreciative differences.
In contrast, participants in the Control group are exposed to a mundane word search task that lacks any perspective, whereas participants in the Empty-Control group are not exposed to the task. This latter condition provides a baseline response of how participants act without any treatment.
For the word search task, participants are given words that relate to the attitudes of their assigned conditions. For example, colorblind words include: equality, unity, sameness, similarity, and blind; multicultural words include: culture, variety, difference, diversity, and multi; and control words include: practical, relaxed, logic, creativity, and friendship.
In addition, all of the word searches contain the same five distractor words: flowers, stars, artistic, world, and music. The distractors are included to prevent participants from guessing what the true nature of the study is?a concept referred to as hypothesis guessing.
Following the word search task, all participants are handed a photograph of someone they think will help them complete additional tasks when they return. In reality, the photograph represents a member of a social group with whom the participant does not identify?the outgroup member.
The dependent variable?the participants? attitudes toward the outgroup member?is then quantified by how close they choose to sit near their supposed partner?s belongings. Note that the outgroup member never makes a physical appearance.
In this case, choosing to sit closer represents a more positive attitude towards the outgroup member versus sitting farther away.
It is hypothesized then that those who are exposed to the multicultural perspective will display more favorable attitudes toward an outgroup member when compared to those who are exposed to the colorblind perspective.
To conduct the study, create three different word searches that consist of five words associated with colorblind, multicultural, or mundane attitudes. Make sure that each search includes the same five distractor words.
Randomly organize the word searches inside packets to ensure that participants? assignments are entirely based on chance and not any preconceived assumptions about the participant. Note that the packet for the Empty-Control group is indeed empty.
In an adjoining room, set up eight chairs. Place the outgroup member?s belongings on the far left chair, giving the participant seven seats to choose from.
To begin the experiment, meet the participant at the lab. Provide each participant with informed consent, which consists of a brief description of the research and procedures, and the potential risks and benefits of participating.
To assign the conditions, hand all participants a word search packet, except for those assigned to the Empty-Control group.
Once the word search is finished, escort the participant to an adjoining room. Hand each participant the same photograph of another person?the outgroup member?and tell them that they will now work together with this person on additional word search tasks upon their return.
Further explain to the participant that their partner has already arrived, but had to run out to their car to get something. Tell them to have a seat.
After participants choose their seat, walk to a different part of the room and note each one?s seat selection.
At the conclusion of the experiment, debrief participants by telling them the nature of the study.
Explain that deception was necessary to capture the participants? natural reactions, as divulging the true intentions of the study beforehand could have influenced their behavior to meet perceived expectations.
To analyze how different perspectives influence participants? attitudes, average the seat choice scores in each group and plot the means across the four conditions. To determine if group differences were found, perform an analysis of variance comparing the four groups.
Note that those in the multicultural group sat closer to the outgroup member, which translates to a higher attitude score. Thus, the exposure to the multicultural ideology led to the most favorable attitude as compared to all other conditions.
Now that you are familiar with multiple group designs, let?s take a look at how other researchers manipulate conditions to maximize experimental control.
For example, to determine the relationship between time spent sleeping and exercise performance, multiple groups are needed. That way, an optimal amount of sleep can be determined, where ultimately the longest amount of sleep may not help, but rather hurt, exercise performance.
Similarly, a multiple group design would be necessary to determine the optimal dose of a medication that achieves the desired outcome without side effects.
A multi-group design is also beneficial to test whether participation in a dance class relieves depression symptoms. In this case, adding a group exposed to social interaction without dancing controls for a potentially confounding variable?the inadvertent social interaction derived from the class.
You?ve just watched JoVE?s introduction to multi-group experimental design. Now you should understand how to design a multi-group experiment by including different levels of the independent variable.
Using a specific example, the video demonstrated how to conduct a multi-group experiment, as well as how to evaluate the results. Finally, through a discussion of applications, you should have a good understanding of how multi-group experimental design can be used to meet specific research needs.
Thanks for watching!?
View the full transcript and gain access to JoVE Science Education videos
Q1: What is the difference between a multi-group design and a two-group design?
A two-group design contains two levels: experimental and control groups, telling you whether an independent variable has any effect. A multi-group design has three or more conditions, revealing how much effect each condition produces. Multi-group designs answer more detailed questions about the magnitude of effects across different levels of treatment.
Q2: What are the four conditions used in the ethnic attitudes multi-group experiment?
The experiment includes Colorblind, Multicultural, Control, and Empty-Control groups. Colorblind and Multicultural are main experimental conditions exposing participants to different perspectives through word searches. The Control group receives a mundane word search, while the Empty-Control group receives no task, providing a baseline response without any treatment.
Q3: How does hypothesis guessing affect multi-group experiment design?
Hypothesis guessing occurs when participants guess the study's true purpose, potentially biasing their responses. To prevent this, researchers include distractor words in word searches that are unrelated to the study's main focus. These distractors mask the experiment's actual intent, ensuring participants respond naturally without altering their behavior based on perceived expectations.
Q4: Why is an empty-control group important in multi-group experimental designs?
An empty-control group provides a baseline measurement of participant behavior without any experimental treatment. This allows researchers to distinguish between effects caused by the independent variable manipulation and effects from simply participating in any task. Comparing all conditions against this baseline strengthens the validity of experimental conclusions.
Q5: How is the dependent variable measured in the multi-group ethnic attitudes study?
The dependent variable—participants' attitudes toward an outgroup member—is measured behaviorally through seating distance. Participants choose where to sit near their supposed partner's belongings. Closer seating indicates more positive attitudes, while greater distance suggests less favorable attitudes toward the outgroup member.
Q6: What statistical test analyzes differences between groups in a multi-group design?
Analysis of variance (ANOVA) compares means across multiple groups to determine if significant differences exist between conditions. Researchers average seat choice scores for each group and perform ANOVA to test whether the four experimental conditions produced statistically different attitudes toward outgroup members.
Q7: What role does informed consent play in conducting multi-group psychology experiments?
Informed consent requires researchers to provide participants with a brief description of research procedures, potential risks, and benefits before participation. This ethical requirement ensures participants understand what they're agreeing to, though researchers may use deception during the study if necessary to capture natural reactions, with full debriefing afterward.