Realism in Experimentation
Source: Laboratories of Gary Lewandowski, Dave Strohmetz, and Natalie Ciarocco—Monmouth University
In an ideal world researchers would conduct their s…
1. Define key variables.
- Create an operational definition (i.e., a clear description of exactly what a researcher means by a concept) of online restaurant reviews.
- For the purposes of this experiment, online restaurant reviews are reviews provided on a website that offer diners’ insights into the restaurant.
- A positive review is one that gives a general rating of 4 stars (out of 5) or higher and also compliments the service.
- A negative review is one that gives a general rating of 2 stars (out of 5) or lower and also criticizes the service.
- For the purposes of this experiment, online restaurant reviews are reviews provided on a website that offer diners’ insights into the restaurant.
- Create an operational definition of tip amount.
- For purposes of this experiment, tip amount is defined as the amount of money the participant allocates to the server in paying the bill.
2. Conduct the study.
- Welcome participants at the lab door, which is set-up as the “Hawk Villa” restaurant.
- Dress and act like a restaurant server (e.g., wear white shirt and black apron, folded at waist).
- Sit participant down at a table.
- Provide participant with informed consent, a brief description of the research (influences on spending behavior), a sense of the procedure, an indication of potential risks/benefits, the right of withdrawal at any time, and a manner to get help if they experience discomfort.
- Give participant a wallet containing $136.10 (3-$20, 4-$10, 5-$5, 10-$1, and $1.10 in coins).
- Independent variable = restaurant review
- Say to participant: “Before you dine, to give you a bit more context, I thought you’d like to see the most recent online review of our restaurant.”
- Provide participant with the positive review (Figure 1).
Figure 1. Positive restaurant review. This review was shown to participants in the positive review group. - Play a video depicting a subpar dining scene.
- Instruct the participant to imagine themselves as one of the diners in the video and to imagine that the researcher is the server in the video.
- Dependent variable = the bill
- Return with the bill (Figure 2) placed in a restaurant billfold and say: “Here is your bill. I’ll take that when you’re ready.”
Figure 2. Restaurant receipt. This bill was given to all participants at the end of watching the dining scene. - After the participant places money in billfold, return and ask, “do you need any change?”
- Participant responds, “No thanks. Keep the change.”
- Return with the bill (Figure 2) placed in a restaurant billfold and say: “Here is your bill. I’ll take that when you’re ready.”
3. Debrief
- Report the nature of the study.
- “Thank you for participating. In this study I was trying to determine if reading an online restaurant review influences how much a person tips. There were two conditions, both of which watched the same video of subpar service. However, one group read a positive online review, while the other read a negative online review. We hypothesized that the group who read the positive online review would be more forgiving of the subpar service and give a higher tip.”
- Explain explicitly why the experiment was run this way.
- “We want to tell you why we ran this study this way. First, we couldn’t explicitly tell you that we were studying online reviews because it may have affected how you tipped. We also had to run this study in a laboratory setting because, as you can easily imagine, a real restaurant would not want to be part of a study involving subpar service where the key variable was their online reviews (particularly the negative ones).”
4. Conduct sections 2 and 3 with a new participant.
- Provide the negative review (Figure 3).
Figure 3. Negative restaurant review. This review was shown to participants in the negative review group. - Everything else should be the same.
5. Data Analysis
- Count the money the participant placed in the billfold.
- Use a calculator to subtract the bill total ($44.67) from the amount the participant left.
- $55.00 in the positive condition = $10.33 tip
- $45.00 in the negative condition = $0.33 tip
- Calculate tip percentage.
- Positive = 23%
- Negative = 0.7%
Conducting research in a realistic setting is optimal, but unfortunately, is not always ethical or even practical.
For example, researchers cannot simply march into a voting booth and observe what factors influence individuals? voting behaviors.
Instead, they can create realism in the laboratory by designing an authentic voting experience, which includes questioning and observing the exact variables that might influence the study?s outcome.
Using a realistic setting, this video will demonstrate how to design, conduct, analyze, and interpret an experiment that investigates whether restaurant reviews are related to a diner?s level of tipping.
In this experiment, a realistic restaurant setting is designed to allow the researcher to manipulate how restaurant reviews?positive and negative? influence participants? dining behavior.
For the positive review group, participants are asked to read a critique that compliments the service. In contrast, the negative review group is asked to read a critique that condemns the service.
After reading one of the reviews, participants are then shown a video that depicts a dining scenario with subpar service and must imagine themselves as one of the diners and the researcher as the server.
Once the video is over, participants are given a bill for the imagined meal. The dependent variable is the amount of money left as a tip.
Thus, participants who read the positive review are hypothesized to be more forgiving of the subpar service and offer a higher tip than diners who read the negative review.
To begin the study, meet the participant at the lab door and welcome them into the Hawk Villa restaurant. Guide all participants through the consent process and discuss the overall plan for the session.
After the participant consents to the experiment, give them a wallet containing $136.10, divided into specific bill and coin amounts.
Randomly divide participants to one of two experimental groups by handing them either a positive or negative review.
When the participants finish reading the reviews, have them watch a video depicting a dining scene. Instruct the participants to imagine themselves as the diner and the researcher as the server.
After showing the video, return to the table with the bill.
Once the participant places money in the billfold, return to the table and ask if they need any change.
To conclude the experiment, debrief the participant and explain why simulating a restaurant in the lab was necessary for the experiment.
To analyze the data, first count the money each participant placed in the billfold. Subtract the bill total of $44.67 from the amount the participant left to calculate the tip amount. Then, calculate the tip percentage.
To visualize the data, graph the mean tip percentages by group. Notice that participants in the positive review condition tipped higher than those in the negative review condition.
Now that you are familiar with how to optimize realism within a laboratory environment, let?s take a look at how you can apply this approach to other forms of research.
Driving simulators are often used in the laboratory to safely investigate driving ability in individuals with visual deficits or those under the influence of a substance, such as alcohol.
In addition, researchers can study navigational skills in individuals by examining task performance in a simulated real-world environment.
Finally, researchers have adapted dance movements to engage patients who express poor mobility and balance, such as those with Parkinson?s disease, and subsequently monitored changes in motor performance.
You?ve just watched JoVE?s introduction to using realism in laboratory experiments. Now you should have a good understanding of how to design and conduct this type of study, and how to calculate results and apply the phenomenon conducting research using realistic settings.
Thanks for watching!?
View the full transcript and gain access to JoVE Science Education videos
Q1: Why can't researchers always study behavior in real-world settings?
Real-world research is often impractical or unethical. For example, researchers cannot observe people in voting booths or private settings. Additionally, real-world environments lack experimental control, making it difficult to isolate specific variables influencing outcomes. Laboratory settings allow researchers to manipulate conditions while maintaining ethical standards.
Q2: What is mundane realism and how does it improve laboratory experiments?
Mundane realism means designing laboratory settings to feel like authentic real-life experiences. Researchers create realistic environments to increase the relevance and validity of their findings. In the restaurant tipping study, participants read reviews and watched a dining video to simulate genuine dining conditions, making their behavioral responses more representative of actual behavior.
Q3: How does the restaurant review experiment demonstrate a two-group design?
The experiment randomly assigns participants to either a positive or negative review condition. Both groups then watch the same dining video with subpar service. By comparing tip amounts between groups, researchers establish a cause-effect relationship psychology experiments require. The independent variable is the review type; the dependent variable is the tip amount.
Q4: What steps should you follow to conduct the restaurant tipping experiment?
First, welcome participants and obtain informed consent. Give them a wallet with $136.10 and randomly assign them to read either a positive or negative review. Have them watch a dining video while imagining themselves as the diner. After the video, present the bill and observe their tipping behavior. Finally, debrief participants and explain the study's purpose.
Q5: How do you calculate and analyze tip data from the experiment?
Count the money each participant left as a tip. Subtract the bill total of $44.67 from the amount left to determine the tip amount. Calculate the tip percentage by dividing the tip by the bill and multiplying by 100. Graph mean tip percentages by group to visualize differences between positive and negative review conditions.
Q6: What other laboratory applications use mundane realism to study behavior?
Driving simulators safely investigate driving ability in individuals with visual deficits or substance impairment. Researchers also use simulated real-world environments to examine navigational skills and task performance. Additionally, adapted dance movements engage patients with poor mobility, such as those with Parkinson's disease, while monitoring motor performance changes.
Q7: Why do psychological studies typically require larger sample sizes than other sciences?
Human behavior is inherently variable and unpredictable, requiring larger sample sizes to adequately represent the population and reduce measurement error. The restaurant tipping study demonstrated results using 200 total participants (100 per condition) rather than just two, ensuring findings reliably reflect actual tipping patterns across diverse individuals.