Overview
Source: Laboratories of Gary Lewandowski, Dave Strohmetz, and Natalie Ciarocco—Monmouth University
One of the challenges in measuring an experimental variable is identifying the technique that will produce the more accurate measurement. The most common way to measure a dependent variable is self-report—asking the participant to describe his/her feelings, thoughts, or behaviors. Yet, people may not be honest. To truly know something about a participant, it may be necessary to see what they actually do in a situation.
This video uses a multi-group experiment to see if feeling close to others results in more favorable attitudes toward environmental consciousness measured both by self-report and behavioral observation.
Psychological studies often use higher sample sizes than studies in other sciences. A large number of participants helps to better ensure that the population under study is better represented, i.e., the margin of error accompanied by studying human behavior is sufficiently accounted for. In this video, we demonstrate this experiment using just one participant. However, as represented in the results, we used a total of 186 participants to reach the experiment’s conclusions.
Procedure
1. Define key variables.
- Create an operational definition (i.e., a clear description of exactly what a researcher means by a concept) of feeling close to people.
- For the purposes of this experiment, feeling close to people involves a perceived connection including feelings of emotional attachment toward another person or people.
- Create an operational definition (i.e., a clear description of exactly what a researcher means by a concept) of environmental consciousness.
- For purposes of this experiment, environmental consciousness is defined as a person’s likelihood to value practices that protect the natural world.
2. Conduct the study.
- Meet the participant in a laboratory/classroom.
- Provide participant with informed consent, a brief description of the research (getting students’ opinions on various social issues), a sense of the procedure, an indication of potential risks/benefits, the right of withdrawal at any time, and a manner to get help if they experience discomfort.
- Participant signs informed consent form.
- Direct participant by saying: “For the next part of the study, I’d like you to sit and look at several pictures. Indicate on each of the index cards how close or connected you feel to the image, where 1 = not at all and 10 = extremely.”
- Manipulation (3 conditions): participant is directed to view slides of one of the three following sets of pictures.
- Feeling-Close-to-People condition: 5 pictures of people together
- Feeling-Close-to-Animals condition: 5 pictures of animals together
- Control/No connection condition: 5 pictures of everyday, non-living items
- Give participant 2 surveys (Appendices 1 and 2) and say: “Please take a minute to consider each question and indicate how you feel about each.”
Appendix 1. Survey 1 given to participants. This survey was used to measure attitudes towards recycling.
Appendix 2. Survey 2 given to participants. This survey served as a distractor.- Note that one is the real survey and one is a filler/distractor.
- While the participant completes the survey, pretend to each lunch out of the participant’s sight.
- The meal should have a range of recyclable and non-recyclable items (Recyclable= glass/plastic bottle, paper plate, paper napkin, plastic spoon, and plastic pudding/yogurt container; Non-recyclable= pretzels, grapes, and apple). Also on the table where the meal is eaten is a crumbled up piece of paper and a broken pen.
- When the participant finishes the surveys, ask the participant to help clear off the table where lunch was eaten so he or she can do the next part of the study.
- The room should have two clearly labeled empty bins, one for trash and one for recycling.
- While the participant clears off the table, the researcher should appear busy looking for/preparing the next part of the study.
3. Debrief.
- “Thank you for participating. In this study I was trying to determine if feeling a close connection to other people would influence your environmental consciousness, measured by your recycling attitudes (measured via the survey) and behaviors (measured via how many items you recycled when clearing off the table). There were three groups in the study, one group saw pictures of people together, another group saw pictures of animals, and the other group saw pictures of common items. We hypothesized that those in the feeling-close-to-other-people condition would have more favorable attitudes toward recycling and would recycle more.”
- “We couldn’t tell you about the study’s true purpose or our hypotheses ahead of time because we wanted to get your honest and natural thoughts and reactions. Because of the nature of the deception, it is quite natural for participants to not realize that they were being deceived.”
4. Go through the “Conducting the Study” procedure three times with three different participants: once for the people condition, once for the animal condition, and once for the household items condition.
5. Data Analysis
- Look at the participant’s survey and circle items 3, 5, and 7.
- Explain that these three items comprise the participant’s mean recycling attitude score, while the other items are simply distractors.
- Look in the bins to count the number of properly recycled items. This is the participant’s behavior score.
- Calculate the means by hand.
Psychologists study many aspects of behavior by asking participants to report their own feelings or by directly observing their performance on specific tasks.
While self-report is often used, the authenticity of reported responses is difficult to determine. Therefore, observing participants’ behaviors during specific situations provides more reliable data.
Here, psychologists compare two forms of measurement, self-report and behavioral observation, to see if participants’ feelings reported through a survey correlate with their actual behavior.
This video demonstrates how to design, conduct, analyze, and interpret an experiment that investigates whether feeling close to others results in more favorable attitudes toward recycling.
During this experiment, participants are randomly divided into one of three groups: Feeling Close to People, Feeling Close to Animals, or No Connection. In this case, feeling close to people or animals involves perceived connections such as emotional attachments toward other people or animals.
Participants are then asked to view slides in which five pictures of people, animals, or non-living household items are shown, according to their group assignment. They are all then given the same two surveys to complete.
The first survey involves self-reported measures regarding environmental practices for protecting the natural world, whereas the second survey serves as a distraction while the researcher creates a scene for participant observations.
In this scene, the researcher pretends to eat lunch out of the participant’s sight. The meal includes a variety of recyclable and non-recyclable items.
After participants finish the second survey, they are asked to help clear off the table where lunch was eaten while the researcher steps away.
The dependent variables, the values reported on the first self-report survey and the number of items that each participant recycled when clearing the lunch table, are compared.
Participants in the Feeling Close to People group are hypothesized to have more favorable attitudes towards recycling and would recycle more items compared to the other groups.
To begin the study, meet the participant in the laboratory. Provide them with informed consent and discuss the overall procedure of the experiment.
Randomly assign each participant to one of three experimental groups: Feeling Close to People, Feeling Close to Animals, or No Connection, to view five pictures of people, animals, or non-living household items, respectively.
To assure that participants pay attention to the images, have each one write down how close or connected they feel to each image on a scale of 1 to 10.
After the participant views the images, hand them two surveys and instruct them to take a minute to consider each question and indicate how they feel.
While the participant fills out the second survey, pretend to eat lunch out of their sight.
After the participant completes the last survey, ask them to help clear off the lunch table while you prepare for the next part of the study. Have them place garbage into the garbage bin and recyclable items in the recycling bin.
When the participant finishes, record how many items were correctly placed into the recycling bin.
To conclude the experiment, debrief the participant and explain why deception was necessary for the experiment.
To analyze the data, first calculate the participants’ recycling attitude score for each condition by averaging the ratings from items 3, 5, and 7 on Survey 1. Note that the other items on this survey, as well as the second survey with yes or no questions, serve as distractors. Graph the mean recycling attitude by condition.
Notice that those in the Feeling Close to People condition indicated that they cared more about recycling than controls.
To determine the observed behavioral score, average the number of items that were counted as being correctly recycled for each condition. Graph the means across conditions. Notice that no significant differences were observed between groups.
Notice that participants who reported favoring attitudes towards recycling did not actually engage in the behavior. In other words, participants’ feelings were not correlated with their observed behaviors.
Now that you are familiar with how to compare self-report surveys against actual observed behavior, you can apply this approach to other forms of research.
Several studies have emphasized the importance of measuring behavior over the use of self-reports to better understand emotion. The most thorough studies collect data from a set of questionnaires, as well as correlate the answers with results from a goal-oriented behavioral task.
In addition, individuals with an addiction, such as cigarette smoking, do not always provide reliable answers to self-report questions. Researchers must use alternative methods, such as fMRI or PET imaging, to better understand smoking behavior.
Finally, self-report questionnaires about sleep are only as informative as the individual’s memory or awareness of their sleep behavior. Therefore, using polysomnography to observe participants during sleep is especially useful when trying to understand abnormal occurrences.
You’ve just watched JoVE’s introduction to compare and contrast measurements of self-reporting and observed behavior. Now you should have a good understanding of how to design and conduct this type of study by administering surveys and directly monitoring behavior, as well as how to analyze and interpret the results.
Thanks for watching!
Subscription Required. Please recommend JoVE to your librarian.
Results
This procedure was repeated 185 times so that the results reflect data from 186 total participants. This large number of participants helps to ensure that the results reflect accurate mean numbers of items recycled. If this research were conducted using just one or two participants, it is likely that the results would have been much different, and not reflective of the greater population. Numbers in Figure 1 reflect participants’ average responses for the recycling questions on the survey. Numbers in Figure 2 represent the number of items (out of 10 total) the participant properly put in the recycling bin.
To determine if there were differences between the three picture conditions, an analysis of variance (ANOVA) was performed. The results indicated that though there were significant differences for the self-report survey, such that those in the feeling-close-to-people condition said they cared about recycling more, there were no significant differences when it came to participants’ actual behavior.
Figure 1. Average recycling attitude score by condition. Means were calculated from survey questions 3, 5, and 7.
Figure 2. Average number of properly recycled items by condition. Means were calculated from researcher’s observations of items placed in the recycling bin.
Subscription Required. Please recommend JoVE to your librarian.
Applications and Summary
This multi-group experiment shows how participants’ feelings measured via a survey may not match up exactly with how they behave. In this case, participants reported more favorable attitudes (especially in the feeling-close-to-people condition), but when it came time to actually recycle, participants did not engage in the behavior.
This study replicates and extends previous research on self-reported recycling attitudes vs. actual recycling by Huffman et al., which showed that actual recycling was not associated with recycling attitudes and weakly correlated with self-reported recycling (i.e., did they recycle in the past).1
Several papers have emphasized the importance of behavioral measures, including Baumeister et al. and Lewandowski and Strohmetz.2,3 In each case, the key take away was that what a person actually does is more revealing than what the same person claims they will do. Though more challenging, measuring behavior is more informative.
Subscription Required. Please recommend JoVE to your librarian.
References
- Huffman, A., Van Der Werff, B. R., Henning, J. B., & Watrous-Rodriguez, K. When do recycling attitudes predict recycling? An investigation of self-reported versus observed behavior. Journal of Environmental Psychology. 382, 62-270. doi:10.1016/j.jenvp.2014.03.006 (2014).
- Baumeister, R. F., Vohs, K. D., & Funder, D. C. Psychology as the science of self-reports and finger movements: Whatever happened to actual behavior?. Perspectives on Psychological Science. 2 (4), 396-403. doi:10.1111/j.1745-6916.2007.00051.x (2007).
- Lewandowski, G. W., Jr., & Strohmetz, D. Actions can speak as loud as words: Measuring behavior in psychological science. Social and Personality Psychology Compass. 3 (6), 992-1002. doi:10.1111/j.1751-9004.2009.00229.x (2009).
