Thinking Too Much Impairs Decision-Making
Source: Peter Mende-Siedlecki & Jay Van Bavel—New York University
When we are considering a tough choice between two or more attractive options, we of…
1. Participant Recruitment
- Conduct a power analysis and recruit a sufficient number of participants and obtain informed consent from the participants.
- Gather two experimenters: One to administer the jam taste test, and another (blind to condition) to obtain participants’ ratings of each jam.
- Recruit volunteers for a study entitled "Jam Taste Test". Instruct volunteer subjects not to eat anything for 3 hrs prior to the study.
2. Data Collection
- Purchase five brands of strawberry jams or preserves that vary in their overall quality.
- To provide participants with a large range of jam quality, the original researchers purchased the 1st-, 11th-, 24th-, 32nd-, and 44th-ranking jams in a report produced by Consumer Reports magazine of 45 different jam brands (“Strawberry Jams," 1985). This report contained expert rankings based on the ratings of seven consultants who were trained sensory panelists. Each expert consultant rated 45 jams across 16 specific sensory characteristics, such as sweetness, aroma, etc. Averages were calculated across all seven expert consultants’ ratings to yield rankings of all 45 jams.
- Prior to running each participant, set up the testing room. You’ll need a table, a chair, plates, spoons, jams, labels, and a trash can.
- Arrange the five jams in a random on the table, labeled A through E.
- Keep track of which jam receives which label for each participant, and in which order the jams are arranged.
- Take one teaspoon of Jam A and place that spoonful on a plate in front of the Jam A jar.
- Repeat for Jams B through E.
- Run participants individually.
- Have Experimenter 1 inform each participant that the purpose of the study is to evaluate different kinds of jams under different conditions, as part of a consumer psychology experiment.
- Explain that some participants will taste jams on crackers, while others will taste jams on plastic spoons.
- Critically, inform the participant that they have been randomly assigned to the condition in which they’ll be tasting jams on spoons. Specifically, after tasting the jams, they will rate their liking for each one.
- Have each participant sign a consent form, giving their informed consent to participate.
- Randomly assign participants to either the “reasons-analysis” condition, or a control condition.
- Provide “reasons-analysis” participants with written instructions asking them to "analyze why you feel the way you do about each" jam," in order to prepare yourself for your evaluations."
- Inform these participants that they will be asked to list their reasons for liking or disliking each jam after tasting. Describe this task as a means of organizing participants’ thoughts prior to rating, and further, inform participants that they will not be asked to hand in their list of reasons.
- Do not give the rest of participants (e.g., control participants) any additional instructions.
- Ask participants to be seated at the table where the five jams have been arranged spoons with approximately 1 tsp of strawberry jam.
- For “reasons-analysis” participants, have Experimenter 1 ask the participants to taste each of the five jams.
- During the tasting, leave the room.
- Upon returning, administer the reasons questionnaire. This questionnaire should ask participants to list their reasons for liking or disliking each of the Jams A through E.
- Reiterate that the purpose of the questionnaire is to organize the subjects' thoughts and that they would not be asked to hand it in.
- Leave the room once again.
- Upon returning, explain that the completed questionnaire will no longer be needed, and should put the questionnaire in the trash can.
- For control participants, have Experimenter 1 also ask the participants to taste each of the five jams.
- During tasting, leave the room.
- Upon returning, administer a filler questionnaire instructing participants to list reasons why they chose their major.
- Leave the room once again.
- Upon returning, collect the completed filler questionnaire.
- Have Experimenter 1 now introduce the participant to Experimenter 2, who must be unaware of the condition that the participant was assigned to.
- Have Experimenter 2 administer a questionnaire to the participant asking them to evaluate each of the jams on a 9-point scale ranging from disliked (1) to liked (9).
- Instruct participants to complete the questionnaire and to place it through a slot in a covered box on the table, to maintain anonymity.
- Leave the room while participants are completing this questionnaire.
- Upon returning, fully debrief the participant regarding the purpose and procedures of the study.
3. Data Analysis
- After running all participants, test whether evaluations of the jams varied between participants in the “reasons-analysis” condition (e.g., those who analyzed how they felt about the jams) and those in the control condition (e.g., those who did not) via a 2 (condition: reasons analysis vs. control) x 5 (jams 1, 2, 3, 4, and 5) multivariate ANOVA.
- To compare the degree of agreement with expert ratings between participants in the “reasons-analysis” condition and those in the control condition, simply compute the Spearman’s rank-order correlation between each participant’s subjective ratings of each jam and the expert ratings of each jam, on a subject-by-subject basis.
- Convert these values to z-scores via Fisher’s r-to-z transformation for purposes of analyses.
- Perform an independent-samples t-test examining how the average participant/expert correlation varies as a function of group.
- Use two, one-sample t-tests comparing the average participant/expert correlation in the “reasons-analysis” condition and the control condition against zero separately.
Decision-making occurs on a continuum of thoughts with varying levels of consciousness. In most cases, people have no idea why they feel the way they do.
At one extreme, individuals may think very little?making decisions quickly and habitually?like pulling out already prepared foods for dinner.
In contrast, choosing where to dine out results in thinking too much?reading through reviews to weigh all of the pros and cons because their friend also asks for an explanation regarding the final selection.
Upon introspection?without knowing why?the individual honed in on irrelevant yet salient details that made sense, but in fact, did not lead to the best decision, at least by experts? standards.
This example capitalizes on the unreliability of thinking consciously?also called the introspection illusion?where an attempt to understand one?s own perceptions and behaviors results in erroneous post-hoc explanations.
Based on the classic experiment by Wilson and Schooler, this video demonstrates whether asking participants to think about reasons for rating different jams interfered with their preferences compared to those who were simply asked to provide rankings.
In this experiment, two groups of participants?reasons-analysis and controls?are asked to taste test different jams that have been previously analyzed by experts, and then evaluate each one based on their preference.
During the tasting phase, those in the reasons-analysis group are asked to analyze the way they feel about each one by listing five reasons for liking or disliking it after tasting. This step allows participants to organize their thoughts and encourage better preparation for providing final ratings.
Each participant in the control group, on the other hand, is asked to complete a filler questionnaire and list reasons why they chose their college major. In either case, these first forms are discarded.
Subsequently, participants are given a second preference questionnaire by a different experimenter blinded to their condition. They are asked to anonymously evaluate each jam on a 9-point scale ranging from 1-disliked to 9-liked.
In this experiment, the dependent variable is the average preference rating of each jam as a function of group, reasons-analysis versus controls.
Participants who spend time deliberating about the reasons supporting their preferences are expected to show much less correspondence with an objective criterion of quality?the expert ratings?compared to controls. Such findings would support the illusion of introspection, where individuals change decisions because they aren?t truly aware of their attitudes.
Prior to the experiment, conduct a power analysis to recruit a sufficient number of individuals for a study entitled "Jam Taste Test". Call those enlisted and instruct them not to eat anything for 3 hrs prior to the study.
To continue preparation, acquire five brands of jam that vary in overall quality based on expert opinions, and randomly label them A through E. Now, set up the testing room.
Arrange the five brands on the table, and for each participant, record the order and what label each specific jam received.
Then, place a spoonful of each jar?s content directly in front.
To begin, escort a participant into the testing room, and have them sign the consent forms.
Explain that the purpose of the study is to evaluate different kinds of jams under different conditions and they have been randomly chosen to taste on spoons. Remind them to rate their liking preference for each one.
For participants randomly assigned to the reasons-analysis group, provide additional written instructions stating that they need to list their reasons behind liking or disliking each one to organize their thoughts prior to the final evaluations. Note, for control participants, do not provide any further instructions.
Have each participant sit at the table where the five jams were arranged, ask them to start their evaluation, and leave the room.
Following the tasting period, administer a questionnaire and remind them that the purpose of it is to simply organize their thoughts and that they do not need to hand it in. For the control participants, administer a filler questionnaire. Exit the room while the forms are being filled out.
Upon return, explain that this information will no longer be needed and throw it in the trash can. Now, introduce a second experimenter who is unaware of the participants? assigned conditions.
To capture participants? ratings, have this new researcher administer preference forms, in which they should place into a covered box on the table when they finish. Leave the room while they record their evaluations.
To end the experiment, fully debrief all participants and discuss the true purpose and procedures of the study.
To visualize the data, first plot the mean ratings across the five jams, in order of the experts? ratings, and split by condition?controls versus reasons. Notice that control participants were more accurate compared to those in the reasons group, as their scores patterned the professional rankings.
To quantify the amount of agreement, compute correlation values between each participant?s and the expert?s ratings and plot the averages by condition. The average correlation in the control group was near 60%, whereas the mean value in the reasons condition was less than 20%, not significantly different from zero.
Thus, participants who spent time deliberating about reasons supporting their own evaluations showed less correspondence to an objective criterion, which supports the introspection illusion: Thinking too much does impair decision-making!
Now that you are familiar with how introspective thoughts affect decision making in an experimental setting, let?s look at other cases where deliberating between choices leads to suboptimal results.
When it comes to large purchases, like buying a car, many people take their time to weigh the options and consider the pros and cons at every angle, before committing to such an investment.
Although there may be economic advantages, psychologists have found that consumers who exhaustively deliberate these tough choices end up feeling less satisfied with their purchase?essentially showing buyer?s remorse.
Interestingly, older consumers tend to vacillate much less in the car buying process by sticking to brands that they have remained loyal to over the years.
In addition, relationships call for moments of intensive introspection, but all of this deep thought can cause people to make decisions that they also later regret. Researchers postulate that this happens because the introspective process causes them to lose touch with their feelings, leading them to act based on a false impression of their true emotions.
You?ve just watched JoVE?s video on how thinking too much impairs decision-making. Now you should have a good understanding of how to design, conduct, and analyze an experiment to study how our inner thoughts influence behavior, and how introspecting on one?s preferences can be applied to consumer behavior and interpersonal relationships.
Thanks for watching!
View the full transcript and gain access to JoVE Science Education videos
Q1: What is the introspection illusion and how does it affect decision-making?
The introspection illusion occurs when attempting to understand your own perceptions and behaviors results in erroneous explanations. When people introspect about their preferences, they often focus on irrelevant yet salient details that seem plausible but don't actually influence their true attitudes. This conscious reflection can lead to decisions that diverge from what would be optimal based on objective criteria.
Q2: How did the Wilson and Schooler jam experiment demonstrate that thinking too much impairs decision-making?
Participants in the reasons-analysis group listed five reasons for liking or disliking each jam before rating them, while control participants completed a filler task. Control participants' ratings correlated with expert ratings at 60%, whereas the reasons-analysis group showed less than 20% correlation. This demonstrated that deliberating about reasons disrupted alignment with objective quality standards.
Q3: Why do people often make poor decisions when they analyze their reasons for preferences?
When introspecting about preferences, people typically lack genuine understanding of why they feel a certain way. They hone in on irrelevant but salient details that offer plausible explanations, causing them to lose touch with their actual attitudes. This disconnect between conscious reasoning and true preferences leads to decisions that contradict their authentic feelings and expert assessments.
Q4: How does exhaustive deliberation affect consumer satisfaction with major purchases?
Psychologists have found that consumers who exhaustively deliberate tough choices like buying a car end up feeling less satisfied with their purchase, experiencing buyer's remorse. In contrast, older consumers who stick to familiar brands tend to vacillate much less and avoid this dissatisfaction. The introspective process of weighing all options can cause people to lose touch with their genuine preferences.
Q5: What role does consciousness play in everyday decision-making?
Decision-making occurs on a continuum of consciousness levels. At one extreme, people make quick, habitual decisions with minimal thought, like selecting prepared foods for dinner. At the other extreme, choices like selecting a restaurant involve extensive conscious deliberation through reading reviews and weighing pros and cons. Most people lack awareness of why they feel the way they do during these processes.
Q6: How can introspection negatively impact relationship decisions?
Relationships require moments of intensive introspection, but excessive deep thought can cause people to make decisions they later regret. Researchers postulate this occurs because the introspective process causes individuals to lose touch with their actual feelings, leading them to act based on false impressions of their true emotions rather than authentic emotional responses.
Q7: What experimental design elements are critical for studying how introspection affects preferences?
Effective experiments require a reasons-analysis group that deliberates about their choices and a control group that completes unrelated tasks. Both groups then provide anonymous preference ratings using objective scales. A blinded experimenter administers final ratings to prevent bias. Comparing participant ratings against expert standards reveals whether deliberation disrupts decision accuracy.