Methods for Presenting Real-world Objects Under Controlled Laboratory Conditions

Carissa  A. Romero; Jacqueline C. Snow

doi:10.3791/59762

Behavior

Methods for Presenting Real-world Objects Under Controlled Laboratory Conditions

Published: June 21, 2019 doi: 10.3791/59762

Carissa A. Romero¹, Jacqueline C. Snow¹

¹Department of Psychology, University of Nevada, Reno

Summary

We describe methods for presenting real-world objects and matched images of the same objects under tightly-controlled experimental conditions. The methods are described in the context of a decision-making task, but the same real-world approach can be extended to other cognitive domains such as perception, attention, and memory.

Abstract

Our knowledge of human object vision is based almost exclusively on studies in which the stimuli are presented in the form of computerized two-dimensional (2-D) images. In everyday life, however, humans interact predominantly with real-world solid objects, not images. Currently, we know very little about whether images of objects trigger similar behavioral or neural processes as do real-world exemplars. Here, we present methods for bringing the real-world into the laboratory. We detail methods for presenting rich, ecologically-valid real-world stimuli under tightly-controlled viewing conditions. We describe how to match closely the visual appearance of real objects and their images, as well as novel apparatus and protocols that can be used to present real objects and computerized images on successively interleaved trials. We use a decision-making paradigm as a case example in which we compare willingness-to-pay (WTP) for real snack foods versus 2-D images of the same items. We show that WTP increases by 6.6% for food items displayed as real objects versus high-resolution 2-D colored images of the same foods -suggesting that real foods are perceived as being more valuable than their images. Although presenting real object stimuli under controlled conditions presents several practical challenges for the experimenter, this approach will fundamentally expand our understanding of the cognitive and neural processes that underlie naturalistic vision.

Introduction

The translational value of primary research in human perception and cognition hinges on the extent to which the findings transfer to real-world stimuli and contexts. A long-standing question concerns how the brain processes real-world sensory inputs. Currently, knowledge of visual cognition is based almost exclusively on studies that have relied on stimuli in the form of two-dimensional (2-D) pictures, usually presented in the form of computerized images. Although image interaction is becoming increasingly common in the modern world, humans are active observers for whom the visual system has evolved to allow perception and interaction with real objects, not images¹. To date, the overarching assumption in studies of human vision has been that images are equivalent to, and appropriate proxies for, real object displays. Currently, however, we know surprisingly little about whether images effectively trigger the same underlying cognitive processes as do the real objects. Therefore, it is important to determine the extent to which responses to images are like, or different from, those elicited by their real-world counterparts.

There are several important differences between real objects and images that could lead to differences in how these stimuli are processed in the brain. When we look at real objects with two eyes, each eye receives information from a slightly different horizontal vantage point. This discrepancy between the different images, known as binocular disparity, is resolved by the brain to produce a unitary sense of depth²^,³. Depth cues derived from stereoscopic vision, together with other sources such as motion parallax, convey precise information to the observer about the object’s egocentric distance, location, and physical size, as well its three-dimensional (3-D) geometric shape structure⁴^,⁵. Planar images of objects do not convey information about the physical size of the stimulus because only the distance to the monitor is known by the observer, not the distance to the object. While 3-D images of objects, such as stereograms, approximate more closely the visual appearance of real objects, they do not exist in 3-D space, nor do they afford genuine motor actions such as grasping with the hands⁶.

The practical challenges of using real object stimuli in experimental contexts
Unlike studies of the image vision in which stimulus presentation is entirely computer-controlled, working with real objects presents a range of practical challenges for the experimenter. The position, order, and timing of object presentations must be controlled manually throughout the experiment. Working with real objects (unlike images) can involve a significant time commitment due to the need to collect⁷^,⁸^,⁹ or make¹⁰ the objects, set up the stimuli prior to the experiment, and present the objects manually during the study. Moreover, in experiments that are designed to compare, directly, responses to real objects with images, it is critical to match closely the appearance of the stimuli in the different display formats⁸^,⁹. Stimulus parameters, environmental conditions, as well as randomization and counterbalancing of real object and image stimuli, must all be controlled carefully to isolate causal factors and rule out alternative explanations for the observed effects.

The methods detailed below for presenting real objects (and matched images) are described in the context of a decision-making paradigm. The general approach can be extended, however, to examine whether stimulus format influences other aspects of visual cognition such as perception, memory or attention.

Are real objects processed differently to images? A case example from decision-making
The mismatch between the kinds of objects that we encounter in real-world scenarios versus those examined in laboratory experiments is especially apparent in studies of human decision-making. In most studies of dietary choice, participants are asked to make judgments about snack foods that are presented as colored 2-D images on a computer monitor ¹¹^,¹²^,¹³^,¹⁴. In contrast, everyday decisions about which foods to eat are usually made in the presence of real foods, such as at the supermarket or the cafeteria. Although in modern life we regularly view images of snack foods (i.e., on billboards, television screens and online platforms), the ability to detect and respond appropriately to the presence of real energy-dense foods may be adaptive from an evolutionary perspective because it facilitates growth, competitive advantage, and reproduction¹⁵^,¹⁶^,¹⁷.

Research outcomes in scientific studies of decision-making and dietary choice have been used to guide public health initiatives aimed at curbing rising obesity rates. Unfortunately, however, these initiatives appear to have met with little to no measurable success¹⁸^,¹⁹^,²⁰^,²¹. Obesity remains a major contributor to the global burden of a disease²² and is linked to a range of associated health problems, including coronary heart disease, dementia, Type II diabetes, certain cancers, and increased overall risk of morbidity²²^,²³^,²⁴^,²⁵^,²⁶^,²⁷. The sharp rise in obesity and associated health conditions over recent decades²⁸ has been linked with the availability of cheap, energy-dense foods¹⁸^,²⁹. As such, there is an intense scientific interest in understanding the underlying cognitive and neural systems that regulate everyday dietary decisions.

If there are differences in the way foods in different formats are processed in the brain, then this might provide insights into why public health approaches to combating obesity have been unsuccessful. Despite the differences between images and real-world objects, described above, surprisingly little is known about whether images of snack foods are processed similarly to their real-world counterparts. In particular, little is known about whether or not real foods are perceived to be more valuable or satiating than matched images of the same items. Classic early behavioral studies found that young children were able to delay gratification in the context of 2-D colored images of snack foods³⁰, but not when they were confronted with real snack foods³¹. However, few studies have examined in adults whether the format in which a snack food is displayed influences decision-making or valuation¹²^,³²^,³³ and only one study to date, from our laboratory, has tested this question when stimulus parameters and environmental factors are matched across formats⁷. Here, we describe innovative techniques and apparatus for investigating whether decision-making in healthy human observers is influenced by the format in which the stimuli are displayed.

Our study⁷ was motivated by a previous experiment conducted by Bushong and colleagues¹² in which college-aged students were asked to place monetary bids on a range of everyday snack foods using a Becker-DeGroot-Marschak (BDM) bidding task³⁴. Using a between-subjects design, Bushong and colleagues¹² presented the snack foods in one of three formats: text descriptors (i.e., 'Snickers bar'), 2-D colored images, or real foods. Average bids for the snacks (in dollars) were contrasted across the three participant groups. Surprisingly, students who viewed real foods were willing to pay 61% more for the items than those who viewed the same stimuli as images or text descriptors -a phenomenon the authors termed the 'real-exposure effect'¹². Critically, however, participants in the text and image conditions completed the bidding task in a group setting and entered their responses via individual computer terminals; conversely, those assigned to the real food condition performed the task one-on-one with the experimenter. The appearance of the stimuli in the real and image conditions was also different. In the real food condition, the foods were presented to the observer on a silver tray, whereas in the image condition the stimuli were presented as scaled cropped images on a black background. Thus, it is possible that participant differences, environmental conditions, or stimulus-related differences, could have led to inflated bids for the real foods. Following from Bushong, et al.¹², we examined whether the real foods are valued more than 2-D images of food, but critically, we used a within-subjects design in which environmental and stimulus-related factors were carefully controlled. We developed a custom-designed turntable in which the stimuli in each display format could be interleaved randomly from trial to trial. Stimulus presentation and timing were identical across the real object and image trials, thus reducing the likelihood that participants could use different strategies to perform the task in the different display conditions. Finally, we controlled carefully the appearance of the stimuli in the real object and image conditions so that the real foods and images were matched closely for apparent size, distance, viewpoint, and background. There are likely to be other procedures or mechanisms that could allow for randomizing stimulus formats across trials, but our method allows for many objects (and images) to be presented in relatively rapid interleaved succession. From a statistical standpoint, this design maximizes power to detect significant effects more so than is possible using between-subjects designs. Similarly, the effects cannot be attributed to a-priori differences in willingness-to-pay (WTP) between observers. It is, of course, the case that in within-subjects designs open the possibility for demand characteristics. However, in our study participants understood that they could 'win' a food item at the end of the experiment regardless of the display format in which it appeared in the bidding task. Participants were also informed that arbitrarily reducing bids (i.e., for the images) would reduce their chances of winning and that the best strategy for winning the desired item is to bid one’s true value³⁴^,³⁵^,³⁶. The aim of this experiment is to compare WTP for real foods versus 2-D images using a BDM bidding task³⁴^,³⁵.

Subscription Required. Please recommend JoVE to your librarian.

Protocol

The experimental protocols were approved by the University of Nevada, Reno Social, Behavioral, and Educational Institutional Review Board.

1. Stimuli and Apparatus

Figure 1: Real object (displayed on the turntable) and matched 2-D image of the same item (displayed on a computer monitor). The stimuli in this experiment consisted of 60 popular snack food items. The real foods (left panel) were photographed on the turntable and their resulting 2-D images (right panel) were matched closely for the apparent size, distance, viewpoint, and background. Please click here to view a larger version of this figure.

Stimuli
1. Real Objects
  1. Purchase 60 popular snack food items (e.g., Figure 1) from local convenience stores. Ideally, make sure the foods span a wide range of caloric densities (e.g., 0.18 to 6.07)⁷. Open the packaging for each food and place both the package and some of the food on a plate. Use white paper plates to maximize stimulus contrast.
2. 2-D Photographs
  1. Place a plate of food on a cell of the turntable (see Figure 2) and photograph the stimulus on the turntable so that the background of the stimulus in the 2-D image matches the real food counterpart (see Figure 1).
  2. Position a camera (see Table of Materials) on a tripod in front of the turntable. Set the distance, height, and angle, of the camera to match that of the participant’s eyes when the stimulus is viewed from straight-ahead. Position the camera 50 cm (or less) from the edge of the turntable to ensure that the photos are perceived to be within the reaching distance of the participant.
  3. Set, and hold constant, the source of illumination in the testing room. Use a direct source of illumination, such as ceiling lights or a lamp, to provide direct illumination of the stimuli on the turntable. Ensure that the same lighting levels and sources are used during the presentation of the real foods during the experiment. Photograph the real foods on the turntable (using the same lighting sources) using a camera with constant F-stop and shutter speed. Match as closely as possible the overall luminance, shading patterns and specular highlights across display formats. Repeat this process for each stimulus.
  4. If necessary, adjust the 2-D images for color, luminance, and visual size using image processing software (see Table of Materials). Click on the Hue/Saturation and Brightness/Contrast tabs and move the sliders until the image looks as similar as possible to its real-world counterpart when mounted on the turntable.
  5. Fine tune the size of the object in the image by positioning the real object next to the computer monitor and increase/decrease the pixel size until the stimuli are matched exactly for size. Make sure that the image processing software page view (zoom) is set at 100% when editing.
  6. Ensure that the monitor used for editing the images is the same (or the same size) monitor that will be used as the participant monitor during the study. Hold the resolution, aspect ratio and pixels per inch of the images as constant. Additionally, confirm that the monitor is large enough to display the largest stimulus at its full size.

Figure 2: Schematic showing turntable components and assembly. (A) Major components of the turntable device and their relative positioning. (B) Assembled turntable apparatus with 20 individual cells. A real object can be placed inside each cell. The vertical dividers prevent participants from viewing items in neighboring cells. Please click here to view a larger version of this figure.

Turntable apparatus
1. Create a circular (wood) base for the turntable that is 2 m in diameter and a round central core (56 cm in diameter and 24 cm H) with 20 slots (1 cm W; see Figure 2). Place the core on top of a rotating cylinder, allowing for an easy rotation (see Figure 2A).
2. Create 20 dividers (H 24 cm x L 62 cm x W 0.5 cm). Slide each divider into the central core of the turntable to form 20 cells (24 cm x 62 cm x 26 cm).
3. Place the circular base on the top of a table (~72 cm H, see Figure 3A). Make sure the table is at a height that would allow a seated participant to comfortably view items on the turntable.
4. Create a vertical partition (81 cm x 127.5 cm) between the turntable and the participant (see Figure 3B). Place the partition 26 cm from the turntable allowing room for an LCD computer monitor behind the partition. Ensure that the space between the partition and the turntable does not place the stimuli out of reach of the participant.
  1. Construct an aperture in the partition. Make sure the width of the aperture is adjustable so that, in the final setup, the participant can see only one item on the turntable at a time (see Figure 3B). Importantly, ensure that the aperture is wide/high enough that it does not interfere with the participants' physical access to the stimuli on the turntable.
5. Create a sliding platform (L 18.5 cm x W 11.5 cm piece of wood with wheels attached to the underside) for the participant monitor (see Figure 3D).
  1. Place the sliding platform and participant monitor between the turntable and the partition to allow for rapid transitions between display format conditions (see Figure 3D). Position the participant monitor within the viewing aperture during image trials; retract the monitor behind the partition on real object trials (see Figure 3).
6. Use a small desk, or create a shelf, for the experimenter monitor (see Figure 3A,C). Use the experimenter monitor to present prompts as to when to set up a real item or an image, and the identity of the object, for the upcoming trial.
7. Attach a keyboard tray, for the mouse, to the turntable base directly underneath the aperture in the partition (see Figure 3B). Attach a curtain (or similar occluder) between the sides of the turntable and the wall to prevent the participant from viewing the stimuli and the experimenter during the experiment.
8. Purchase (or make) computer-controlled liquid crystal occlusion glasses³⁷ (See Figure 3B,C and Table of Materials).
  NOTE: The occlusion glasses provide millisecond control of stimulus viewing time. The glasses become opaque ('closed state') during the inter-trial interval and transparent ('open state') during stimulus presentation. Computer commands for controlling the glasses (and all other scripts and files necessary for running the protocol described here) are available at http://www.laboratorysys.com/Data/JoVE_Real%20Object_Code.zip).
  1. Test that the glasses open and close correctly (i.e., use 'GlassesTest' script, available at http://www.laboratorysys.com/Data/JoVE_Real%20Object_Code.zip) before the start of the experiment.

Figure 3: How to set-up and use the turntable apparatus for testing. (A) Setup of the turntable apparatus ready for testing. Once the turntable has been assembled it should be placed on a table at a comfortable height for a seated participant. A vertical partition should be created and placed between the participant and the turntable. Within the partition, there should be a viewing aperture. A 'participant monitor' is used for viewing the 2-D images. The LCD monitor should be positioned behind the vertical partition and viewing aperture, and in front of the turntable. The monitor is mounted on a sliding platform that allows it to move in and out of the participant's view across trials. An 'experimenter monitor', which is placed out of view of the participant, is used to inform the experimenter of which stimulus to present on upcoming trials. (B) View of the apparatus and a real object stimulus from the participants' perspective. Only one food item should be visible to a participant at a time. A keyboard tray should be attached to the desk directly in front of where the participant is seated. Participants make responses with a computer mouse. (C) Side view showing the participant monitor mounted on the sliding platform. For image trials, the experimenter slides the participant monitor into the viewing aperture. The participant monitor is retracted behind the vertical partition on real object trials. (D) Aerial schematic showing the setup of the turntable apparatus. A single real object can be placed in each of the 20 cells of the turntable. The participant should be seated in front of the viewing aperture while wearing the computer-controlled visual occlusion glasses. The experimenter can view upcoming trials on the experimenter monitor and manually rotate the turntable, or move the participant monitor, as necessary. Panel C of this figure has been reprinted from reference⁷ with permission from Elsevier. Please click here to view a larger version of this figure.

2. General Procedure: Randomization and Design

Create a script using MATLAB that will randomly interleave real and image trials. Ensure that half of the participants see a given snack food (e.g., an apple) as a real object, and the remaining participants see the item as a 2-D image. For each participant, randomize the order in which the different snack foods are presented within the experiment. Have the script list which real items to place on the turntable, and in what order, prior to the start of the experiment (see 'runStudy' script, available at http://www.laboratorysys.com/Data/JoVE_Real%20Object_Code.zip).
Place the items on the turntable in the correct order (see Figure 3A).
NOTE: Depending on the number of stimuli in the study, set up time may take up to 30 min.
Place the monitor in the aperture and make sure all other items and the experimenter are masked from the participant's view (see 1.2.7).
Seat the participant approximately 50 cm from the turntable and play white noise, either via a white noise machine or via earphones, so that the participant is unable to predict (i.e., from the sound of the sliding monitor) the stimulus format on the upcoming trial.
Give the participant the glasses to put on and make sure that the glasses are in the closed/opaque state. Explain to the participant that the glasses are currently closed but will open when they need to.
View the experimenter monitor to see what type of condition (i.e., real or image) the upcoming trial will be (see Figure 3A).
1. On 'real object' trials, retract the participant monitor from the viewing aperture, via the sliding platform, so that the object is visible to the participant on the turntable (see Figure 1A and 3).
  1. Make a computer command (e.g., a button press) to trigger the opening and closing of the glasses allowing for the real food to be visible on the turntable for 3 s. Once the glasses close, position the participant monitor back in front of the aperture and press a key to open the glasses for the participant to make a response (e.g., a bid). Have the glasses automatically close once the participant enters his/her response (see 'runStudy' script, available at http://www.laboratorysys.com/Data/JoVE_Real%20Object_Code.zip).
View the experimenter monitor to prepare for the next trial. Press a key to advance to the next trial.
1. For 2-D image trials, place the LCD monitor within the viewing aperture (see Figure 1B and Figure 3). Press a key to open the glasses. Leave the monitor in the viewing aperture and press a key to open the glasses for the participant to make a response. Make sure the next stimulus is ready for viewing. Press a key to advance to the next trial.

3. Procedure for Randomization and Design

Create a preference- and familiarity-rating task using the food item images (not the real foods; see 'runStudy', 'LikeSurvey', and 'FamSurvey' scripts, available at http://www.laboratorysys.com/Data/JoVE_Real%20Object_Code.zip). Create two different blocks for the preference- and familiarity-rating tasks and counterbalance the order of blocks across observers (see Figure 4).
1. For each participant, randomize the order of the images presented within each block and create an analog slider for participants to perform their ratings after viewing each food image (see Figure 4, 'runStudy', 'like_slider', and 'Fam_slider' scripts, available at http://www.laboratorysys.com/Data/JoVE_Real%20Object_Code.zip).
Add a bidding task to the script. Randomize as described in 2.1. Add a food auction to the script. Have the computer select randomly one of the 60 food items from the bidding task. Have the computer place a random bid of its own on the selected item from $0 - $3 in 25 cents increments (see Figure 4 and 'bidModule' script, available at http://www.laboratorysys.com/Data/JoVE_Real%20Object_Code.zip).

Figure 4: Experimental design for the current study. The experiment consisted of 4 phases: (1) food preference- and familiarity-rating task, (2) bidding task, (3) food auction, (4) in-lab waiting period. Participants first complete either a preference- or familiarity-rating task (counterbalanced across participants). In the preference task, participants viewed an image of each snack food item for 3 s and then rate how much they liked the item (using a -7 to 7 rating scale) using a sliding analog bid bar. For the familiarity rating task, participants indicated how familiar they were with the item (using a 0 to 3 rating scale). Next, participants completed a bidding task in which they rated how much they were willing to pay ($0-$3) for each snack food item. Half of the stimuli were presented as real foods and half were presented as 2-D images. Viewing time on each trial was controlled using computer-controlled visual occlusion glasses. At the start of the trial, the glasses transitioned to the 'open' (transparent) state for 3 s, before returning to the 'closed' (opaque) state for a 3 s inter-trial interval. The spectacles then opened to allow the participant to record a response. Once the bidding task had been completed, an 'auction' was conducted to determine whether a participant 'won' a food item, and at what price. The auction was followed by a mandatory 30 min waiting period in the lab. If the participant won a food item, they could consume the food during the waiting period. All participants were asked to remain in the lab for the waiting period whether or not a food item was won during the auction. This figure has been reprinted from reference⁷ with permission from Elsevier. Please click here to view a larger version of this figure.

4. Participant Screening and Scheduling

Recruit participants who self-report that they enjoy eating and frequently consume snack foods, and who are familiar with a wide variety of snack foods (typical to the region). Ensure that participants are not actively dieting to lose weight, have any food allergies, dietary restrictions (i.e., vegetarian, gluten-free) or food-related diseases, and are not pregnant.
In accordance with the BDM bidding task¹²^,³⁵, make sure to schedule participants in the late afternoon (e.g., between 1:00 pm and 7:00 pm), which is when snack foods are typically consumed. Remind the participant to refrain from eating for 3 h prior to the experiment¹².
NOTE: This is to ensure that the participant is hungry and will bid accurately for the foods.

5. Questionnaire Procedure

For each participant, collect personal demographic information (i.e., age, sex), ask whether they have a normal or corrected-to-normal vision and record height and weight (these data are useful for calculating body mass index).

6. Preference- and Familiarity Rating Task Procedure

Ask participants to rate how familiar they are with each of the 60 snacks. Have the participant respond via a mouse click on an analog sliding bar (e.g., "0" = not very familiar; "3" = very familiar). Ensure responses are self-paced (see Figure 4).
Ask participants to rate how much they like each of the 60 snacks via a mouse click on an analog sliding bar (e.g., "-7" = strongly dislike; "0" = indifference; "7" = strongly like). Ensure responses are self-paced.

7. Bidding Task Procedure

Inform the participant that they will be given a $3 allowance which can be used to bid on 60 common snack food items. Relay the rules of the bidding task¹⁴^,³⁵.
1. Emphasize that the best strategy is not to bid based on retail prices, but rather, to bid one’s true value: how much one is willing-to-pay to eat the item at the end of the experiment.
2. Remind the participant that there is a mandatory 30 min waiting period in the laboratory at the end of the study (see Figure 4). Explain to the participant that if they 'win' the bidding task they will be able to consume one food item at the end of the experiment; if they 'lose' the bid they will nevertheless be asked to remain in the lab for the duration of the waiting period, without consuming any outside food or drinks.
Seat the participant in the testing room (see 2.4-2.5). Perform a practice auction with three items that are not part of the 60 experimental items. Place the three items in front of the participant one at a time. Ask the participant to rate how much they like the item (-7 to 7).
Place the items in front of the participant once again one at a time. Ask the participant to bid ($0 - $3) on each item. Ensure the participant understands the instructions -ask questions to cross-check comprehension.
Place $3 next to the mouse close to the participants' hand and remind him/her that the allowance is theirs to keep and that they may bid up to $3 per item.
Refer to sections 2.6.1 - 2.7.1. for conducting the real object and images trials. Figure 4 illustrates the bidding task procedure.

8. Food Auction/ 30 Min Waiting Period Procedure

Check to see whether or not the participant 'won' a snack food item and at what price (see 'runStudy' script, available at http://www.laboratorysys.com/Data/JoVE_Real%20Object_Code.zip).
NOTE: The computer will make a bid consisting of a random number between $0 and $3, in 25 cents increments. If the computer's bid is less-than-or-equal-to the participant's bid, then the participant 'wins' the item for consumption. The participant pays the experimenter the price of the computer's bid from their $3 allowance. A number of previous studies have provided an in-depth consideration of the rationale for the BDM bidding task³⁴^,³⁶^,³⁸.

9. Calorie Estimation Procedure

For each food item displayed in the main experiment, present a text display (i.e., 'Snickers bar') and ask the participant to estimate (write down with a pen) how many calories they think are in the serving size.

10. Data Analysis

Use statistical analysis software (see Table of Materials) to perform a linear mixed effects modeling analysis. Use a linear mixed effects model to account for the nested responses within participants (i.e., the dependence of observations from the same participant). Create a dataset with the following variables: Participants, Item, Display Format, Preference, Caloric Density, Estimated Calories, and Bid. Create a model by clicking Analyze, then Mixed Models, then Linear.
1. Transfer the variable Participants into the Subjects: box then press continue. Transfer Bid into the Dependent Variable: box. Next, transfer Subject and Display Format into the Factor(s): box. Then, transfer Preference into the Covariate(s): box.
2. Click Fixed, then select and add all of the variables except Participants into the Model box then click Continue. Click Random, then select and add Participants into the Combinations box in order to account for variability in responses within and across observers. Click Continue.
3. Click Statistics, then check the Descriptive statistics, Parameter estimates, and Tests for covariance parameters boxes. Click Continue. Click EM Means and then select and transfer all of the Factors and Factor Interactions into the Display Means for box. Click Continue. Lastly, press OK.

Subscription Required. Please recommend JoVE to your librarian.

Representative Results

Representative results from this experiment are presented below. A more detailed description of the results, together with a follow-up study, can be found in the original publication⁷. We used a linear mixed effects model with the dependent variable of Bid, and independent variables of Display Format, Preference, Caloric Density, and Estimated Calories. As expected, and in line with previous studies¹²^,¹⁴, there was a strong positive relationship between Preference ratings and Bids (F(1,1655) = 1803.69, p < .001) such that a one unit increase in Preference was associated with an increase of $0.15 in Bid value (β = .15, t(1655) = 42.47, p < .001; d = 8.03). There was also a significant main effect of Caloric Density on Bids (F(1, 1649) = 6.87, p < .01). A one unit increase in Caloric Density was associated with an increase of $.024 in Bids (β = .024, t(1649) = 2.62, p < .01; d = 0.50). The main effect of Estimated Calories was also significant (F(1, 1672) = 6.88, p < .01)¹¹. A one unit increase in Estimated Calories was associated with an increase of $.009 in WTP (β = .009, t(1671) = 2.62, p < .01; d = .50). In other words, observers rated foods that were perceived to be of greater caloric content to be more valuable than foods of lower caloric content. Critically, after controlling for all other factors, we found a significant main effect of Display Format (F(1, 1645) = 7.99, p < .01, d = .53) in which there was a 6.62% increase in Bids for real foods versus food images. The amplification in WTP for real foods (vs. images) was relatively consistent across participants, with 20 out of 28 participants showing the effect. For illustrative purposes, Figure 5 displays average Bid values for each snack food item as a function of Preference, separately for foods shown as real objects (red) and images (blue). Similarly, Figure 6 displays average Bid values for each snack food as a function of Caloric Density, separately for foods in each Display Format. The amplification in WTP for real foods vs. images is evident in both Figure 5 and Figure 6. Importantly, the effect of Display Format on bids was constant across food Preference (F(1, 1644) = .025, p = .88), Caloric Density (F(1, 1643) = 2.54, p = .11) and Estimated Calories (F(1,1643) = .11, p = .74), and there were no significant higher-order interactions between any other factors (all p-values ≥ .11).

Although we observed an effect of estimated calories on Bids, the effect was relatively weak. This result may be explained by the fact that participants performed the estimation task in response to text prompts after the main experiment, rather than while looking at the foods at the time of stimulus presentation. Furthermore, estimating the number of calories in a given food item is not necessarily an intuitive task; many observers are unaware (or do not pay attention to) the caloric density of the foods they consume.

Figure 5: Average monetary bids for each snack food plotted as a function of preference and display format. As expected, there was a strong positive relationship between monetary bids and food preference ratings, with higher bids for foods that were more strongly liked. Importantly, there was a significant main effect of Display Format in which bids for real foods were greater than matched food images. There was no significant interaction between the effect of display format and preference. Mean bid values ($) for the foods are displayed separately for the real foods (red) and 2-D images (blue). Each data point represents the group average bid for each food item, separately for foods in each display format. Solid red and blue lines represent lines of best fit for the real object and image conditions, respectively. This figure has been reprinted from from reference⁷ with permission from Elsevier. Please click here to view a larger version of this figure.

Figure 6: Average monetary bids for each snack food plotted as a function of caloric density and display format. We found a significant positive relationship between bids and actual caloric density, with higher bids for foods of higher caloric density. There was no significant interaction between the effect of display format and caloric density. Mean bid values ($) for the foods are displayed separately for the real foods (red) and 2-D images (blue). Each data point represents the group average bid for each food item, separately for foods in each display format. Solid red and blue lines represent lines of best fit for the real object and image conditions, respectively. This figure has been reprinted from from reference⁷ with permission from Elsevier. Please click here to view a larger version of this figure.

Subscription Required. Please recommend JoVE to your librarian.

Discussion

The overarching goal of the current paper is to facilitate future studies of 'real world' object vision by providing detailed information about how to present large numbers of real-world objects (and images) under controlled experimental conditions. We present an ecologically-valid approach for studying the factors that influence dietary choice and food valuation. We describe methods employed in a recent study of human decision-making⁷ in which we examined whether snack foods presented in the form of real-world objects are valued differently to foods presented as 2-D images. In our experiment⁷, hungry college students placed monetary bids on a range of everyday snack foods. Using a within-subjects design, half of the stimuli were presented to each observer as real foods and the remainder were presented as high-resolution colored 2-D photographs of foods. The real foods and food images were matched closely for apparent size, distance, background, viewpoint, and illumination. In an important departure from previous studies⁷, environmental conditions and stimulus timing were identical across the different display formats. The order of trials in each display format was randomized throughout the experiment using a custom-built turntable device. At the start of the testing session, participants rated their preference for, and familiarity with, sixty different appetitive snack foods (presented as images). In the main experiment, observers indicated their willingness-to-pay (WTP) for each of the sixty foods which were displayed either as real objects or 2-D images. Assignment of food items to the real object or image conditions was counterbalanced across observers. Following from an earlier study that addressed a similar question¹², we measured WTP using a Becker DeGroot Marschak (BDM)³⁵ bidding task in which observers entered a monetary bid ($0-$3) for each snack food to ‘win’ the opportunity to consume a food at the end of the experiment. Given the nested structure of the data, we used linear mixed effects modeling to determine the extent to which WTP was influenced by display format, food preference, caloric content, and estimated calories. We found that observers were willing to pay 6.62% more for foods displayed as real objects versus food images⁷. The amplification in value for real food displays was consistent across all levels of food preference, as well as across actual and estimated caloric content of the foods. These results are surprising because participants knew that they could receive the same (real) snack food reward at the end of the experiment regardless of the format in which the food was presented during the bidding task. Importantly, the findings confirm that there is a reliable 'real-food exposure effect' on willingness-to-pay⁷^,¹² that cannot be accounted for by differences in environmental context, stimulus presentation method, or trial timing across display formats.

In summary, we have provided detailed methods that describe how to prepare real object stimuli and closely-matched 2-D computerized images of the same items, as well as methods for creating a manually-operated turntable for presenting large numbers of real objects and images in interleaved succession. We provided instructions for controlling stimulus presentation and viewing time across all trials, for example, by using computer-controlled display glasses. The methods presented here open up new avenues to examine the underlying mechanisms for the observed effects. For example, future studies could assess directly the impact of stereopsis by presenting real-world stimuli under monocular viewing conditions (which could, for example, be tested easily using monocular vs. binocular states of the computer-controlled glasses described here). This would form a nice comparison with the image-based trials in which both motion parallax and stereopsis provide conflicting depth information.

Although we have offered practical solutions for presenting real-world objects under controlled viewing conditions, working with real objects in the laboratory is undeniably challenging, costly, and time-consuming. In addition to the technicalities associated with controlling stimulus parameters such as illumination, position, size and timing, collection and careful preparation (i.e. mounting) of real object stimuli can be painstakingly slow compared to the time that would be required to prepare images alone. The experimenter(s) must be well-practiced with locating the correct exemplars prior to each trial within required time limits and there are obvious possibilities for experimenter error. In some cases where trial numbers are limited, such as in fMRI⁸^,³⁹ and patient¹⁰ studies of real-object vision, we use a video camera to record which exemplars were presented on each trial and the recordings are cross-checked post-hoc for accuracy. There are additional challenges with working with foods, which are perhaps a unique class of real object stimuli. Depending on the number of items used in the study, a relatively large selection of foods must be kept fresh, on-hand, and in relatively close proximity to the testing room. In decision-making paradigms involving foods, the stimuli are typically shown with the packaging opened and some of the contents visible. Although many manufactured foods seem to have an indefinite shelf life (i.e., the Twinkie) most items need to be replaced regularly to maintain freshness and visual appeal. Together, these conditions make it difficult to control exactly the appearance of the foods between real and image formats to the degree that we have found is possible with non-perishable stimulus classes, such as objects and tools. It is also important to note that we modified our turntable apparatus from the way it appeared in the original study⁷ (black) to the way it is depicted here (white) because we found that the white apparatus was easier to clean and stimulus contrast was improved.

The above considerations raise the critical question of whether or not the time and resource costs of working with real objects are justified, or whether similar results can be obtained using more convenient image displays. The results from our decision-making paradigm⁷ indicate that real food displays elicit a constant increase in valuation (i.e., a linear effect) that does not interact with other factors such as preference or caloric density. These results from decision-making dovetail with findings from other domains of human cognition. For example, real-world objects are more easily recognized¹⁰^,⁴⁰^,⁴¹, enhance memory⁴², and capture attention⁴³^,⁴⁴ more so than images do. Compared to 2-D images, fMRI repetition suppression effects are reduced for real objects⁸. Similarly, fine-grained examination of the temporal dynamics of brain responses to real objects as measured by high-density EEG reveals that real objects (vs. images) elicit stronger and more prolonged desynchronization of the mu rhythm -a signature of activation in visuo-motor networks involved in automatic planning of motor actions⁹. The amplification in mu desynchronization for real objects is independent of early signal differences related to stereopsis⁹. Taken together, these findings suggest that the pattern of results that could be obtained using image displays may be broadly consistent, but just less compelling, than what might otherwise have been observed had real-world objects been used. In other words, if findings from studies of image vision transfer predictably to real object vision, then the translational value of basic research studies of image vision is preserved. Although there is currently insufficient data to make firm conclusions on this issue, recent evidence for dissociations in the effects of real objects across motor areas in the left versus right hemispheres⁹ and across egocentric distances⁶ raise concerns about this assumption. For example, the effect of real objects on attentional capture fall to the levels observed for 2-D and 3-D images when the objects are positioned outside of reach of the observer, or when they are within reach but behind a transparent barrier⁶, suggesting that the potential for manual interaction with a real object (but not an image) determines how it is processed. Future studies could use the protocols described here to investigate whether similar underlying causal mechanisms modulate ‘real-food exposure effects' on willingness-to-pay. For example, a distance or barrier manipulation⁶ could be employed to determine whether real snack foods that are reachable or graspable are processed differently to those that are not (and to determine whether the same manipulation has any effect on processing of food images). Future studies using ecologically-valid real-object stimuli are required to make definitive conclusions on this issue. Importantly, it may not be the case that similar mechanisms are at play in different cognitive domains, or in different tasks. Nevertheless, our approach to working with real-world objects promises to provide important new insights into the underlying processes and mechanisms that drive naturalistic vision.

Subscription Required. Please recommend JoVE to your librarian.

Disclosures

The authors have nothing to disclose.

Acknowledgments

This work was supported by grants to J.C. Snow from the National Eye Institute of the National Institutes of Health (NIH) under Award Number R01EY026701, the National Science Foundation (NSF) [grant 1632849] and the Clinical Translational Research Infrastructure Network [grant 17-746Q-UNR-PG53-00]. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH, NSF or CTR-IN.

Materials

Name	Company	Catalog Number	Comments
EOS Rebel T2i Body Camera	Canon	4462B001
MATLAB	MathWorks	R2017b	Computer programming software. Download this additional free toolbox: PsychToolbox 3.0.14
Photoshop	Adobe	CS6
PLATO Visual Occlusion Glasses	Translucent Technologies Inc.	N/A
SPSS	IBM	Version 22	Statitical analysis software
ToTaL Control System (USB)	Translucent Technologies Inc.	N/A	The ToTaL Control System controls the PLATO spectacles

DOWNLOAD MATERIALS LIST

References

Gibson, J. J. The Ecological Approach to Visual Perception. , Houghton Mifflin. Boston. (1979).
Westheimer, G. Three-dimensional displays and stereo vision. Proceedings of the Royal Society B: Biological Sciences. 278 (1716), 2241-2248 (2011).
Julesz, B. Foundations of cyclopean perception. , U. Chicago Press. Oxford. (1971).
Sprague, W. W., Cooper, E. A., Tošić, I., Banks, M. S. Stereopsis is adaptive for the natural environment. Science Advances. 1 (4), (2015).
Nityananda, V., Read, J. C. A. Stereopsis in animals: evolution, function and mechanisms. Journal of Experimental Biology. 220, Pt 14 2502-2512 (2017).
Gomez, M. A., Skiba, R. M., Snow, J. C. Graspable Objects Grab Attention More Than Images Do. Psychological Science. 29 (2), 206-218 (2018).
Romero, C. A., Compton, M. T., Yang, Y., Snow, J. C. The real deal: Willingness-to-pay and satiety expectations are greater for real foods versus their images. Cortex. 107, 78-91 (2018).
Snow, J. C., et al. Bringing the real world into the fMRI scanner: Repetition effects for pictures versus real objects. Scientific Reports. 1, 130 (2011).
Marini, F., Breeding, K. A., Snow, J. C. Distinct visuo-motor brain dynamics for real-world objects versus planar images. NeuroImage. , (2019).
Holler, D., Behrmann, M., Snow, J. C. Real-world size coding of solid objects, but not 2-D or 3-D images, in visual agnosia patients with bilateral ventral lesions. Cortex. , (2019).
Tang, D. W., Fellows, L. K., Dagher, A. Behavioral and neural valuation of foods is driven by implicit knowledge of caloric content. Psychological Science. 25 (12), 2168-2176 (2014).
Bushong, B., King, L. M., Camerer, C. F., Rangel, A. Pavlovian processes in consumer choice: The physical presence of a good increases willingness-to-pay. The American Economic Review. 100 (4), 1556-1571 (2010).
Rangel, A. Regulation of dietary choice by the decision-making circuitry. Nature Neuroscience. 16 (12), 1717-1724 (2013).
Plassmann, H., O'Doherty, J. P., Rangel, A. Appetitive and aversive goal values are encoded in the medial orbitofrontal cortex at the time of decision making. Journal of Neuroscience. 30 (32), 10799-10808 (2010).
Berthoud, H. R. Metabolic and hedonic drives in the neural control of appetite: who is the boss. Current Opinion in Neurobiology. 21 (6), 888-896 (2011).
Sclafani, A. Conditioned food preferences. Bulletin of Psychonomic Society. 29 (2), 256-260 (1991).
Volkow, N. D., Wise, R. A. How can drug addiction help us understand obesity. Nature Neuroscience. 8 (5), 555-560 (2005).
Drewnowski, A., Darmon, N. Food choices and diet costs: An economic analysis. Journal of Nutrition. 135 (4), 900-904 (2005).
Imram, N. The role of visual cues in consumer perception and acceptance of a food product. Nutrition and Food Science. 99 (5), 224-230 (1999).
Marteau, T. M., Hollands, G. J., Fletcher, P. C. Changing human behavior to prevent disease: The importance of targeting automatic processes. Science. 337 (6101), 1492-1495 (2012).
Neal, D. T., Wood, W., Quinn, J. M. Habits: A repeat performance. Current Direction in Psychology. 15 (4), 198-202 (2006).
Wellman, N. S., Friedberg, B. Causes and consequences of adult obesity: Health, social and economic impacts in the United States. Asia Pacific Journal of Clinical Nutrition. 11, 705-709 (2002).
Canoy, D., et al. Coronary heart disease incidence in women by waist circumference within categories of body mass index. European Journal of Preventive Cardiology. 20 (5), 759-762 (2013).
Whitmer, R. A., et al. Central obesity and increased risk of dementia more than three decades later. Neurology. 71 (14), 1057-1064 (2008).
Bean, M. K., Stewart, K., Olbrisch, M. E. Obesity in America: Implications for clinical and health psychologists. Journal of Clinical Psychology in Medical Settings. 15 (3), 214-224 (2008).
Brownell, K. D., Gold, M. S. Food and addiction: A comprehensive handbook. , Oxford University Press. (2012).
Klein, S., et al. Waist circumference and cardiometabolic risk: A consensus statement from shaping America's health: Association for Weight Management and Obesity Prevention; NAASO, the Obesity Society; the American Society for Nutrition; and the American Diabetes Association. Obesity. 15 (5), 1061-1067 (2007).
Zhang, Y., et al. Obesity: Pathophysiology and Intervention. Nutrients. 6 (11), 5153-5183 (2014).
Afshin, A., et al. Health effects of overweight and obesity in 195 countries over 25 years. New England Journal of Medicine. 377 (1), 13-27 (2017).
Mischel, W., Moore, B. Effects of attention to symbolically presented rewards on self-control. Journal of Personality and Social Psychology. 28 (2), 172-179 (1973).
Mischel, W., Ebbesen, E. B., Zeiss, A. R. Cognitive and attentional mechanisms in delay of gratification. Journal of Personality and Social Psychology. 21 (2), 204-218 (1972).
Gross, J., Woelbert, E., Strobel, M. The fox and the grapes-how physical constraints affect value based decision making. PLoS One. 10 (6), 0127619 (2015).
Müller, H. The real-exposure effect revisited - purchase rates vary under pictorial vs. real item presentations when consumers are allowed to use their tactile sense. International Journal of Market Research. 30 (3), 304-307 (2013).
Becker, G. M., DeGroot, M. H., Marschak, J. Measuring utility by a single-response sequential method. Behavior Science. 9 (3), 226-232 (1964).
Becker, G. M., DeGroot, M. H., Marschak, J. Stochastic models of choice behavior. Behavior Science. 8 (1), 41-55 (1963).
Plassmann, H., O'Doherty, J., Rangel, A. Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions. Journal of Neuroscience. 27 (37), 9984-9988 (2007).
Milgram, P. A spectacle-mounted liquid-crystal tachistoscope. Behavior Research Methods. 19 (5), 449-456 (1987).
Johnson, E. J., Haubl, G., Keinan, A. Aspects of endowment: A query theory of value construction. Journal of Experimental Psychology: Learning Memory and Cognition. 33 (3), 461-474 (2007).
Freud, E., et al. Getting a grip on reality: Grasping movements directed to real objects and images rely on dissociable neural representations. Cortex. 98, 34-48 (2018).
Chainay, H., Humphreys, G. W. The real-object advantage in agnosia: Evidence for a role of surface and depth information in object recognition. Cognition Neuropsychology. 18 (2), 175-191 (2001).
Humphrey, G. K., Goodale, M. A., Jakobson, L. S., Servos, P. The role of surface information in object recognition: Studies of a visual form agnosic and normal subjects. Perception. 23 (12), 1457-1481 (1994).
Snow, J. C., Skiba, R. M., Coleman, T. L., Berryhill, M. E. Real-world objects are more memorable than photographs of objects. Frontiers in Human Neuroscience. 8, 837 (2014).
Gomez, M. A., Snow, J. C. Action properties of object images facilitate visual search. Journal of Experimental Psychology: Human Perception and Performance. 43 (6), 1115-1124 (2017).
Gerhard, T. M., Culham, J. C., Schwarzer, G. Distinct visual processing of real objects and pictures of those objects in 7- to 9-month-old infants. Frontiers in Psychology. 7, 827 (2016).

Behavior