Summary
We discuss a novel method forviewpoint-rotation of visual stimuli, and demonstrate using a mirror stereoscopethe three-dimensional percept of rotation-in-depth. The technique can be used to investigate the role of stereoscopic cues in encoding viewpoint-rotated figures.
Abstract
The speed and accuracy of object recognition is compromised by a change in viewpoint; demonstrating that human observers are sensitive to this transformation. Here we discuss a novel method for simulating the appearance of an object that has undergone a rotation-in-depth, and include an exposition of the differences between perspective and orthographic projections. Next we describe a method by which human sensitivity to rotation-in-depth can be measured. Finally we discuss an apparatus for creating a vivid percept of a 3-dimensional rotation-in-depth; the Wheatstone Eight Mirror Stereoscope. By doing so, we reveal a means by which to evaluate the role of stereoscopic cues in the discrimination of viewpoint rotated shapes and objects.
Introduction
Many studies have shown that the speed and accuracy of object recognition is compromised by a change in viewpoint1-7. For example, face recognition is best when the face is viewed in the fronto-parallel plane, i.e. "front on", and declines systematically as the face is rotated in depth away from the fronto-parallel plane. Changes in viewpoint can occur either when the observer, or the viewed object, moves. Importantly, the viewpoint cost is often reduced by the introduction of stereoscopic cues to the observer1-4,6,8. It is, however, important to distinguish between the rotation of 3D and planar objects. The former may reveal new textures or structural information under rotation while this does not occur with planar objects. In this communication we discuss a method for simulating the appearance of planar shapes and objects that have been subject to rotation-in-depth, or RID. We describe an apparatus for turning the two-dimensional projection of a RID image into a vivid 3D (three-dimensional) percept of RID that is based on the Wheatstone mirror stereoscope9. Finally, we report a psychophysical method for determining whether stereoscopic (3D) cues influence our sensitivity to viewpoint change. Movie 1 demonstrates how the appearance of a shape is altered by a viewpoint change/RID.
Simulating a Viewpoint Change
When an object is subject to RID it does not of course physically change in identity or shape. However from the observer’s viewpoint, the shape does change. For instance, if you hold a sheet of paper directly in front of you in landscape view, the image of the paper in your eye, termed the retinal image, is a horizontal rectangle. If you then rotate the paper around its central vertical axis, the retinal image becomes foreshortened along the horizontal (see Figure 1 for a demonstration). Ultimately, if you rotate the paper through 90° the retinal image is a thin strip the thickness of the paper. Thus the largest change to the image of the sheet of paper that has been subject to RID is a horizontal compression of its shape. A smaller change gradient of change occurs in the vertical extent with the image being taller for the nearest edge unchanged at the axis of rotation and shorter for the farther edge.
A RID can be simulated by appropriate transformation of images using software packages or by writing one’s own code8,10. Readers wishing to know more about RID can consult relevant texts and websites concerned with viewpoint transformations, or learn about built-in functions in the image processing software of packages such as those listed in our Materials. However, when transforming an image of a stimulus on a screen to represent a particular angle of RID there is an additional consideration: which type of projection to use, orthographic or perspective? When we view a scene, nearby objects produce larger retinal images than farther away objects. For a 2D (two-dimensional) object that is subject to RID this means that the far side is compressed more than the near side, as Figure 1 illustrates (see in particular the bottom right figure which has a RID of 60°). The magnitude of the compression asymmetry is dependent upon the size of the object being subject to RID, and on our distance from it. Perspective projection incorporates this compression asymmetry whereas orthographic projection simply ignores it by assigning the same dimensions to the near and far sides of the RID object. Since an observer viewing a planar object rotate will be presented with this asymmetry we will use perspective projection here. Another advantage of our method is that the projection takes into account viewing distance from observer to screen. We have previously used our RID method to simulate viewpoint rotated shapes in both 2D perspective projection and in 3D8,10.
Simulating a 3D Viewpoint Change
In order to simulate stereoscopic RID one needs to present slightly different images to the left and right eyes. This difference between the two images is known as binocular disparity11, and is a function of both viewing distance and between-eye separation, specifically the inter-pupillary distance. Returning to the example of our sheet of paper in landscape view, if we rotate the piece of paper around the vertical center, clockwise by 45° (so the left edge is receding), each eye’s view of the paper is slightly different. The image projected onto the left eye is slightly less than 45° rotation while the image projected onto the right eye is slightly more than 45° rotation (see Figure 2 for an illustration - observers who can free fuse the top two images will perceive a RID in 3D). If the appropriate RID angles are presented separately to the left and right eyes (the stereo condition) they will produce a disparity gradient and the observer is likely to report a vivid impression of a 3D RID. However if identical images, representing a 45° rotation from the central viewing point, are presented to both eyes (the nonstereo condition) the observer will see a 2D representation of the RID stimuli. An established method for presenting separate images to the left and right eyes using a single monitor is to use a modification of the mirror stereoscope originally developed by Sir Charles Wheatstone in the mid 19th century12 – see Figure 3. When the two stereo-half-images are positioned at corresponding locations in each eye’s view, they can be binocularly fused to produce a 3D percept (see Howard and Rogers12 for an extensive review of stereo methods and limitations).
By combining a simulated viewpoint change with stereoscopic presentation we can determine, for example, the role that stereoscopic cues play in our ability to detect changes in viewpoint. The demonstration results below reveal that when the size of a planar object, in this case a curved contour, is known or familiar to the observer, the change in the 2D width of the curve (sometimes termed its sag) is used to detect the change in RID, i.e. viewpoint (see Bell et al.10 for the full manuscript on this effect). Below we describe in detail a psychophysical method for measuring sensitivity to RID change for 2D- and 3D-defined planar shapes.
Subscription Required. Please recommend JoVE to your librarian.
Protocol
1. Prescreening
- Prior to testing, obtain written and informed consent from each participant.
- Ask the subject if they have normal or corrected-to-normal acuity (i.e. glasses).
- Verify the stereo acuity of the participant using a simple handheld stereo test.
2. Achieving Binocular Fusion in the Stereoscope
- Seat the participant at the desk in front of the mirror stereoscope and adjust the height of their chair for comfort.
- Turn off the lights so that they may dark adapt for a minimum of 3 min. The visual system performs best when it is adapted to the prevailing lighting conditions.
- Have the participant look through the stereoscope. They will see a cross displayed on the monitor for each eye. They must adjust the horizontal position of the fixation crosses until they see one cross only, in the center of their field of view. If they see two crosses, then this separation is incorrect and horizontal adjustment is needed. The targets are centered on the crosses when displayed and this procedure ensures fusion at the correct distance.
- The participant is now ready to commence the experiment.
3. Task Instructions for the Observer
- "On each trial you will see two curves that have rotation-in-depth (RID). You are required to decide which of the two curves has a greater RID, in other words which is more rotated away from your view. Press the left mouse button if the first stimulus has a greater RID. Press the right mouse button if the second curve has more RID."
- "This is a forced-choice task in which the next trial does not commence until a response has been made to a previous trial."
- "If you are not sure which stimulus has more RID, please take your best guess."
4. Measuring Sensitivity to Viewpoint Change/RID in Contours
- To evaluate the role of stereoscopic cues in judgments of RID, the performance of the participant is measured separately in conditions where stereoscopic cues to RID are presented (stereo conditions) and where they are not presented (nonstereo conditions). To present stereoscopic cues, the computer displays a different RID angle of curve to each eye of the observer, through the stereoscope. To present the curves without stereo cues, the computer presents the same RID curve to both eyes (see Figure 2 for an example of this). The methodological steps below are then repeated for both conditions.
- Initiate the procedure, composed of 140 trials, for the participant. Note: The Method of Constant Stimuli (MOCS13) is used to ensure equal numbers of trials for each value of RID of the test curve employed. The RIDs can be either greater than, less than or equal to 45°, with the particular value being randomly chosen from the set for each trial. The RID of the reference curve is held constant at 45° RID. The presentation order of reference and test stimuli is randomized on each trial.
- Allow the program to record how often the participant chooses the test as being at the greater RID angle.
- Give the participants a break after each block of trials. In this example the 140 trials are made up of 20 repetitions at each of 7 different RID angles for the test curve: 3 angles less than 45°, 3 angles greater than 45° and 1 RID at precisely 45° - the same as the reference curve.
- At the end of each run the computer plots out data describing the proportion of times that the participant chose the test stimulus as being at a RID angle greater than 45°, as a function of the presented RID angle (see example data points in Figure 4).
- Let the software fit the data with a Cumulative Gaussian function in order to determine the precision with which they are able to make the RID discrimination. This precision is given by the parameter of the fitted function that determines slope (see example solid lines in Figure 4).
- Obtain the slope from each plot. The slope estimates for the stereo and nonstereo conditions can then be statistically compared in order to determine whether stereoscopic cues have contributed to an observer’s ability to detect a change in RID.
Subscription Required. Please recommend JoVE to your librarian.
Representative Results
Figure 4 plots example results obtained from such an experiment. The y-axis in each figure describes the proportion of times that the test pattern was chosen as having a greater RID. Rather than plotting the absolute number of times this occurred, this is given as a proportion. The x-axis describes the physical RID of the test pattern; remembering that here, the reference pattern was held at a constant 45° RID. The data points represent the proportion of times that the observer chose the test curve as having the greater RID for a range of physical RIDs. The computer only scores the number of times that the test pattern is chosen as having a greater RID; this means that data for an "ideal" observer should range between zero: never chose the test as greater in RID when it had a smaller RID than the reference; and one: always chose the test as greater in RID when it had a larger RID than the reference.
From this plotted data we can determine an observer’s sensitivity to changes in RID. To do this we fit the data to a Cumulative Gaussian function using a statistical package (see Materials). The fit enables us to estimate the precision, or sensitivity of an observer’s judgment of RID. In the figure, it is clear from the steepness of the solid lines in each plot that observers are highly sensitive to a change in RID angle. This means that observers are highly sensitive to a RID manipulation of a planar object; here a curved contour. However, in our paradigm we have also chosen to compare performance in two conditions: stereo and nonstereo. Since the data representing these two conditions physically overlaps, it appears that on this task there is no difference in precision between stereo and nonstereo conditions. An additional statistical comparison of precision estimates can be performed in order to determine whether or not the data statistically differ.
Movie 1. Demonstrates the change in the two dimensional (2D) retinal image of a shape that is being subject to a rotation-in-depth (RID) around vertical. Here the viewer can see that the main change to the image of a shape that has RID is a horizontal compression of its shape.
Figure 1. Static illustration of how the retinal image of a sheet of paper is altered as the paper is RID. Here the viewer can also appreciate the perspective projection method that we apply. That is, as the paper is increasingly RID the reader can see that the far (left) edge of the paper is being more compressed in vertical size than the near edge. This asymmetry in the size of each edge occurs because near objects produce larger retinal images while far objects produce smaller retinal images. Perspective projections, which we have used, incorporate this asymmetry while orthographic projections ignore this effect. The title above each image shows its angle of RID. Click here to view larger image.
Figure 2. Using binocular disparity to present in 3D. If two matched images at slightly different RID angles are presented separately to the left and right eye, a stereo (3D) RID can be perceived. For example, to achieve the percept of a sheet of paper that is RID 45° in 3D (left edge receding) the image projected to the left eye should be one at an angle less than 45° while that projected to the right eye should be at an angle greater than 45°. Click here to view larger image.
Figure 3. A top down view of our adapted Wheatstone eight mirror stereoscope, with top cover removed. The red lines and arrows depict the separate paths of light from the monitor to each eye. This mirror stereoscope incorporates 8 first surface mirrors; four along the path to each eye. Using a series of mirrors in this way allows for a viewing distance of approximately 1 m from the screen to the observer: a comfortable viewing distance. All nonmirrored surfaces have been painted in matte black to avoid shadow and reflectance issues. Click here to view larger image.
Figure 4. Example results for three observers showing sensitivity to RID/viewpoint change (vertical axes) as a function of the RID angle of the stimuli. Raw data points plot the proportion of time that the test stimulus, here a curved contour, was chosen as being greater than 45° RID. The horizontal axis plots the RID of the test curve. The reference curve was constant at 45° in all conditions. Solid lines show the fitted Cumulative Gaussian function to each set of obtained data. Black points and lines show stereo (different RID image projected to each eye) condition data while Grey points and lines show nonstereo (same RID image projected to each eye) condition data. Click here to view larger image.
Subscription Required. Please recommend JoVE to your librarian.
Discussion
The methods discussed in this communication can be used by any researcher seeking to address questions related to discrimination and/or recognition of viewpoint rotated objects. Importantly, the method of perspective projection is not limited to the stimuli demonstrated here; instead it can be applied to a wide range of experimental stimuli, from lines to pictures of complex objects. Our introduction describes ways to apply this transformation using custom software, or library software such as MATLAB.
There is considerable interest in understanding how we recognize objects across different viewpoints. It should be noted that there are several ways to simulate a change in the 3D structure and orientation of surfaces and objects, including through the use of shading or brightness variations6,14 and occlusion11. In fact, changes in viewpoint are often correlated with variations in these attributes, and so future studies must investigate the role of different types of cues in isolation as well as in combination. Our method provides a means for contributing to that goal.
Our method also offers promise for researchers outside of the fields of visual perception and cognition. For instance, researchers have begun to investigate the effects of viewpoint change on a number of different behavioral dimensions, including on postural sway15, and also, on the reading of material presented on computerized displays16. Our method provides a means by which to control the 2D and 3D information presented to the observer, during such tasks. This could allow for a more in depth understanding of the visual inputs to the vestibular and language systems.
Creating a Mirror Stereoscope
An exciting aspect of the current method is the use of the mirror stereoscope to facilitate the percept of a 3D rotation-in-depth, or RID. The stereoscope apparatus that we have described here (Figure 3) was custom built for our lab. Should researchers wish to build such a device, it is important to note that they must use front surface, or first surface mirrors. These are mirrors with the mirrored surface on the front, as opposed to the back of the glass, as in traditional mirrors. The importance of this point can easily be appreciated if readers put their finger on the glass surface of a traditional mirror; they will invariably observe a faint ‘ghost’ reflection of their finger caused by light reflecting back from the front surface of the glass. This makes traditional mirrors unsuitable for use in visual experiments, especially when the light path travels across multiple mirrored surfaces as in our stereoscope. First-surface mirrors can be purchased through specialist companies. Alternatively, researchers may prefer to buy a ready-made stereoscope. Many companies offer "ready-to-go" stereoscope solutions that can be fitted directly onto existing monitors and laptops alike. See our Materials section for listings of relevant companies.
Limitations and Troubleshooting
Presenting information in 3D represents an opportunity to move beyond the traditional flat panel display and to create a more realistic representation of your experimental stimulus. It is however critical that you establish first that your experimental participants have stereo vision. There are a number of different prescreening devices for this purpose. Many of these can be purchased on line for a small cost (see Materials for an example). The test we employ here determines whether an individual’s stereo acuity falls into the normal range. If these tests reveal that your observers cannot effectively see in stereo then they are not suitable for testing on stereo tasks. Such observers could however still participate in experiment aimed at exploring 2D representations of a RID.
Before testing for the first time, it is important that the experimenter is trained in the use of the prescreening stereo tests, in running experimental control software in MATLAB, and in the statistical analysis of experimental results from this paradigm. Prior to each testing sessions, ensure that all hardware connections are secure and that the mirror stereoscope is positioned correctly against the display monitor: such that no additional light can enter the stereoscope device.
Subscription Required. Please recommend JoVE to your librarian.
Disclosures
The authors declare that they have no competing financial interests. The first author Jason Bell, is a full-time employee of the Australian National University; David Badcock and J Edwin Dickinson are full-time employees of the University of Western Australia; Fred Kingdom is a full-time employee of McGill University.
Acknowledgments
This research was supported by the Australian Research council. ARC Grant # DP110101511 given to JB; National Sciences and Engineering Research Council, NSERC of Canada Grant # RGPIN121713-11 given to FK ; ARC grant # DP130102580 and DP110104553 to DRB.
Materials
| Name | Company | Catalog Number | Comments |
| Stereo testing | http://www.stereooptical.com/products/stereotests | Titmus stereofly test | Handheld testing devices |
| Prebuilt stereo devices | http://www.stereoaids.com.au | prebuilt stereoscopes | Suitable for desktop and laptop computers |
| Mirror stereoscope | Custom made; or, Edmundoptics | 8 mirror stereoscope | www.edmundoptics.com |
| Graphics card | NVidia | NVidia or ATI recommended | |
| Experimental control software | MATLAB | MATLAB, Python, C++, Visual Basic recommended | |
| Experimental control software | Psychtoolbox.org | Psychtoolbox | Free to download but requires MATLAB |
| Statistical software | Graphpad | Prism V5 | http://www.graphpad.com/ |
| Statistical software | Palamedes | Psychophysics guide | http://www.palamedestoolbox.org |
| 3D Projection documentation | Wikipedia | http://en.wikipedia.org/wiki/3D_projection |
References
- Bennett, D. J., Vuong, Q. C. A stereo advantage in generalizing over changes in viewpoint on object recognition tasks. Percep. Psychophys. 68, 1082-1093 (2006).
- Bulthoff, H. H., Edelman, S. Y., Tarr, M. J. How are three-dimensional objects represented in the brain? Cereb. Cortex. 5, 247-260 (1995).
- Burke, D. Combining disparate views of objects: Viewpoint costs are reduced by stereopsis. Vis. Cogn. 12, 705-719 (2005).
- Burke, D., Taubert, J., Higman, T. Are face representations viewpoint dependent? A stereo advantage for generalising across different views of faces. Vis. Res. 47, 2164-2169 (2007).
- Edelman, S., Bulthoff, H. H. Orientation dependence in the recognition of familiar and novel views of three-dimensional objects. Vis. Res. 32, 2385-2400 (1992).
- Lim Lee, Y., Saunders, J. A. Stereo improves 3D shape discrimination even when rich monocular shape cues are available. J. Vis. 11, (2011).
- Parr, L. A., Siebert, E., Taubert, J. Effect of familiarity and viewpoint on face recognition in chimpanzees. Perception. 40, 863-872 (2011).
- Bell, J., Dickinson, J. E., Badcock, D. R. Radial frequency adaptation suggests polar-based coding of local shape cues. Vis. Res. 48, 2293-2301 (2008).
- Wade, N. J. On the late invention of the stereoscope. Perception. 16, 785-818 (1987).
- Bell, J., Kanji, J., Kingdom, F. A. A. Discrimination of rotated-in-depth curves is facilitated by stereoscopic cues, but curvature is not tuned for stereoscopic rotation-in-depth. Vis. Res. 77, 14-20 (2013).
- Regan, D. Human Perception of Objects. , Sinauer Associates, Inc. (2000).
- Howard, I. P., Rogers, B. J. Binocular Vision and Stereopsis. , Oxford University Press. (1995).
- Kingdom, F. A. A., Prins, N. Psychophysics: A Practical Introduction. , Academic Press: an imprint of Elsevier. (2010).
- Kingdom, F. A. Color brings relief to human vision. Nat. Neurosci. 6, 641-644 (2003).
- Ustinova, K. I., Perkins, J. Gaze and viewing angle influence visual stabilization of upright posture. Brain Behav. 1, 19-25 (1002).
- Shieh, K. -K., Lee, D. -S. Preferred viewing distance and screen angle of electronic paper displays. Appl. Ergon. 38, 601-608 (2007).
