Methods to Explore the Influence of Top-down Visual Processes on Motor Behavior

Summary

It is unclear how top-down signals from the ventral visual stream affect movement. We developed a paradigm to test motor behavior towards a target on a 3D depth inversion illusion. Significant differences are reported in both deliberate, goal-directed movements and automatic actions under illusory and veridical viewing conditions.

Abstract

Kinesthetic awareness is important to successfully navigate the environment. When we interact with our daily surroundings, some aspects of movement are deliberately planned, while others spontaneously occur below conscious awareness. The deliberate component of this dichotomy has been studied extensively in several contexts, while the spontaneous component remains largely under-explored. Moreover, how perceptual processes modulate these movement classes is still unclear. In particular, a currently debated issue is whether the visuomotor system is governed by the spatial percept produced by a visual illusion or whether it is not affected by the illusion and is governed instead by the veridical percept. Bistable percepts such as 3D depth inversion illusions (DIIs) provide an excellent context to study such interactions and balance, particularly when used in combination with reach-to-grasp movements. In this study, a methodology is developed that uses a DII to clarify the role of top-down processes on motor action, particularly exploring how reaches toward a target on a DII are affected in both deliberate and spontaneous movement domains.

Introduction

Vision-for-Perception vs. Vision-for-Action

In order to successfully navigate the environment, information from the visual system is utilized to help coordinate human movement. How visual information is selected and prioritized to influence motor actions remains unclear. Two major anatomical projections arise from the primary visual cortex to form the ventral ("what", or "vision for perception") pathway, extending to the temporal area, and the dorsal ("where", or "vision for action") pathway, to the parietal lobe^1-2. The ventral stream is implicated in utilizing visual information for perceptual processes such as object recognition and identification, whereas the dorsal stream is thought to exclusively process signals for action guidance and spatial awareness. The question asked is whether or not top-down processes from the ventral stream shape the way in which movements are executed.

The famous case study of Patient DF, evaluated by Goodale and Milner in 1992, provided strong evidence and support for the visual two-streams hypothesis, which claims that ventral and dorsal stream processes are separable for perception and action³. In theory, bottom-up signals of motion parallax and binocular disparity can override top-down perceptual information such as prior knowledge and familiarity in order to accurately guide our actions, suggesting that motor planning is impervious to ventral stream control. DF, who suffered from visual form agnosia caused by bilateral ventral occipital lesions, retained accurate grasping ability towards objects that she had difficulty recognizing, supporting the premise of the visual two-streams hypothesis^3-4. Because of case studies like DF, it was assumed that the functional ventral-dorsal stream dichotomy also existed in healthy, nonpathological individuals. However, whether or not these findings provide evidence for an absolute division of labor for perception and action in neurotypical populations has been hotly debated over the past twenty years^5-10.

The Use of Illusions to Segregate Perception and Action

To test the visual two-streams hypothesis in neurotypical subjects, researchers employ visual illusions to investigate how skewed perceptual judgments of the environment affect our motor actions. The Ebbinghaus/Titchener Illusion, for example, uses a disk target surrounded by smaller disks that appears to be larger than another disk of the same size surrounded by larger circles; this is due to a size-contrast effect¹¹. When participants reach to grasp the disk target, if the two-streams hypothesis holds true, then the grip aperture of the hand grabbing at the disk target would be unaffected by the illusion, causing the participant to act on the true geometry of the disk target rather than rely on incorrect perceptual size estimates. Aglioti et al. in fact report this behavior, reasoning that separate visual processes govern skilled actions and conscious perception¹¹. Conversely, other groups have contested these results, finding no dissociation between perception and action processes when carefully controlling the matching of perceptual and grasping tasks, proposing an integration of visual stream information rather than a separation¹². Despite several follow-up studies conducted to validate or refute the visual two-streams hypothesis using the Ebbinghaus Illusion, there are competing pieces of evidence to support both sides of the argument¹³.

To further explore the influence of visual perception on action processes, 3D depth inversion illusions (DII) have also been utilized. DIIs produce illusory motion and perceived depth reversal of scenes in which physically concave angles are perceived as convex and vice versa¹⁴. The Hollow Face Illusion is an example of a DII that generates the perception of a normal, convex face although the stimulus is physically concave, implicating the role of top-down influences such as prior knowledge and convexity bias to elicit the illusory percept^15-16. Despite efforts to characterize motor behavior in reaching towards targets on the Hollow Face Illusion, evidence remains equivocal: one study reports an effect on motor output¹⁷ while another does not¹⁸. These studies rely on comparing perceptual depth estimates to endpoint distance calculations of the hand relative to targets located on the Hollow Face Illusion. Conflicting results on actions performed on this type of stimuli may be a result of the variations in methods used by researchers. Because the way in which ventral and dorsal stream information is utilized is still up to debate, this controversy sparks the need for a more robust stimulus with additional advanced measures of motor behavior.

This is precisely why a technique was developed using reverse-perspective stimuli, commonly referred to as "reverspectives", which form another class of DIIs¹⁴. Linear perspective cues that are painted on piecewise 3D planar surfaces produce competition between the physical geometry of the stimulus and the actual painted scene. Data-driven sensory signals, such as binocular disparity and motion parallax favor the veridical percept of the physical geometry, whereas experience-based familiarity with perspective favors the depth-inversion percept (Figure 1). The advantage of the reverspective is that it allows for the placement of a target on a stimulus surface whose perceived spatial orientation under the illusion differs by nearly 90 degrees from its physical orientation (Figures 1e and 1f). This huge difference greatly facilitates testing whether reach-for-grasping movements are or are not influenced by the illusion. This notion is key to exploring whether or not motor actions performed on the reverspective are affected by top-down influences from the ventral stream.

Movement Classes in Perception-Action Models

If different motor strategies are employed under illusory and veridical percepts when grabbing towards a target on a reverspective stimulus, then it can be easily tracked by studying the curvature of the hand's approach. Moreover, an analysis of the entire unfolding movement from initiation of the goal-directed movement to the spontaneous, automatic retraction of the hand back to its resting state may in fact bypass any shortcomings found in past methods of testing for perceptual influence on motor output. Recent studies highlight the significance of studying the balance between these two movement classes as well as the use of the spontaneous segments by the nervous systems for predictive and anticipatory control^19-21,23-24. The newly statistically defined class of spontaneous-automatic movements provides new metrics and features that turn out to be as crucial as the goal-directed ones have been thus far to track sensory-motor changes and to quantify subtle aspects of natural behaviors.

To our knowledge, existing research on the visual two-streams hypothesis only focuses on goal-directed acts, thereby ignoring any effects on automatic transitional movements that are significant components to completing the visuomotor action loop. Emphasis therefore must be placed on the importance of automatic motions in order to fully capture both modes of motor behavior in the present paradigm to clarify issues concerning visual perception-action models. Here methods are developed to investigate the role of top-down signaling in the visual ventral stream on modulating motor behavior in the deliberate, goal-directed action domain in conjunction with spontaneous, transitional movements using a robust DII reverse-perspective stimulus.

Rationale

It is hypothesized that, if top-down visual processes influence the sensory-motor system, full movement trajectories toward the embedded target in the 3D reverse-perspective scene under the illusory percept will differ from the target approach elicited by the veridical percept (Figures 1e and 1f). Moreover, since the illusory percept of the reverspective stimulus is very similar to that obtained by a proper ("forced") perspective stimulus, reaches performed toward an embedded target on a reverspective should therefore be similar in characteristics to reaches conducted under the influence of the illusion on the reverspective stimulus (Figures 1c and 1f).

If top-down visual influences do not impact the movement trajectory, then it is hypothesized that reaches made under the illusory percept would exhibit the same characteristics as reaches made under the veridical percept on the reverspective stimulus (Figure 1e). In other words, both illusory and veridical percept reaches would be similar in nature, such that both forward trajectory paths would act on the true geometry of the stimulus. How effects observed in the forward reach translate in the automatic retraction of the hand is unknown. By employing a full motor analysis, we aim to advance our understanding of action and perception loops to clarify the existing issues at hand.

Protocol

1. Building the Stimulus Apparatus

1. Construct a moveable platform on a sliding track. Each stimulus will be placed on the moveable platform depending on the type of trial called for.
2. Secure the track onto a table at an appropriate height that allows for the stimulus platform to be at eye-level with the participant to be seated in front of the table.
3. Attach a retractable spring mechanism to the stimulus platform. Connect the input to the spring mechanism to a circuit board.
4. Place a set of lamps behind the participant's seat, facing the stimulus platform. It is important to illuminate the stimulus platform evenly because uneven lighting may cast shadows that interfere with the illusory percept. Connect the set of lamps to a converter that links it to the circuit board.
5. Attach a switch box to the edge of the table closest to the where the participant will be seated. Participants place their hand on the switch box at the beginning of each trial and activate the switch as soon as they lift their hand to execute the reach movement. Link the switch box input to the circuit board.
6. Connect each output pin of the circuit board to a pin on the microcontroller to control the simultaneous activation of the retraction of the moving platform via the spring mechanism and the turning off of lights once the switch box is triggered. The stimulus must retract and the lights must turn off after the initiation of the reach movement in each trial to prevent any online visual corrections and haptic feedback from occurring. The switch box is employed so that the stimulus retraction and darkness onset are performed only after movement begins, making this an immediate reach task.
7. Write a MATLAB program that controls the microcontroller signals. Use the MATLAB code to store a sequence of trials and instruct the experimenter what stimuli and viewing conditions to use for each trial.
8. Construct training stimuli, the reverse-perspective stimulus, and the proper-perspective stimulus (Figures 1 and 2). Training stimuli consist of two rectangular panels representing the isolated right surface wall of the middle building embedded in the reverse-perspective stimulus and the proper-perspective stimulus. The purpose of the training stimuli will be discussed in the experimental procedure. Affix red planar disk targets to the right of the midline of the stimuli.

2. Participants

1. Obtain written informed consent of the IRB approved protocol in compliance with the Declaration of Helsinki before beginning the experimental session.
2. Test the participant for visual acuity in each eye, stereopsis (using a Randot-Stereo Test), and eye dominance.
3. Set-up the motion capture system. Use fourteen electro-magnetic sensors at 240 Hz and motion-tracking software. The high-resolution recording system allows for the in-depth analysis of the unfolding of movement in three dimensions of fourteen sensors simultaneously, that past studies lack.
   1. Place twelve of the fourteen sensors on the following body segments using sports bands designed to optimize unrestricted movement of the body: head, trunk, right and left shoulders, left upper arm, left forearm, left wrist, right upper arm, right forearm, right wrist, right hand index finger, and right hand thumb.
   2. Place the remaining two sensors on the backside of the stimuli directly behind the target location to attain an accurate position of the target in 3D space relative to the participant during the training and experimental blocks.

3. Experimental Procedure

1. Place all stimuli out of view from the participant at this time. Turn off all lights except for the lamps used to illuminate the stimulus platform. Dim any computer screens that are in use to run the experiment so that their lights do not interfere with the even lighting projected onto the apparatus.
2. Before beginning any trials, inform the participant of the experiment flow. Notify them of the stimulus retraction and turning off of lights once they initiate movement by lifting their hand off the switch box. Remind them not to try to follow the retracting platform, but to only grab at where the target was last seen. Demonstrate how to grab at where they last remember seeing the target by approaching it normal to the perceived surface.
3. Begin practice trials. These trials allow for the participant to become comfortable with the setup. There is no test stimulus on the platform - only a black board with a center pole protrusion used to attach stimuli. Instruct the participant to reach at the center pole and to bring the hand back to rest upon completing the reach, at his/her own pace; repeat for three trials. Note: It is important not to give instructions on how to retract the hand; this component should be automatic and below conscious control.
4. Initiate training trials. Ask the participant to close his/her eyes after each trial for the remainder of the experiment. While the participant's eyes are closed, affix the training stimulus called for in the MATLAB program to the center pole; the order of training stimulus presentation is randomized by the MATLAB program for a total of eight trials, four for each stimulus. Training stimuli help demonstrate the curvature of the reach when asked to grab at targets on physical surfaces representative of the targets used in the experimental stimuli.
5. Begin experimental trials. There are three stimulus conditions for the experimental trials: (1) reverspective under illusory percept, as in Figure 1f (REV-ILLU), (2) reverspective under veridical percept, as in Figure 1e (REV-VER), and (3) proper-perspective (PRO), as in Figure 1c. Recall that conditions (1) and (2) utilize the same physical reverspective stimulus.
   1. First present the reverspective stimulus. Ask the participant if he/she can stabilize the illusory percept of the middle building "popping out" towards him/her. If the participant has trouble stabilizing the illusory percept, place a de-focusing lens on the nondominant eye to weaken stereopsis in order to preserve the illusory percept while maintaining reaching distance to the target¹⁸. If the participant requires the de-focusing lens, then make sure to instruct him/her to put them on before each REV-ILLU trial.
   2. After the first REV-ILLU trial, the MATLAB program will randomize the order of trials. For each trial, give the following instructions depending on the stimulus condition:
      REV-ILLU: "View the middle building as popping out towards you."
      REV-VER: "View the middle building as caving in away from you."
      PRO: "View the middle building as popping out towards you."
      Once the participant confirms a stable percept, ask them to grab at the target. Perform twelve trials for each condition for a total of 36 experimental trials.

4. Data Analysis

1. To analyze the movements in terms of the goal-directed reach and automatic retractions, first decompose the data into two movement classes by detecting the point at which the velocity of the movement, after its initiation, nears instantaneous zero velocity.
2. To look for differences in the curvature of hand path trajectories for each stimulus condition, perform the Wilk's Lambda Test Statistic on the 3-dimensional dataset at each point in time during the trajectory. The Wilk's Lambda Test reduces the likelihood test statistic Λ to a scalar value by way of determinants to help us deduce whether or not the mean trajectory vector for REV-ILLU is similar to REV-VER or PRO²².
3. To study the orientation of the hand towards the target at the end of the goal-directed reach, compare the angle formed between the unit approach vector generated by the thumb, index, and wrist sensor positions relative to the target's unit vector normal to the surface (Figures 5a and 5b).

Representative Results

1. Hand Path Trajectories

Results are shown for Representative Subject VT. The Wilk's Lambda Test Statistic allows for the reduction of our three-dimensional space data into a scalar value by the use of determinants. The Wilk's lambda statistic uses the likelihood ratio test , in which the 'within' sum of squares and products form matrix E, and the 'total' sum of squares and products form matrix (E+H). The rule states that, when , the null hypothesis is rejected. In , is the level of confidence, is the number of variables or dimensions, and and are the degrees of freedom for the hypothesis and error, respectively, in which is the number of conditions and is the number of trials. In our case, and . Therefore, we obtain from the look-up table found in Rencher's Methods for Multivariate Analysis²².

Hand path trajectory analysis using the Wilk's Lambda Test reveal a statistically significant difference between REV-ILLU and REV-VER conditions in the forward, goal-directed movement (Figure 3a), as throughout the entire path's progression (Figure 3d). This behavior is also preserved in the noninstructed retraction as seen in the graph (Figures 4a and 4d). As expected, the comparison between the REV-VER and PRO conditions differs significantly in both the forward and retractory movements (Figures 3b, 3e, 4b, and 4e). Since the unfolding of movement is critical in determining differences in approach, Wilk's lambda values are plotted based on the percentage of hand path trajectory complete (Figures 3d-3f and 4d-4f). Averaging these values over the entire path would not fully recapitulate the kinematics of the entire action loop. Wilk's lambda values for forward hand path trajectories in the REV-VER vs. REV-ILLU comparison (Figure 3d) are similar to those found in the REV-VER vs. PRO comparison (Figure 3e). The same holds true for the retraction of the hand (Figures 4d and 4e). The REV-ILLU and PRO conditions do not differ significantly in either movement class, as for all lambda values based on the percentage of path complete in both the forward and retraction cases (Figures 3c, 3f, 4c, and 4f).

2. Hand Orientation

When examining the orientation of the hand as it approaches the target in each condition, hand-approach vectors in REV-VER cases differ from those in the REV-ILLU and PRO cases (Figure 5c). REV-ILLU and PRO conditions produce similar hand postures when orienting towards the perceived target for REV-ILLU and the physical target for PROPER conditions. The angle formed between the mean unit approach vector for REV-ILLU trials and the unit vector normal to the target surface produces a 97.5197° ± 3.2228 difference (Figure 5d). Recall that the reverse-perspective stimulus generates nearly 90-degree maximal differences between illusory and veridical states. This therefore suggests that Representative Subject VT oriented her hand towards the perceived target and not the physical location of the target under the illusory percept.

Figure 1. The Proper- and Reverse- Perspective Stimuli. (a-c) The proper- or "forced" perspective. (a) Front view of the painted stimulus. (b) Orthographic views. (c) Top view: the veridical percept of a concave scene with an arrow illustrating a typical reach trajectory for the target. (a, d-f) The reverse-perspective gives rise to two percepts, shown in parts (e) and (f). (a) Front view of the painted stimulus. (d) Orthographic views. (e) Top view: the veridical percept of a convex scene with an arrow illustrating a typical reach trajectory. (f) Top view: the illusory percept of a concave scene – shown by dotted lines – with an arrow illustrating a typical reach trajectory. The dotted-line figure shows the perceived illusory 3D shape only. The position of the object is not accurate; in fact, the illusory object was deliberately offset toward the observer in order to clarify the reach trajectory. The curvature of all trajectories is exaggerated to illustrate the differences that might result, depending on the percept. The percept of (f) provides an excellent test for examining whether the reaching trajectory is governed by the illusion (trajectory of (f)) or by the physical surface (trajectory of (e)). Note that the proper- and reverse-perspectives share the same front view (a). Please click here to view a larger version of this figure.

Figure 2. Training Stimuli. (a-b) The rectangular panel has the same spatial orientation as the right wall of the middle building in the proper-perspective 3D stimulus. (c-d) The rectangular panel has the same orientation as the right wall of the middle building in the reverse-perspective 3D stimulus. (a, c) Schematic drawings of top views to illustrate the placement of the panels, with arrows indicating typical reach trajectories. The curvature of the trajectories is exaggerated to illustrate the difference. (b, d) Photographs of the stimuli as they appeared to participants. Please click here to view a larger version of this figure.

Figure 3. Forward Hand Path Trajectory Analysis. (a-c) Mean trajectories plotted in white with confidence intervals (colored tubes) for each point in the trajectory for reverse-perspective veridical (REV-VER in green), reverse-perspective illusory (REV-ILLU in blue), and proper-perspective (PROPER in red) conditions for the goal-directed, intended forward movement. (d-f) Lambda values for pairwise comparisons of conditions based on percentage of path complete. Using the Wilk's Lambda Test, when , the null hypothesis is rejected. is given by the dotted line. In (d) REV-VER vs. REV-ILLU and (e) REV-VER vs. PROPER comparisons, , indicating a significant difference between hand path trajectories. For the (f) REV-ILLU vs. PROPER .5comparison, , therefore hand path trajectories between conditions do not differ significantly. Please click here to view a larger version of this figure.

Figure 4. Retraction Hand Path Trajectory Analysis. (a-c) Mean trajectories plotted in white with confidence intervals (colored tubes) for each point in the trajectory for reverse-perspective veridical (REV-VER in green), reverse-perspective illusory (REV-ILLU in blue), and proper-perspective (PROPER in red) conditions for the spontaneous, automatic retraction of the hand. (d-f) Lambda values for pairwise comparisons of conditions based on percentage of path complete. Using the Wilk's Lambda Test, when , the null hypothesis is rejected. is given by the dotted line. In (d) REV-VER vs. REV-ILLU and (e) REV-VER vs. PROPER comparisons, , indicating a significant difference between hand path trajectories. For the (f) REV-ILLU vs. PROPER comparison, , therefore hand path trajectories between conditions do not differ significantly. Please click here to view a larger version of this figure.

Figure 5. Hand Orientation. (a) The unit approach vector of the hand is defined by sensors located on the thumb, index, and wrist positions. (b) Unit (dashed) vectors normal to the target surface for the proper-perspective (top) and reverse-perspective (middle and bottom) stimuli. Under the illusory percept (bottom panel, dotted lines) this vector is perceived as nearly perpendicular to the physical unit vector (middle). (c) Hand approach vectors plotted for REV-VER (green), REV-ILLU (blue), and PROPER (red) trials, (d) Mean angle values formed between the unit approach vector and the physical unit vector normal to the target surface corresponding to the stimulus used. Please click here to view a larger version of this figure.

Discussion

Our methods provide a platform to test the validity of perception-action models by analyzing the entire unfolding of movement in relation to the experimental task. The paradigm can be modified to test other types of visual stimuli to broaden this area of research. For example, other 3D DIIs can be tested on the apparatus to see how interactions between top-down and bottom-up processes translate to various stimuli. The methods can also be tailored to test clinical populations that may have perturbations in perception and action processes. Moreover, the motion capture system utilized in our study can be replaced with other types of recording equipment to best suit the experimental task. The possible generalization of these methods for other applications therefore holds significant value in the advancement of human behavioral research.

However, as with any technique, the current paradigm has its limitations. Because of the removal of haptic feedback and online visual control by turning off the lights and retracting the stimulus, the present study does not allow for the simultaneous recording of eye movements in conjunction with the execution of movement. Eye movements can help identify whether or not participants use an allocentric or egocentric frame of reference to employ a top-down or bottom-up strategy²⁵. Because the current design does not have the capability of implementing this additional measure, it is constrained to only capturing the body's kinematic function. Alternative strategies to remove haptic feedback and online visual control may be sought out to capture eye movement measures.

Besides this setback, the experimental design has several advantages over existing methods. Since past studies focused on the deliberate, goal-direction actions and end-point data, researchers overlooked any effects in the non-instructed, automatic retraction, and in the actual unfolding of the movement from initiation to rest. The protocol presented here takes into account both deliberate and automatic forms of movement to help build a better understanding of sensory-motor behavior under different perceptual states. Unlike other strategies, this paradigm focuses on both spatial and temporal effects to gain a full understanding of the visuomotor loop. Moreover, the strength of the reverspective stimulus used in this experiment trumps other DIIs used in the past (e.g. hollow-face illusion) as its configuration generates nearly 90° differences in perceived surface orientation under veridical and illusory states while remaining close enough to the participant for he/she to interact with it. This maximal difference aids in the disambiguation of the role of top-down processes on sensory-motor behavior.

Since the study of top-down influences on sensory-motor processes is important not just in the normative system, but also in clinical populations, this paradigm may prove to be a useful tool to study them. Future applications of this protocol may include tailoring the study for pathologies such as Schizophrenia (SZ). It is known that a certain subpopulation of patients with SZ exhibit a decrease in top-down functioning and have known issues in perceptual organization^26-28. Thus, understanding how this translates to the motor domain can advance our knowledge to develop better diagnostic tools and therapies for SZ.

This protocol was carefully designed to investigate the role of top-down processes on sensory-motor behavior, specifically when a participant is asked to reach at a target on a stimulus that produces multiple percepts. The critical steps within this protocol are in the selection of the stimuli and in the high resolution of motion capture from the initiation of movement back to resting state. Also, the powerful statistical analyses help elucidate whether or not the illusory percept influences motor strategies. Because this experimental design allows for the high resolution recording of natural intended and spontaneous motor behaviors, the analytical platform developed may help elucidate the existing issues in perception-action models that have long been debated. The preliminary results for Representative Subject VT illustrate this potential.

Materials

Name	Company	Catalog Number	Comments
Laboratory bench
Slidable Track with Retractable Spring			built in-house
Retractable Spring
Adjustable Lamps
Switch Box
Circuit Board
Arduino	Smart Projects, Italy
MATLAB	The MathWorks Inc., Natick, MA, USA
Randot-dot Stereo Test
Reverse-Perspective Stimulus			built in-house
Proper-Perspective Stimulus			built in-house
Training Stimuli			built in-house
Polhemus Motion Capture System	Liberty, Colchester, VT, USA
The Motion Monitor Motion-Tracking Software	Innovative Sports Training, Inc., Chicago, IL
Sport Sweatbands
De-Focusing Lens