Efficiently Recording the Eye-Hand Coordination to Incoordination Spectrum

Summary

Cerebral injury can damage both ocular and somatic motor systems. Characterization of motor control post-injury affords biomarkers that assist in disease detection, monitoring, and prognosis. We review a method to measure eye-hand movement control in health and in pathologic incoordination, with look-and-reach paradigms to assess coordination between eye and hand.

Abstract

The objective analysis of eye movements has a significant history and has been long proven to be an important research tool in the setting of brain injury. Quantitative recordings have a strong capacity to screen diagnostically. Concurrent examinations of the eye and upper limb movements directed toward shared functional goals (e.g., eye-hand coordination) serve as an additional robust biomarker-laden path to capture and interrogate neural injury, including acquired brain injury (ABI). While quantitative dual-effector recordings in 3-D afford ample opportunities within ocular-manual motor investigations in the setting of ABI, the feasibility of such dual recordings for both eye and hand is challenging in pathological settings, particularly when approached with research-grade rigor. Here we describe the integration of an eye tracking system with a motion tracking system intended primarily for limb control research to study a natural behavior. The protocol enables the investigation of unrestricted, three-dimensional (3D) eye-hand coordination tasks. More specifically, we review a method to assess eye-hand coordination in visually guided saccade-to-reach tasks in subjects with chronic middle cerebral artery (MCA) stroke and compare them to healthy controls. Special attention is paid to the specific eye- and limb-tracking system properties in order to obtain high fidelity data from participants post-injury. Sampling rate, accuracy, permissible head movement range given anticipated tolerance and the feasibility of use were several of the critical properties considered when selecting an eye tracker and an approach. The limb tracker was selected based on a similar rubric but included the need for 3-D recording, dynamic interaction and a miniaturized physical footprint. The quantitative data provided by this method and the overall approach when executed correctly has tremendous potential to further refine our mechanistic understanding of eye-hand control and help inform feasible diagnostic and pragmatic interventions within the neurological and rehabilitative practice. 

Introduction

A critical element of the neurological function is eye-hand coordination or the integration of ocular and manual motor systems for the planning and execution of combined function towards a shared goal, for example, a look, reach and grab of the television remote. Many purposeful tasks depend on visually guided actions, such as reaching, grasping, object manipulation and tool use, which hinge on the temporally and spatially coupled eye and hand movements. Acquired brain injuries (ABI) cause not only limb dysfunction but also ocular dysfunction; more recently, there is also evidence pointing to the dysfunction of eye-hand coordination¹. Coordinated eye-hand motor control programs are susceptible to insult in neurological injuries from vascular, traumatic and degenerative etiologies. These insults may cause a breakdown between any of the indispensable relationships needed for the integrated and rapid motor control^2,3,4,5,6. Many studies on the manual motor function have been completed and have leveraged visual guidance as a core pillar of the paradigm without a method or protocol in place to analyze eye movements concurrently.

In ABI, conspicuous motor deficits are often detected during the bedside clinical examination. However, concurrent ocular motor impairments and complex impairments involving the integration of sensory and motor systems may be subclinical and necessitate objective recording to be identified^{7,8,9,10,11,12,13,14,15,16}. Ocular-manual motor coordination depends on a large and interconnected cerebral network, highlighting the need for a detailed study. An eye-hand coordination evaluation with dual objective recordings provides an opportunity to assay both cognitive and motor function in multiple populations, including healthy controls and subjects with a history of brain injury, thus providing insight into cerebral circuitry and function³.

While saccades are ballistic movements that can vary in amplitude depending on task need, studies have shown dependencies between saccade and hand movement during visually guided action^17,18,19,20. In fact, recent experiments have demonstrated that control systems for both movements share planning resources^21,22. The motor planning hub for eye-hand coordination lies in the posterior parietal cortex. In a stroke, there are well-known deficits in motor control; hemiparetic patients have been shown to generate inaccurate predictions given a set of neural commands, when asked to perform visually guided hand movements, using either the more affected (contralateral) or less affected (ipsilateral) limb^{23,24,25,26,27,28,29}. Furthermore, eye-hand coordination and related motor control programs are susceptible to insult following neurological injuries, decoupling the relationships, temporally and spatially, between effectors³⁰. Objective recordings of eye and hand control are paramount to characterizing the incoordination or degree of coordination impairment and improves the scientific understanding of eye-hand motor control mechanism in a functional context.

Although there are many studies of eye-hand coordination in healthy controls^{17,31,32,33,34}, our group has advanced the field by our setting of neurological injury, for instance during stroke circuitry assessment, have investigated the spatial and temporal organization of hand movements, often in response to visually displayed spatial targets. Studies that have expanded the objective characterization to eye and hand have almost exclusively focused on the performance capacity to record both effectors post-stroke or in pathologic settings; the described protocol enables robust characterization of ocular and manual motor control in unconstrained and natural movements. Here we describe the technique in an investigation of visually-guided saccade-to-reach movements in subjects with chronic middle cerebral artery (MCA) stroke relative to healthy controls. For the simultaneous recording of saccade and reach, we employ concurrent eye and hand motion tracking.

Protocol

1. Participant

1. Recruit control participants older than 18 years, without a history of neurological dysfunction, significant eye injury, significant depression, major disability and/or electrical implants.
2. Recruit Stroke participants older than 18 years, with a history of brain injury in the middle cerebral artery (MCA) distribution, have the ability to complete the Fugl-Meyer Scale, maintain a full range of eye movements^35,36, have the ability to perform pointing tasks, and without the history of additional neurological dysfunction, significant eye health comorbidity, significant depression, major disability and/or electrical implants.
3. Ask participants to sign a consent form approved by the Institutional Review Board of New York University’s School of Medicine.
4. Participant Screening (for detailed exclusion criteria please see Rizzo et al³⁷)
   1. Take history and perform clinical examinations as discussed below.
      1. Assess the cognitive state of participants with Mini Mental State Examination (MMSE)³⁸.
      2. Perform neurological examination.
      3. Examine extraocular muscles and eye movements.
         1. Ask the participants to follow the researcher’s finger with their eyes while keeping their head in one position. Draw an imaginary H letter in front of them and make sure that your finger moves far enough out and up/down, assessing center, up, down, left, right, down/left, down/right, up/left, and up/right.
         2. Ask the participants to follow and maintain the gaze on an object moved slowly through their visual field to assess smooth pursuit. Cover a distance of approximately 24 inches and using a pencil as a target, sweep back and forth slowly in horizontal and vertical directions, repeating each three times.
         3. Ask participants to look as fast as possible between 2 targets that are placed 24 inches apart to assess saccades. Use a pencil and a pen as targets and direct gaze to the targets in a back and forth manner three times horizontally and vertically.
         4. Ask the participants to fixate on an object as it moves slowly towards to their eyes to assess convergence, centering the target, a pencil, on the bridge of their nose. Following this procedure, repeat the test by bringing the same target from the nose back out to the starting position (divergence).
         5. Ask the patient to cover one eye and look at the researcher’s nose. Move the hand out of the patient’s visual field and then bring it in, wag the finger slowly and ask the patient to let the researcher know when the hand comes back into view, repeat this for upper left, upper right, lower left, and lower right quadrants.
            NOTE: When the patient covers their right eye, cover the left eye, and vice versa.
      4. Assess the visual impairment by a visual-motor Integration test.
      5. Assess the visual acuity by Snellen chart^39,40.
      6. Assess the visual field with confrontation and if in question, perform Goldman or Humphrey visual field testing^41,42.
      7. Assess hemi-spatial neglect via line bisection test and the single letter cancellation test⁴³.
      8. Quantify the extent of disability via 25-item National Eye Institute Visual Functioning Questionnaire (NEI-VFQ-25) and a 10-item supplement survey⁴⁴.

2. Preparation for the experiment and the physical configuration of equipment

1. Equipment:
   1. Choose an eye tracker
      1. Choose an eye tracker that is capable of head-mounted use (to avoid interference with desk-based reach movements) high spatial resolution (≤0.1^o) and high temporal resolution (≥250 Hz).
      2. Record the binocular eye movement with the eye tracker at a sampling rate of 250 Hz (sampling eye position every 4 ms) tracking both pupil and corneal reflection.
   2. Choose of a limb tracker
      1. Choose a limb tracker that can map the movement in the x, y, z position, ³ 0.08 cm accuracy, Latency ³ of 3.5 ms.
   3. Choose a laptop capable of running a customized script that controls real-time integration of data acquired from two systems and co-registering the signals in real-time (Table of Materials).
   4. Choose a display monitor capable of integrating with the chosen laptop and that is large enough to support one-to-one correspondence between monitor and tabletop reach space
   5. Define a rectangle identical in size to the display monitor on a table surface between the participant and the display monitor, to use as a functional reaching space for experimental work.
2. Set up preparation:
   1. Set up a table with the height adjustable chair.
   2. Place a display monitor 40 cm from the far edge of the table (Table of Materials).
   3. Place a tabletop board (reaching surface) with the 1-1 ratio dimension with the display monitor.
   4. Set up the limb tracker by mounting the electromagnetic source under the table (Table of Materials).
   5. Set up the eye tracker, host PC (Table of Materials).
      1. Attach four infrared (IR) illuminators to four corners of the monitor using straps.
      2. Set the eye tracker configurations from eye tracker setup options screen.
         1. Select 13-point calibration from the pre-set configuration of the eye tracker.
         2. Select the high saccade sensitivity to detect small saccade.
         3. Select Pupil-CR mode to record both pupils and cornea.
         4. Select a sampling rate at 250 Hz.
3. Participant physical preparation
   1. Seat participants on a height-adjustable chair at the table with the computer display.
   2. Position the participant 60 cm away from the display monitor (Table of Materials).
   3. Fix the motion sensor (Table of Material) to the distal aspect of the index finger of the hand of the to be tested arm (dominant arms for controls, and both arms in participants with stroke)
   4. Place the eye tracker on the participants’ headband and adjust the headband and cameras (Table of Materials).
      1. Fitting the headband
         1. Adjust the tightness and position of the headband (using headband knobs) so that the front pad is in the center of the forehead and the side pads above the participant’s ears.
         2. Make sure that the headband camera is in the center of the forehead and over the bridge of the nose.
         3. Ask participants to raise their eyebrows, and if the headband moves, refit it higher or lower on the forehead.
      2. Adjust the camera and corneal illuminator position. Ask the participants to look at the display monitor.
         1. From the camera screen, select the head camera image, verify that it shows four large spots from the IR markers that are positioned in the center of the head camera image. If they are not in the center, adjust accordingly.
         2. From the camera setup screen, select one eye at the time. Adjust the two eye cameras by lowering and raising the eye camera handle till the pupil of the eye is in the center of the camera image
         3. Focus the eye camera by rotating the lens holder.
         4. Set the pupil threshold by pressing the Auto Threshold button on the camera setup screen.
         5. Perform the same adjustment for the other eye.
4. Calibration
   1. Calibrate the limb tracker output to reaching surface using a 9-point calibration, ask the participant to place their sensor attached finger on reaching surface (tabletop) locations as displayed on the monitor screen.
   2. Calibrate the eye tracker, ask participants to look at the calibration target that appears as a blue dot and maintain fixation until the next dot appears on the screen
      NOTE: Calibration targets appear in 13 randomly selected positions on the screen
   3. Calibrate the eye tracker at least twice per session, first one at the start of the experiment and at its halfway point.

3. Experiment

1. Ask participants to move their finger onto the start position, covering the start circle on the screen with the finger-indicator dot (red dot), while fixating (eye) the start position on the screen.
   NOTE: The start position is a correspondent location of the fixation point (blue dot) displays on the center of the screen (Figure 1a). The position of the finger is represented as 4 mm radius red dot on the screen.
2. Require participants to maintain finger position on the start circle for 150 ms until the target appears.
3. Ensure that participants fixate the start position until they hear a beep sound (“go beep”). (Figure 1)
   NOTE: The duration between target appearance and the go signal is randomized between 250 to 750 ms to prevent anticipation of the go signal.
4. Instruct participants to move both their eyes and fingertip quickly and accurately to the designated target as they hear the beep sound (Figure 1)
   1. Designated target appears 1 cm radius white circle
5. Instruct participants to touch the tabletop location at the position of the virtual target as displayed on the screen by lifting the hand and finger and re-connecting the fingertip and tabletop
   1. Make sure participants make a pointing movement by lifting the hand and finger rather than dragging the hand and finger on the tabletop.
   2. Display the end location of the reach as a red dot, following reach completion.
   3. Determine reach completion by a combination of low-velocity (<5% peak) and 3 mm z-plane threshold.
6. Ask participants to perform a series of familiarization trials before starting data acquisition.
7. Start data acquisition after participants touched 5 of the last 10 targets successfully.
8. Ask participants to perform a series of look and reach trials as they were instructed during familiarization trials.
   1. Have participants perform a total of 76 trials.
9. Have control participants perform the experiment with their dominant hand.
10. Whenever it is possible, have participants with stroke perform the experiment with both the hands, more-affected and less-affected.
11. Participants complete the entire experiment with at least one hand.

Figure 1. Schematic view of setup and experiment. (a) Schematic representation of display monitor and reaching surface during a trial. (b) Sequencing of actions within visually-guided reach. First Fixation (F) appears. The target (T) appears after a randomized length of time. The ‘go’ signal occurs as auditory beep sound (signified by the light-grey vertical bar) after an unpredictable time interval (concurrent offset of F) following by target appearance. Hand (H) and Eye (E) movements follow the go-signal. Please click here to view a larger version of this figure.

Results

Thirty participants participated in the research study. There were 17 participants in the control cohort, and 13 participants in the stroke cohort. Two participants couldn’t finish the whole experiment, so their data were excluded from the analysis.

Demographics and Questionnaire Assessments

Table 1 shows the clinical and demographic c...

Discussion

The advent of eye and hand tracking systems as available tools for objectively exploring the characteristics of ocular-manual motor systems has accelerated research studies, enabling a nuanced recording approach for an essential task in daily activities – eye-hand coordination. Many natural action-dependent tasks are visually guided and depend on vision as a primary sensory input. Gaze is programmed through ocular motor commands which point central vision at key spatial targets; this information is pivotal and assi...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
27.0" Dell LED-Lit monitor	Dell	S2716DG	QHD resolution (2560 x 1440)
ASUS ROG G750JM 17-Inch	AsusTek Computer Inc
Eye Link II	SR-Research	500 Hz binocular eye monitoring 0.01 º RMS resolutions
Matlab	MathWorks
Polhemus MicroSensor 1.8	Polhemus		240 Hz, 0.08 cm accuracy