Virtual Prism Adaptation Therapy: Protocol for Validation in Healthy Adults

Summary

This experimental protocol demonstrates the use of virtual prism adaptation therapy (VPAT) in healthy adults and the association between VPAT and functional near infrared spectroscopy to determine the effect of VPAT on cortical activation. Results suggest that VPAT may be feasible and could induce similar behavioral adaptation as conventional prism adaptation therapy.

Abstract

Hemispatial neglect is a common impairment after stroke. It is associated with poor functional and social outcomes. Therefore, an adequate intervention is imperative for the successful management of hemispatial neglect. However, the clinical use of various interventions is limited in real clinical practice. Prism adaptation therapy is one of the most evidence-based rehabilitation modalities to treat hemispatial neglect. To overcome any possible shortcoming that may occur with prism therapy, we developed a new system using immersive virtual reality and depth-sensing camera to create a virtual prism adaptation therapy (VPAT). To validate the VPAT system, we designed an experimental protocol investigating the behavioral errors and changes in cortical activation via the VPAT system. Cortical activation was measured by functional near infrared spectroscopy (fNIRS). The experiment consisted of four phases. All four included clicking, pointing or rest applied to right-handed healthy people. Clicking versus pointing was used for investigating the cortical region related with the gross motor task, and pointing with VPAT versus pointing without VPAT was used for investigating the cortical region associated with visuospatial perception. The preliminary results from four healthy participants showed that pointing errors by the VPAT system was similar to the conventional prism adaptation therapy. Further analysis with more participants and fNIRS data, as well as a study in patients with stroke may be required.

Introduction

Hemispatial neglect, which affects the ability to perceive the contralateral hemispatial visual field, is a common impairment after stroke^1,2. Although rehabilitation after hemispatial neglect is important, due to its association with poor functional and social outcomes, rehabilitation is often underutilized in real clinical practice^3,4.

Among the various existing rehabilitation approaches suggested for hemispatial neglect, prism adaptation (PA) therapy has proven effective for recovery and improvement in hemispatial neglect in patients with subacute or chronic stroke^5,6,7,8. However, conventional PA is underutilized due to several drawbacks^9,10. These include 1) high cost and time requirement due to the prism lens needing to be changed to adjust to the degree of deviation; 2) the need to set up additional materials to be pointed at and to mask the hand trajectory; and 3) PA can only be used by patients who can sit and control their head position.

A recent study reproducing the adaptation effects in the virtual reality (VR) environment reported that it may be possible for the virtual prism adaptation therapy (VPAT) to have different effects depending on the subtypes of neglect¹¹. It was also suggested that cortical activation for PA may vary according to brain lesions¹². However, little is known about the cortical activation pattern seen in VR-induced PA.

To overcome these obstacles and promote the use of PA in a clinical setting, we developed a new PA therapy system using an immersive VR technology called virtual prism adaptation therapy (VPAT), via the use of a depth-sensing camera. We designed an immersive VR system with the ability to provide visual feedback about the position of a virtual limb to promote spatial realignment¹³. Using this immersive VR technology, which mimicked the effect of conventional PA, we designed an experiment to validate the VPAT system in healthy participants.

By conducting our visualized experimental protocol, we investigated whether the new VPAT system can induce behavioral adaptation, similar to conventional PA. Additionally, we would like to explore whether the VPAT system can induce the activation in the cortical regions associated with visuospatial perception or recovery of hemispatial neglect after stroke.

Protocol

All procedures were reviewed and approved by the Seoul National University Bundang Hospital Institutional Review Board (IRB). To recruit healthy participants, posters were used to advertise around the hospital.

1. Experimental set-up

1. Participant recruitment
   1. Perform the subject screening process using the following inclusion criteria: 1) healthy, between 18 and 50 years old; 2) right-handed, assessed by Edinburgh handedness inventory¹⁴; 3) able to wear the head mount display for VR and to detect objects within VR; and 4) no history of diseases affecting the brain, such as stroke, Parkinson's disease, or traumatic brain injury.
      NOTE: These criteria were designed to screen participants with the ability to participate in the experiment and regulate factors affecting the results.
   2. Recruit participants and provide a detailed explanation of the entire study and expected clinical issues. Consent must be obtained prior to inclusion.
2. Experimental system
   NOTE: A customized VPAT system using an immersive VR system and depth-sensing camera was used. Functional infrared spectroscopy (fNIRS) was simultaneously used to investigate the cortical activation. VPAT and fNIRS were linked together for the experiment (Figure 1).
   1. VPAT system
      NOTE: The VPAT system consists of a head mount display for VR implementation, a hand tracking sensor that can recognize hand gestures for intuitive input by the user, and a hardware push button. The overall composition is shown in Figure 1.
      1. Make sure that the hand tracking sensor is not tilted in front of the head mount display.
      2. Check that the reference camera for the VR system is properly installed on top of the front monitor.
      3. Secure the push button in a location near the hand to be used by the participant for the experiment.
      4. Run the software to make sure there are no errors.
         NOTE: The virtual environment was implemented to match the actual environment as close as possible. The task was performed through hand pointing within the virtual environment and button input through the push button.
   2. fNIRS
      1. Use a commercial fNIRS system including a personal computer (PC), 31 optodes (15 light sources and 16 detectors), textile EEG caps, and data recording software.
   3. Linkage between VPAT system and fNIRS (Figure 1).
      1. Use the remote keyboard control software using TCP/IP communication to synchronize the starting event in the VPAT system with the timing of recording in the fNIRS system.
      2. Use the remote command key in the computer to start fNIRS recording.

2. Experimental set-up (Figure 2)

1. fNIRS measurement setting
   1. Place the participant in a chair with his/her back in a straight posture, about fifteen centimeters away from the table. Confirm that the participant's hand does not hit the table when reaching out.
   2. For fNIRS cap setting, select the cap size according to the participant's head circumference. Place the cap so that the vertex (Cz) is located at the intersection of the midpoint between the inion and nasion and the midpoint between the left preauricular and right preauricular areas. Display the montage on the screen and connect 15 sources and 24 detectors to the montage. If necessary to improve the gain from the light source, use conductive gel after hair preparation and insert the optode. Have the participant wear a retaining cap.
      NOTE: The study used three different sizes of textile EEG caps with circumferences of 54, 56, and 58 cm.
   3. For software setting (calibration, etc.), run the fNIRS system software and load the neglect montage.
   4. Let the montage be displayed on the screen and set 15 sources and 24 detectors according to the montage (Figure 3).
   5. Press the calibrate button. If "Lost" is displayed on the screen, repeat the hair preparation, and then recalibrate.
2. VPAT system setting
   1. Connect the HMD, reference camera, and Leap motion camera, and push the button connecting the computer to set up the VPAT system.
   2. Mount the virtual reality head-mounted display (VR HMD) on the participant's head over the cap for fNIRS. Make sure to avoid movement of the cap.
   3. Run the VPAT software. Enter the participant's information (name abbreviation, age, handedness) and press the "Start" button.
   4. Confirm the visualization of the virtual hand in the display. Proceed with a two-step calibration (i.e., screen calibration and target distance calibration).
   5. Instruct the participant to watch the red cross mark (+) in the center, then press the "r" key to calibrate the screen.
      NOTE: Screen calibration places the virtual space in front of the user's visual range by recentering the coordinate system.
   6. Instruct the participant to point to the target (i.e., ball) with his or her right hand, then press the "O" key to calibrate the hand position.
      NOTE: In our study, the object that the participant had to target was a white ball on a pink stick that came down from the top of the view. Target distance calibration places the target within the reach of the user. This is used to correctly position the target during the experiment.
   7. After calibration, press the "w" key to begin the experiment.
3. VPAT and fNIRS linkage setting
   1. Use the event synchronization software to enter the trigger for analysis into fNIRS and connect VPAT to fNIRS.
   2. For time synchronization between VPAT and fNIRS, connect the computers with the two systems to the same network, and then synchronize them through the self-produced key transferring program.
   3. After connecting through the IP and Port inputs of both computers, start the experiment session via the "w" key in the VPAT program. The event synchronization software is executed automatically, and triggers during execution are automatically transferred to fNIRS and saved.
   4. After the experiment, obtain the software auto-termination and VPAT data. Then stop the VPAT and fNIRS system software.
      NOTE: The participants must return their hands to their original position after pointing during the VPAT experiment.

3. Experiment to validate VPAT system

1. Block designed experiment with fNIRS recording (Figure 4)
   1. After completing the set-up process in step 2, confirm the participant's readiness to start the experiment.
   2. Start the VPAT system without the prism mode and instruct the participant to point to the target in the VR system immediately for familiarization with the procedure.
   3. Each phase consists of blocks for pointing, clicking, or resting (Figure 4). Again, instruct the participant to click on the button or point to the target in the VR system with their right index finger as quickly as possible.
   4. Start the experiment with four phases simultaneously with fNIRS recording by clicking the start key.
      NOTE: During the pointing task, the white ball had to be touched within a fixed time.
      1. Instruct the participants to point, click, or rest when the appropriate icon appears.
         NOTE: During the task, pointing and clicking were indicated by an icon directly above the white ball and right side of the timer bar. The time to perform the task was indicated by the timer bar as shown in Figure 2.
      2. Tell the participant to touch the target that appears on the left or right side within 3 s. For the clicking block, instruct the participant to press the push button.
         NOTE: The target set containing the white ball was located at a distance of -10° or 10° from the participant's center, obtained by calibration. The target set appeared randomly on the right or left side. According to the experimental design, the target appeared for 3 s, then disappeared, and then regenerated to a new position.
      3. Ensure that the participant performs the same way when the phase is switched.
         NOTE: In the pointing task, Virtual Prism Adaptation Mode showed a deviation of 10° or 20° to the left side of the imaginary hand in the VR space relative to the participant's head. Zero degrees indicated that the positions of the virtual hand and the actual hand coincided.
         NOTE: The experiment (Figure 4) consists of a total of four phases, with each phase consisting of pointing and clicking or rest alternately (Phase 1 and 4 were pointing and clicking, and phase 2 and 3 were pointing and resting).

4. Data analysis

1. Pointing error analysis
   NOTE: The data were stored from the moment the experimenter pressed the start button "w". The data were automatically stored at about 60 Hz every frame through the VPAT software. The phase name, elapsed time, and virtual index finger position were stored over time. The pointing error was the angle value between the target and index finger, centered on the participant's head position.
   1. Classify the pointing task data by phases (pre-VPAT, VPAT 10°, VPAT 20°, post-VPAT).
   2. Classify the data of the pointing task and the clicking task in the data of each phase (phase 1 and 4).
   3. Classify the data by sub-phase in units of 30 s according to each phase and each type of task.
   4. Extract the median value of 10 trial error (pointing error) values from the index finger position data for median pointing error analysis.
   5. Use the repeated measures analysis of variance test (ANOVA) to analyze the difference between each phase.
      NOTE: In the case of hand tracking using the Leap motion sensor, outliers were due to occlusion or false detection of the hand posture. With the exception of false hand position data, the median value was used to find the representative pointing error value in the sub-phase.
2. fNIRS data processing
   1. Launch the fNIRS analysis software and load the raw data file and probe information.
   2. Perform a marker setting process by editing the event record to verify each condition during the experiment.
   3. Carry out data preprocessing by deleting the experimentally irrelevant time intervals, remove artifacts, such as steps and spikes, and apply frequency filters to exclude experimentally irrelevant frequency bands.
      NOTE: All data sets were filtered with a 0.01 Hz high-pass filter and a 0.2 Hz low-pass filter to remove instrumental or physiological noise contributions.
   4. Specify wavelengths by entering the value of the peak illumination wavelengths (i.e., 760 and 850 nm). Use a physical distance of 3 cm between the source and detector for channel.
   5. Select the baseline field, which refers to the time period that corresponds to a baseline wherein participants are typically resting quietly.
      NOTE: We selected the baseline field as the full-time course of the data set, which was the default setting.
   6. Compute the time series of hemodynamic states to finish the preprocessing from the filtered data.

Results

Data from four healthy participants (1 man and 3 women) were used as representative results. A pointing error is shown in Figure 5A, with the averages of median value of 10 trials in the sub-phase of each pointing task lasting 30 s. Values on average for the median pointing errors in the first block of each phase were 0.45 ± 0.92 (pre-VPAT), 4.69 ± 3.08 (VPAT 10°), 5.43 ± 2.22 (VPAT 20°), and -5.17 ± 1.60 (post-VPAT). The trend of pointing error change was statistically sign...

Discussion

This study implemented the prism adaption therapy using a translated hand movement in a VR environment. It investigated whether the deviation implemented was causing angle overshooting and behavioral adaptation, as in conventional prism adaption therapy.

In the median pointing error result (Figure 5) and the first pointing error result, the pointing error changed significantly when the phase was switched. Although some hand-recognition errors were eliminated, ther...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
EASYCAP	Easycap	C-SAMS	Platform to accommodate fNIRS optodes
Leap Motion 3D Motion Controller	Ultrahaptics	FBA_LM-C01-US	Hand detection device attached HMD
Leap Motion VR Developer Mount for VR Headset	Ultrahaptics	VR-UAZ
Matlab R2015a	Mathworks		Programming language running with NIRStar
NIRScout	Medical Technology LLC	NSC-CORE	fNIRS system
nirsLAB v201605	Medical Technology LLC		Software for analyzing data collected with NIRScout
NIRStar 14.1	Medical Technology LLC		NIRScout Acquisition Software
Occulus Rift DK2	Occulus		VR HMD
PowerMate USB Multimedia Controller	Griffin Technology	NA16029	Push Button in task
SuperLab 5.0	Cedrus Corp.		Synchronize the stimulus presentations allied to NIRScout