A Human-machine-interface Integrating Low-cost Sensors with a Neuromuscular Electrical Stimulation System for Post-stroke Balance Rehabilitation

Summary

A novel low-cost human-machine interface for interactive post-stroke balance rehabilitation system is presented in this article. The system integrates off-the-shelf low-cost sensors towards volitionally driven electrotherapy paradigm. The proof-of-concept software interface is demonstrated on healthy volunteers.

Abstract

A stroke is caused when an artery carrying blood from heart to an area in the brain bursts or a clot obstructs the blood flow to brain thereby preventing delivery of oxygen and nutrients. About half of the stroke survivors are left with some degree of disability. Innovative methodologies for restorative neurorehabilitation are urgently required to reduce long-term disability. The ability of the nervous system to reorganize its structure, function and connections as a response to intrinsic or extrinsic stimuli is called neuroplasticity. Neuroplasticity is involved in post-stroke functional disturbances, but also in rehabilitation. Beneficial neuroplastic changes may be facilitated with non-invasive electrotherapy, such as neuromuscular electrical stimulation (NMES) and sensory electrical stimulation (SES). NMES involves coordinated electrical stimulation of motor nerves and muscles to activate them with continuous short pulses of electrical current while SES involves stimulation of sensory nerves with electrical current resulting in sensations that vary from barely perceivable to highly unpleasant. Here, active cortical participation in rehabilitation procedures may be facilitated by driving the non-invasive electrotherapy with biosignals (electromyogram (EMG), electroencephalogram (EEG), electrooculogram (EOG)) that represent simultaneous active perception and volitional effort. To achieve this in a resource-poor setting, e.g., in low- and middle-income countries, we present a low-cost human-machine-interface (HMI) by leveraging recent advances in off-the-shelf video game sensor technology. In this paper, we discuss the open-source software interface that integrates low-cost off-the-shelf sensors for visual-auditory biofeedback with non-invasive electrotherapy to assist postural control during balance rehabilitation. We demonstrate the proof-of-concept on healthy volunteers.

Introduction

An episode of neurological dysfunction caused by focal cerebral, spinal, or retinal infarction is called stroke¹. Stroke is a global health problem and fourth leading cause of disability worldwide¹. In countries like India and China, the two most populous nations of the world, neurologic disability due to stroke is being labeled as hidden epidemic². One of the most common medical complications after a stroke are falls with a reported incidence of up to 73% in the first year post-stroke³. The post-stroke fall is multifactorial and includes both spinal and supraspinal factors like balance and visuospatial neglect⁴. A review by Geurts and colleagues⁵ identified 1) multi-directionally impaired maximal weight shifting during bipedal standing, 2) slow speed, 3) directional imprecision, and 4) small amplitudes of single and cyclic sub-maximal frontal plane weight shifts as the balance factors for fall risk. The consequent impact on activities of daily living can be significant since prior works have shown that balance is associated with ambulatory ability and independence in gross motor function^5,6. Moreover, Geurts and colleagues⁵ suggested that supraspinal multisensory integration (and muscle coordination⁷) in addition to muscle strength is critical for balance recovery which is lacking in current protocols. Towards multisensory integration, our hypothesis⁸ on volitionally driven non-invasive electrotherapy (NMES/SES) is that this adaptive behavior can be shaped and facilitated by modulating active perception of sensory inputs during NMES/SES-assisted movement of the affected limb such that the brain can incorporate this feedback into subsequent movement output by recruiting alternate motor pathways⁹, if needed.

To achieve volitionally driven NMES/SES-assisted balance training in a resource-poor setting, a low-cost human-machine-interface (HMI) was developed by leveraging available open-source software and recent advances in off-the-shelf video game sensor technology for visual-auditory biofeedback. NMES involves coordinated electrical stimulation of nerves and muscles that has been shown to improve muscle strength and reduce spasticity¹⁰. Also, SES involves stimulation of sensory nerves with electrical current to evoke sensations where preliminary published work¹¹ showed that subsensory stimulation applied over the tibialis anterior muscles alone is effective in attenuating postural sway. Here, the HMI will make possible sensory-motor integration during interactive post-stroke balance therapy where volitionally-driven NMES/SES for the ankle muscles will act as a muscle amplifier (with NMES) as well as enhance afferent feedback (with SES) to assist healthy ankle strategies^12,13,14 to maintain upright stance during postural sways. This is based on the hypothesis presented in Dutta et al.⁸ that an increased corticospinal excitability of relevant ankle muscles effected through non-invasive electrotherapy may lend to an improved supraspinal modulation of ankle stiffness. Indeed, prior work has shown that NMES/SES elicits lasting changes in corticospinal excitability, possibly as a result of co-activating motor and sensory fibers^15,16. Moreover, Khaslavskaia and Sinkjaer¹⁷ showed in humans that concurrent motor cortical drive present at the time of NMES/SES enhanced motor cortical excitability. Therefore, volitionally-driven NMES/SES may induce short-term neuroplasticity in spinal reflexes (e.g., reciprocal Ia inhibition¹⁷) where corticospinal neurons that project via descending pathways to a given motoneuron pool can inhibit the antagonistic motoneuron pool via Ia-inhibitory interneurons in humans¹⁸, as shown in Figure 1, towards an operant conditioning paradigm (see Dutta et al.⁸).

Figure 1: The concept (details at Dutta et al.²¹) underlying interactive human machine interface (HMI) to drive the center of pressure (CoP) cursor to the cued target to improve ankle muscle coordination under volitionally driven neuromuscular electrical stimulation (NMES)-assisted visuomotor balance therapy. EEG: electroencephalography, MN: α-motoneuron, IN: Ia-inhibitory interneuron, EMG: electromyogram, DRG: dorsal root ganglion. Reproduced from⁸ and³⁷​. Please click here to view a larger version of this figure.

The antero-posterior (A-P) displacements in center of mass (CoM) are performed by ankle plantarflexors (such as medial gastrocnemius and soleus muscles) and dorsiflexors (such as the anterior tibial muscle) while medio-lateral (M-L) displacements are performed by ankle invertors (such as the anterior tibial muscle) and evertors (such as peroneus longus and brevis muscles). Consequently, stroke-related ankle impairments including weakness of the ankle dorsiflexor muscles and increased spasticity of the ankle plantarflexor muscles lead to impaired postural control. Here, agility training programs⁶ can be leveraged in a virtual reality (VR) based gaming platform that challenge dynamic balance where tasks are progressively increased in difficulty which may be more effective than static stretching/weight-shifting exercise program in preventing falls⁶. For example, subjects can perform volitionally driven NMES/SES assisted A-P and M-L displacements during a dynamic visuomotor balance task where the difficulty can be progressively increased to ameliorate post-stroke ankle-specific control problems in weight shifting during bipedal standing. Towards volitionally driven NMES/SES assisted balance therapy in a resource-poor setting, we present a low-cost HMI for Mobile Brain/Body Imaging (MoBI)¹⁹, towards visual-auditory biofeedback which can also be used for data collection from low-cost sensors for offline data exploration in MoBILAB (see Ojeda et al.²⁰).

Protocol

Note: The HMI software pipeline was developed based on freely available open-source software and off-the-shelf low-cost video game sensors (details available at: https://team.inria.fr/nphys4nrehab/software/ and https://github.com/NeuroPhys4NeuroRehab/JoVE). The HMI software pipeline is provided for data collection during a modified functional reach task (mFRT)²¹ in a VR based gaming platform for visuomotor balance therapy (VBT)⁸.

Figure 2a shows the diagnostic eye tracker setup where the gaze features are extracted offline for the quantification of post-stroke residual function so that the visual feedback in VR can be customized accordingly.

Figure 2b shows the experimental setup for VBT.

Figure 2: (a) Schematic of the human-machine-interface for the evaluation of post-stroke pursuit eye movements. (b) Schematic of the human-machine-interface where the software interface integrates biosignal sensors and motion capture to record mobile brain/body imaging data with neuromuscular electrical stimulation system (NMES) and sensory electrical stimulation (SES) for post-stroke NMES/SES-assisted visuomotor balance therapy. NMES: Neuromuscular Electrical Stimulation, SES: Sensory Electrical Stimulation, EMG: Electromyogram, EEG: Electroencephalogram, EOG: Electrooculogram, CoP: Center of Pressure, PC: Personal Computer. Reproduced from ⁸ and ³⁷​. Please click here to view a larger version of this figure.

1. Software Installation for Mobile Brain/Body Imaging During VBT

1. Install drivers for the Motion Capture (installation procedures provided at https://code.google.com/p/labstreaminglayer/wiki/KinectMocap)
   1. Download and install Kinect Runtime from http://go.microsoft.com/fwlink/?LinkId=253187 (Motion Capture sensor should not be plugged into any of the USB ports on the computer).
   2. Plug in the powered Motion Capture Sensor into a USB port via the interface cable. The drivers will load automatically.
2. Install drivers for the Eye Tracker Sensor (installation procedures provided at http://github.com/esdalmaijer/EyeTribe-Toolbox-for-Matlab)
   1. Download the software from http://theeyetribe.com, launch the application and launch the application to install the software (Eye Tracker sensor should not be plugged into any of the USB ports on the computer).
   2. Plug in the powered Eye Tracker Sensor and the drivers will load automatically.
3. Install drivers for the Balance Board (installation procedures provided at (installation procedures provided at http://www.colorado.edu/intphys/neuromechanics/cu_wii.html)
   1. Download and extract CU_WiiBB.zip from http://www.colorado.edu/intphys/neuromechanics/CU_WiiBB.zip
   2. Copy the WiiLab folder to Microsoft Window operating system's standard Program Files directory.
   3. Open the WiiLab folder in the Program Files directory and run as an administrator the InstallWiiLab.bat file to install the Balance Board.
4. Install drivers for EEG/EOG (installation procedures provided at http://openvibe.inria.fr/how-to-connect-emotiv-epoc-with-openvibe/ )
   1. Download and install Emotiv SDK from http://www.emotiv.com/apps/sdk/209/
   2. Download and install OpenViBE Acquisition Server with labstreaminglayer (LSL) from https://code.google.com/p/labstreaminglayer/downloads/detail?name=OVAS-withLSL-0.14.3-3350-svn.zip for distributed multi-sensor signal transport, time synchronization and data collection system (installation procedures provided at https://code.google.com/p/labstreaminglayer/).
5. Install the drivers for the commercial NMES stimulator (details at http://www.vivaltis.com/gammes/phenix/phenix-usb-neo-50-554-1.html#content).

2. Low-cost Sensor Placement for Mobile Brain/Body Imaging (MoBI): The Open-source HMI Software Pipeline Provides Mobile Brain/Body Imaging (MoBI) ¹⁹ with Low-Cost Off-the-Shelf Sensors (Figure 2b) Which Can be Adapted for Other Agility Training Programs.

1. Visual Feedback for MoBI:
   1. Begin by obtaining a projection screen to display the visual biofeedback at the one end of the room (recommended distance from subject 0.6 m).
   2. Adjust the height so that the center of the screen will be at the subjects' eye-level.
2. Motion Capture for MoBI:
   1. Place the motion capture sensor in front of the projection screen, and aim it at the volume of motion capture.
   2. Confirm that the volume of motion capture is 1.5 m to 2.5 m in front of the motion capture sensor.
3. Balance Board Placement for MoBI:
   1. Place the Balance Board on the floor, about 2.0 m away from the motion capture sensor.
   2. Leave enough room around the Balance Board to ensure full-body movement (i.e., during modified functional reach task²¹).
4. EEG/EMG/EOG Sensor Placement for MoBI
   1. Ask the subject to sit on a chair facing the Motion Capture and with their feet on the Balance Board.
   2. Place the recording (EMG) cum stimulation (NMES/SES) electrodes bilaterally on the Medial Gastrocnemius (MG) and Tibialis Anterior (TA) muscles of the subject. Then, connect them to the wireless electrical stimulator (NMES/SES) system.
   3. Place the electroencephalogram (EEG) cap on the subjects head following the International 10 - 20 system. Then, place the EEG electrodes with conductive paste at -Fz, C3, Cz, C4, P3, Pz, P4, PO7, Oz, PO8 - before connecting them to the wireless EEG headset.
   4. Place two EEG electrodes with conductive paste above and below one of eyes for vertical EOG and put two electrodes with conductive paste at the outer canthus of each eye for horizontal EOG. (Note: In case Eye Tracker sensor is not used in the post-stroke subject then bilateral EOG is preferable).
   5. Place two EEG electrodes on earlobes as reference electrodes.

3. Eye Tracker Based Evaluation of Post-stroke Pursuit Eye Movements

1. Ask the subject to sit with the chin resting comfortably on the height-adjustable Chin-Rest. Then, raise the computer monitor to a convenient height such that the eyes are roughly facing the center of the computer monitor (Figure 2a).
2. Place the Eye Tracker roughly 50 cm from the Chin-Rest and ask the subject to look straight at the computer monitor for visual cues.
3. Run EyeTribeWinUI.exe in the 'SmartEye' folder to calibrate the Eye Tracker sensor. The subject will be asked to look at various targets on the PC monitor for roughly 2 sec each. A typical user calibration process takes approximately 20 sec to complete. The (x, y) coordinates of the subject's gaze point are recorded for different cued targets for calibration.
4. Run 'Visual_Stimulus.exe' in the SmartEye folder to execute the virtual reality based interface. Subsequently run the 'SmartEye.exe' program present in the 'SmartEye' folder to acquire the subjects' eye gaze data that is synchronized with the virtual reality based task. This data will be used for the evaluation of post-stroke pursuit eye movement.

Figure 3: (a) Cursor representing the center of pressure (CoP) which needs to be volitionally driven to the cued target during visuomotor balance therapy , (b) Visuomotor balance therapy protocol where the subject steers the computer cursor to a peripheral target driven by volitionally generated CoP excursions. The Reset can be assisted with Neuromuscular Electrical Stimulation (NMES) and sensory electrical stimulation (SES), (c) Experimental setup for visually-cued visuomotor balance therapy. Reproduced from ⁸ and ³⁷​. Please click here to view a larger version of this figure.

4. NMES/SES-Assisted Visuomotor Balance Therapy (VBT) under MoBI

1. Connect the eye-tracker and balance board sensors to the visual feedback computer (Figure 2).
   1. Make sure that the Eye Tracker sensor is powered on, connected to computer, and that it has fully booted. Start the 'EyeTribe server.exe' and 'EyeTribeWinUI.exe' available in the 'VBT' folder (see steps 1.3).
   2. Make sure that the Balance Board sensor is powered on. Then, press the button on the Balance Board sensor to make the remote discoverable in the menu. Then, click on the show or hide icon in the system's taskbar and click on Bluetooth device icon. Then, click on the 'Add a Device' option and pair the Balance Board sensor as a Bluetooth device without using the code to the visual feedback computer. Once the Balance Board sensor is connected to the visual feedback computer, open the 'VBT' folder and run the WiiBBinterface.m file to establish Matlab- Balance Board sensor interface (see steps 1.6).
   3. Make sure that the Motion Capture sensor is powered on, connected to the computer and that it has fully booted (there is a green LED on the front). Open the LSL folder and start 'Mocap' software to begin streaming of the motion capture sensor data (see steps 1.6).
   4. Make sure that the EEG/EOG data acquisition systems are powered on. Then, double-click on the openvibe-acquisition-server-withlsl.cmd available in the LSL folder (see steps 1.6). From the menu, select the respective sensor hardware (i.e., 'Emotiv EPOC') and configure the module, if necessary, by clicking on the 'Driver Properties'. Then, click on 'Connect', and then click on 'Play' to start the acquisition server.
2. Calibrate the Sensors for VBT
   1. Ask the post-stroke subject to stand on the Balance Board with safety harness (and partial body weight support, if necessary).
   2. Set a minimum baseline NMES level (pulse-width and current level) necessary for upright standing according to clinical observation (i.e., zero body weight support)²². For setting the minimum baseline NMES level, one can set the stimulation frequency at 20 Hz and then increase the pulse-width and/or current level until upright standing is achieved. Here, NMES of knee extensors is required to generate enough torque to prevent knee buckling.
   3. Ask the subject to perform various reach movements that affects CoM and CoP location.
   4. Run the 'CalibSensors.m' program available in the 'DataCollect' folder in order to collect multi-sensor calibration data while the subject performs various self-initiated maximal reach movements in different directions that affect center of mass (CoM) and center of pressure (CoP) location on the visual feedback.

5. Multi-sensor Data Collection from Low-cost Sensors During VBT (Figure 2b)

1. Run the 'CollectBaseline.m' program in the 'DataCollect' folder to collect baseline resting-state, eyes-open, multi-sensor data by asking the subject to stand still for 2 min while looking straight at the CoP target on the PC monitor (Figure 3a).
2. Connect the visual feedback computer's video output to the projection screen and run the SmartEyeVRTasks.exe file in the 'VBT' folder in the visual feedback computer to launch the SmartEyeVRTasks GUI. Also, run 'CollectVBT.m' program in the 'DataCollect' folder to collect sensor data during VBT.
   1. From upright standing, called the 'Central hold' phase, ask the subject to steer the cursor, driven by the CoP, as fast as possible towards randomly presented peripheral target as cued by visual feedback (Figure 3b).
   2. Following this 'Move' phase, ask the subject to hold the cursor at the target location for 1 sec during the 'Peripheral hold' phase.
   3. Following the 'Peripheral hold' phase, the cursor will 'Reset' back to the center when the subject needs to return back to upright standing - the 'Central hold' position. NMES/SES is triggered for the muscle when its EMG level goes above a set threshold to assist the volitional effort required to return the CoP to the 'Central hold' position.
      Note: The difficulty of the mFRT can be increased by decreasing the gain, , or increasing the noise variance,, within subject-specific feasible range:
      
      where the CoP excursions, , drive the computer cursor, , in discretized time, , with time-step, .

Results

Figure 4 shows the eye gaze features that were extracted offline for the quantification of an able-bodied performance during a smooth pursuit task. The following features were extracted as shown in Table 1:

Feature 1 = percentage deviation between target stimulus position and the centroid of participant's fixation points when the stimulus is changing position in the horizontal direction.

Discussion

A simple-to-use, clinically valid low-cost tool for movement and balance therapy will be a paradigm shift for neurorehabilitation in a low-resource setting. It is likely to have a very high societal impact since neurological disorders like stroke will dramatically increase in future due to aging world population². There is, therefore, a pressing need to leverage cyber physical systems where the ability to customize, monitor, and support neuro-rehabilitation at remote sites has recently become possible with the...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
NMES stimulator	Vivaltis, France	PhenixUSBNeo	NMES stimulator cum EMG sensor (Figure 2b)
Balance Board	Nintendo, USA	Wii Balance Board	Balance Board (Figure 2b)
Motion Capture	Microsoft, USA	XBOX-360 Kinect	Motion Capture (Figure 2b)
Eye Tracker	Eye Tribe	The Eye Tribe	SmartEye Tracker (Figure 2a)
EEG Data Acquisition System	Emotiv, Australia	Emotiv Neuroheadset	Wireless EEG headset (Figure 2b)
EEG passive electrode	Olimex	EEG-PE	EEG passive electrode for EOG and references (6 in number) (Figure 2b)
EEG active electrode	Olimex	EEG-AE	EEG active electrode (10 in number) (Figure 2b)
Computer with PC monitor	Dell		Data processing and visual feedback (Figure 2)
Softwares, EMG electrodes, NMES electrodes, and cables