Brain-Computer Interface-controlled Upper Limb Robotic System for Enhancing Daily Activities in Stroke Patients

Summary

This study introduces a brain-computer interface (BCI) system for stroke patients, which combines electroencephalography and electrooculography signals to control an upper limb robotic hand, enhancing daily activities. The evaluation used the Berlin Bimanual Test for Stroke (BeBiTS).

Abstract

This study introduces a Brain-Computer Interface (BCI)-controlled upper limb assistive robot for post-stroke rehabilitation. The system utilizes electroencephalogram (EEG) and electrooculogram (EOG) signals to help users assist upper limb function in everyday tasks while interacting with a robotic hand. We evaluated the effectiveness of this BCI-robot system using the Berlin Bimanual Test for Stroke (BeBiTS), a set of 10 daily living tasks involving both hands. Eight stroke patients participated in this study, but only four participants could adapt to the BCI robot system training and perform the postBeBiTS. Notably, when the preBeBiTS score for each item was four or less, the BCI robot system showed greater assistive effectiveness in the postBeBiTS assessment. Furthermore, our current robotic hand does not assist with arm and wrist functions, limiting its use in tasks requiring complex hand movements. More participants are required to confirm the training effectiveness of the BCI-robot system, and future research should consider using robots that can assist with a broader range of upper limb functions. This study aimed to determine the BCI-robot system's ability to assist patients in performing daily living activities.

Introduction

Impairment of upper extremity function due to stroke limits the ability to perform daily activities, especially bimanual tasks¹. Hand rehabilitation is, therefore, a key component of stroke rehabilitation, with mirror therapy² and Constraint-Induced Movement Therapy (CIMT)³ being well-known approaches. Recent research indicates that EEG-based Brain-Computer Interface (BCI) robot systems can be an effective assistive therapy for improving hand function recovery in stroke patients^4,5,6. BCI robotic systems focus on coupling the patient's active intention to attempt a motor movement with its performance. Research is actively being conducted to determine whether this approach is effective for rehabilitation^{7,8,9,10,11,12,13}.

In this study, we present a BCI-controlled upper limb assistive robotic system designed to help stroke patients perform bimanual activities. The system utilizes electroencephalograms (EEG) to detect and interpret brain signals associated with motor imagery and combines them with electrooculograms (EOG) for additional control inputs. These neurophysiological signals enable patients to control a robotic hand that assists with finger movements¹⁴. This approach bridges the gap between a patient's desire to move and physical ability, potentially facilitating motor recovery and increasing independence in daily tasks.

Researchers at the Charité Medical University in Berlin developed the Berlin Bimanual Test for Stroke (BeBiTS), a comprehensive assessment tool, to evaluate the efficacy of this BCI robotic system¹⁵. The BeBiTS provides a quantitative measure of functional improvement by assessing the ability to perform ten bimanual activities essential to daily living. The assessment scores each task individually and evaluates five components of hand function: reaching, grasping, stabilizing, manipulating, and lifting. It enables a comprehensive evaluation of patients' functional improvements, focusing on activities of daily living. Furthermore, it allows us to quantify the contribution of the BCI robot system in enhancing specific hand functions. This study, therefore, aims to develop an effective BCI assistive robot system by comparing BeBiTS scores before and after training sessions in stroke patients.

Protocol

The Seoul National University Bundang Hospital Institutional Review Board reviewed and approved all experimental procedures (IRB No. B-2205-756-003). We recruited eight stroke patients and thoroughly explained the relevant details before obtaining their consent. After obtaining informed consent, the protocol proceeds as follows: we perform a BeBiTS assessment before BCI training, followed by BCI training using EOG and EEG. Afterward, participants wear the robot to perform another BeBiTS assessment (Figure 1).

1. BCI-robot training system setup

1. Patient recruitment
   1. Perform the screening process using the following inclusion criteria.
      1. Select patients aged 20 to 68 with impaired upper limb functions.
      2. Select patients who are unable to flex or extend the fingers of the paralyzed hand.
      3. Select patients with a single subcortical stroke (including both ischemic and hemorrhagic stroke).
      4. Select patients who are more than 6 months post brain injury.
      5. Select patients with an impaired Fugl-Meyer score of less than 31.
   2. Provide all recruited patients with detailed information about the experimental procedure and obtain signed informed consent.
      ​NOTE: A legal representative must provide the signed informed consent If a patient meets the criteria but cannot sign the consent.
2. BCI system: EEG device
   1. Use a EEG viewer system (see the Table of Materials) for data recording.
   2. Use a personal computer (PC) with custom BCI software and connect the PC to the EEG device.
3. Upper Limb assistance robotic hand
   1. An assistive robotic hand¹⁶ that supports three fingers is used on the patient's impaired upper limb. This robot is a soft, leather-like glove designed to cover only three fingers: the thumb, index, and middle fingers. Connect the robot wirelessly to the computer using a USB dongle.
   2. When connected to the computer via a dongle, the robot's initial state is neutral, meaning that its hand is fully open. The designed BCI system sets up the robot so that its hand can only close when it recognizes the intention to make a fist, based on whether the measured EEG value meets the threshold. Figure 2 shows the schematic of this BCI-robot system.

2. BCI-robot assessment

1. Berlin Bimanual Test for Stroke (BeBiTS)
   NOTE: The BeBiTS assessment is a tool for evaluating the performance of 10 bimanual activities of daily living in stroke patients.
   1. Position the patient comfortably in an armchair facing a desk. Ensure the patient is positioned close to the desk, approximately 30 cm away from the objects used in the assessment, to perform the evaluation tasks using both arms and hands.
   2. Keep the following 10 items for the assessment of bimanual activities: Open a jar; open a plastic packet; open a water bottle; pour a glass of water; cut meat-like putty; clean a dish; lift a pot; open a toothpaste tube; apply toothpaste on a toothbrush; close the zipper of a jacket.
      NOTE: The 10 evaluation tasks use assessment items, all of which are performed sitting at a desk. Each action is scored out of 10, with an overall score of 100.
   3. Evaluate and score five hand function components alongside the individual item scores. These components and their respective scores are reaching (20), grasping (20), stabilizing (10), manipulating (33), and lifting (17), with the total score adding up to 100 points (Figure 3).

3. BCI-robot training system

1. EEG setup
   1. Open the BCI system. Figure 4 shows the entire screen when the BCI system is opened.
   2. Put on a cap and connect the amplifier.
   3. In the Source module, select EegoModule | impedance mode. Press Start in the source module, which shows blue light (Figure 5A).
   4. Ensure the impedances are below 10k Ohms, then press Stop in the source module.
   5. Change the mode to EEG to stream the data and press start. Check the signal with the given software (Figure 5B).
2. EOG calibration
   1. In the preprocessing module, select the Pipeline specific to the EOG montage (e.g., SMR-EOGleft for EOG electrode on the left eye; Figure 6A).
   2. In the Task module, set the Number of cues (e.g., 10). The Task module sets Directions contralateral to the robotic hand (Figure 6B).
   3. Instruct the participant to perform brief lateral eye movements in the direction of the 10 arrows that appear (Figure 6C).
   4. Check the result graph immediately after training. The gray lines represent each eye movement attempt; confirm that the orange line, which is the average, touches the baseline, indicating that the training is successful (Figure 7).
   5. If the training is successful, record the threshold parameter value.
      ​NOTE: Consistent repetition of eye movements is essential. Although it varies from participant to participant, EOG training is generally practical after about three attempts and usually improves with repeated training.
3. EEG calibration
   1. In the Task module, choose EEGCalibrationTaskModule (Figure 8A). Set the Number of cues in the Task module to 5.
   2. In the Feedback module, set Laterality to the side of the robotic hand and ensure Display Pac-Man is unselected (Figure 8B).
   3. Instruct the participant to imagine clenching a fist when the prompt 'Imagine making a fist' appears on the black screen (Figure 8C). Ask the participant to relax when the screen is black. Repeat this process, with the fist-clenching imagination lasting 5 s and the rest period randomly varying between 10 s to 15 s.
   4. Review the result graph after EEG calibration and use the characteristic ERD pattern in the 8-12 Hz mu-band during motor imagery to determine the participant's appropriate frequency range. Record the parameter value of 11 Hz from the left graph in Figure 9, which shows the ERD response during motor imagery compared to the rest state. In the right graph of Figure 9, highlight the distinct red line, which represents fist-clenching imagery, and the blue line, indicating the relaxed state, in relation to the horizontal dotted threshold line.
   5. If the EEG calibration training is adequately performed, record the reference value and threshold below the right-hand graph.
      ​NOTE: However, there may be cases where short-term calibration training is challenging due to issues such as lack of understanding of the instructions provided or BCI-illiteracy.
4. Training with feedback
   1. After completing EOG and EEG training, set parameter values that distinguish the intention of making a fist for the specific target frequency of interest, reference value, and threshold identified by the result graph in the previous training.
   2. Using the configured parameters, proceed with feedback training using Pac-Man. If the parameters are correctly set after training, observe that Pac-Man's mouth will gradually close when the participant imagines clenching their fist. If Pac-Man's mouth does not close properly, repeat the EEG calibration training while adjusting the reference value and threshold (Figure 10).
5. Experiment: BeBiTS assessment using the BCI robot system
   1. After completing all feedback training (approximately 30 minutes), have the participant wear the robot and conduct the postBeBiTS assessment using the trained BCI system.
      NOTE: If the patient's hand spasticity is severe, special care is needed when wearing the robot. Additionally, the three-finger assistance provided by the robot may not be sufficient for some patients, making it difficult to perform all the movements required in the BeBiTS assessment. In such cases, only the movements that can be performed are evaluated.
   2. Wait for the white light indicating the ready and start state to appear on the screen, as shown in Figure 11A. After confirming the white light, the participants move their eyes to one side to change the light to green (Figure 11B). When the green light appears, the participants imagine clenching their fist.
      NOTE: If the system recognizes the participant's intention well, it activates the robot worn by the participant to clench its fist.
   3. Have the participant clench their fist with the help of the robot, and then perform the action shown in Figure 11C.
   4. Figure 11D displays a red light, indicating that the robot is maintaining a fist-clenched position. Upon completing the action, the participant observes this red light on the screen, as illustrated in Figure 11D. At this point, if the participant wants to open their hand again with the help of the robot, they can move their eyes to change the light color back to white.

Results

Figure 12 shows the results of EOG and EEG training. Figure 12A represents the results of a well-trained participant. The EOG training values are consistent, with the average (orange bold line) properly reaching the threshold line. The EEG training results also clearly distinguish between the blue (resting state) and the red (motor imagery) lines.

In contrast, Figure 12B shows the results of a participant...

Discussion

This research presented a BCI upper limb assistive robotic system to support stroke patients in executing daily tasks. We assessed the efficacy of bimanual tasks through the BeBiTS test¹⁵ and implemented training for the operation of the upper limb assistive robot via the BCI system¹⁴. This approach, in contrast to conventional rehabilitation procedures, allows patients to actively engage in their recovery by controlling the robot's operations according to their intenti...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
BCI2000	open-source		general-purpose software system for brain-computer interface (BCI) research that is free for non-commercial use
BrainVision LSL Viewer	Brain Products GmbH		a handy tool to monitor its LSL EEG and marker streams.
eego mini amplifier with 8-channel (F3, F4, C3, Cz, C4, P3, P4, EOG) waveguard original caps	Ant Neuro, Netherlands		Compact and lightweight design: The eego mini amplifier is small and lightweight, offering excellent portability and suitability for EEG recording in various environments.
Neomano	neofect, Korea		Glove Material: Leather, velcro, Non-slip cloth Wire Material: Synthetic Thread Weight: 65 g (without batt.) cover three fingers: the thumb, index, and middle fingers
personal computer (PC) with custom BCI software			window laptop