Mobile Game-based Virtual Reality Program for Upper Extremity Stroke Rehabilitation

Summary

Here, we present a protocol to develop and apply a mobile game-based virtual reality program for the recovery of upper limb dysfunction in patients with stroke. The present study shows that the mobile program is feasible and effectively promotes upper limb recovery in stroke patients.

Abstract

Stroke rehabilitation requires repetitive, intensive, goal-oriented therapy. Virtual reality (VR) has the potential to satisfy these requirements. Game-based therapy can promote patients' engagement in rehabilitation therapy as a more interesting and a motivating tool. Mobile devices such as smartphones and tablet PCs can provide personalized home-based therapy with interactive communication between patients and clinicians. In this study, a mobile VR upper extremity rehabilitation program using game applications was developed. The findings from the study show that the mobile game-based VR program effectively promotes upper extremity recovery in patients with stroke. In addition, patients completed two weeks of treatment using the program without adverse effects and were generally satisfied with the program. This mobile game-based VR upper extremity rehabilitation program can substitute for some parts of the conventional therapy that are delivered one-on-one by an occupational therapist. This time-efficient, easy to implement, and clinically effective program would be a good candidate tool for tele-rehabilitation for upper extremity recovery in patients with stroke. Patients and therapists can collaborate remotely through these e-health rehabilitation programs while reducing economic and social costs.

Introduction

Stroke is one of the most common causes of neurological impairment in adults. Recovery from impairment after a stroke is usually incomplete, and approximately 50% of patients are left with disabilities, making them dependent on others¹. In particular, upper limb dysfunction makes stroke survivors dependent on others for assistance with activities of daily living (ADL)². Regaining the lost function in the upper extremities may be more difficult to achieve than returning the normal function of ambulation to the lower extremities. Although bilateral lower extremity movement is indispensable for locomotion, patients can perform ADL with unilateral upper extremity movement. This leads to a learned non-use phenomenon of the affected limb³. This phenomenon is an obstacle to the rehabilitation of the upper extremity in stroke survivors. Therefore, a tremendous amount of research is focused on the upper limb function recovery. Studies have highlighted the importance of extensive practice and repetitive task-specific training^4,5,6.

Virtual reality (VR) technology has recently been introduced into the field of rehabilitation⁷. VR allows users to interact with a simulated environment and receive continuous, immediate feedback related to performance. VR has the potential to apply basic concepts of neurorehabilitation in stroke patients, such as intensive, repetitive, and task-oriented training⁸. Specifically, non-immersive VR does not require high-level graphics performance or special hardware. Therefore, non-immersive VR is a good candidate for providing a low-cost, ubiquitous, and interesting treatment program. Previous studies used computers, monitors, and special devices, such as consoles, sensor gloves, joy-sticks, and commercial gaming systems for non-immersive VR⁹. Higher start-up costs and sufficient space were mandatory for using such systems. Recently, low-cost tools, such as commercial gaming devices, have been utilized to develop new rehabilitation systems^10,11. However, the consoles with sensors in those devicesare not sufficiently small and lightweight for carrying. Nevertheless, to improve the popularity of non-immersive VR as a post-stroke upper extremity treatment method and to create a ubiquitous rehabilitation environment for stroke survivors, portable and inexpensive tools are needed.

Furthermore, game-based therapy can be a good option for stroke rehabilitation. Many patients complain that conventional occupational therapy (OT) for upper limb function recovery is boring and monotonous^12,13. A more interesting and motivating tool for the therapy is, therefore, necessary to promote patients' engagement in rehabilitation training. Many studies that involve the use of commercial games have been conducted^14,15,16. However, the games used do not target the desired movement of the upper extremity in patients with stroke, and they lack special consideration for the spasticity that may be present after a stroke.

This paper describes the development of a mobile game-based VR program and its use for patients who have experienced a stroke and suffer from upper limb dysfunction (Figure 1).

Protocol

The study was approved by the Seoul National University Bundang Hospital Institutional Review Board, and all participants gave written informed consent before screening.

1. Game Contents Development

NOTE: The mobile game-based upper extremity virtual reality program for patients with stroke (MoU-Rehab) consists of mobile game applications.

1. Install the studio and environment for development using a programing language.
2. Collect data on x, y and z-axis locations of patients' movements through the built-in sensors of the smartphone.
3. Allocate storage for data on the motion, location, and velocity with the following source code:
   float[ ] gravity_data = new float[3];
   float[ ] accel_data = new float[3];
   float[ ] m_acc_data = new float[3];
   final float alpha = (float)0.8;
   Gather data measured by the built-in sensor with the following source code.
   SensorManager sm = (SensorManager) getSystemService(SENSOR_SERVICE);
   Sensor mSensor = sm.getDefaultSensor(Sensor.TYPE_ACCELEROMETER);
4. Send the data to the tablet PC after image processing through the short-distance wireless connection to display x, y, and z-coordinates as images on the screen.
5. Design all game applications to improve the strength, endurance, range of motion, control, speed, and accuracy of movement in the upper extremity.
   NOTE: Board-certified physiatrists (rehabilitation medicine doctors) and occupational therapists gathered and discussed what therapeutic maneuvers would be appropriate to transform into game programs. Movements in each joint was targeted based on the conventional occupational therapy methods that promote the recovery of upper limb function and improve the strength, endurance, range of motion, control, speed, and accuracy of movement in the upper extremity
6. Recommend game applications to each patient according to their upper extremity function measured by the Brunnström stage.
   NOTE: The Brunnström stage (B-stage) for the arm and hand evaluates the upper extremity recovery stages (1=flaccid, no voluntary movements; 7= normal function)¹⁶. For example, patients categorized as Brunnström stage 1 used applications that induce shoulder flexion/extension movements with the assistance of the unaffected arm. Patients categorized as Brunnström stage 5 used applications that require precise control of movements.
7. Adjust the level of difficulty of the game applications individually, by changing the speed, maintenance time for the specific posture, and range of motion according to the severity of the upper extremity dysfunction. Refer to the examples with detailed information as below (Figure 2).

2. Study Design

NOTE: A quasi-randomized, double-blind, controlled trial was conducted to evaluate the program's feasibility and effectiveness. Participants who (1) were diagnosed with ischemic stroke; (2) had the ability to follow a one-step command; (3) had the medical stability to participate in active rehabilitation, and (4) had upper extremity impairment, were included. Patients were excluded if they (1) had delirium, confusion, or other severe consciousness problems, (2) suffered from uncontrolled medical conditions, (3) were unable to follow commands because of severe cognitive impairment, (4) had a visual disturbance, and (5) had poor sitting balance. Participants were recruited at the university hospital.

1. To make the participants completely blinded to the group assignment, assign them to either control or experimental group by admission period because participants needed to share the OT room.
2. Ensure that the patients in the intervention group received 30 min of conventional OT and 30 min of the mobile game-based VR upper extremity rehabilitation program using a smartphone and a tablet PC.
3. Ensure that the patients in the control group received conventional OT alone for 1 h per day.
   NOTE: The rehabilitation program for both groups consisted of 10 sessions of therapy, 5 days per week, for 2 weeks.
4. Educate the patients in the intervention group on how to use the program in the first treatment session, although each game application provided brief instructions at the beginning. Ensure that the patients used the program for 30 min on their own during the study.
5. Assess the outcome measurements at the beginning, at the end of the treatment and at a one-month follow-up.
6. Let a single blinded evaluator who is an experienced occupational therapist perform all the clinical assessments during all the testing sessions.
   NOTE: The primary outcome measure was the Fugl-Meyer Assessment of the upper extremity (FMA-UE), which measures the motor function of the patients' hemiparetic arm (0 = lowest score; 66 = highest score).¹⁷ The secondary outcome was evaluated by the Brunnström stage (B-stage) for the arm and hand and manual muscle testing (0–5).

3. Usage of Mobile Game-based VR Upper Extremity Rehabilitation Program

1. Have the participant sit on the desk.
2. Place the tablet PC on the desk and turn the tablet PC on.
3. Turn the smartphone on and turn on the short-distance wireless connection on the smartphone.
4. Select the game application on the tablet PC by touching the screen of the tablet PC.
5. Enable short-distance wireless connection on the tablet PC and connect to the smartphone.
6. Put the smartphone in the armband and attach the armband to the upper arm or forearm using a commercially available smartphone armband according to the desired movement.
7. Choose either the basic version or the customized version.
8. Select the game speed, time, a number of target movements, and the expected achievement of the game on the touchscreen of the tablet PC on the customized version according to the participants’ ability.
9. Let the participant touch the start button. Ask the participant to take a posture following the commands of the game. Move the maximal range of motion for calibration.
10. Start playing the game and move to the next game when the game ends (generally 5–10 min long).
    1. "Honey Pot Guard" Game
       NOTE: This game targets the elbow flexion and extension motion.
       1. Ask the patients to beat a bear from the honey pot by throwing apples using the movement of the elbow joint.
       2. Let the patient throw apples when the flexed elbow is extended.
       3. Adjust the level of difficulty by the location where the bear appears and the speed at which a new bear appears based on the available range of motion and speed of the motion in the patient's elbow joint.
    2. "Protect the Bunny" Game
       NOTE: This game targets the shoulder abduction and adduction motion.
       1. Ask the patients to protect the bunny by catching stones with a net using the movement of the shoulder joint.
       2. Ensure that the net moves when the adducted shoulder is abducted.
       3. Adjust the level of difficulty by the location where the stone appears and the speed with which a new stone appears based on the available range of motion and speed of motion in the patient's shoulder joint.
    3. "Put out Fire" game
       NOTE: This game targets the shoulder abduction and adduction or shoulder flexion and extension.
       1. Ask the patients to put the fire out with the water hose through the window; the water hose is moved according to the motion of the shoulder.
       2. Ensure that the water hose moves right and left and up and down according to the motion of the shoulder joint; the time needed to put out the fire depends on the size of the fire.
       3. Adjust the level of difficulty by the location where the fire appears, the speed with which a new fire appears, and the size of the fire based on the available range of motion and the speed and endurance of motion of the patient’s shoulder joint.
    4. "Flower Splash" game
       NOTE: This game targets the shoulder abduction and adduction, elbow flexion and extension, or wrist pronation and supination motion.
       1. Ask the patients to water the moving seed with a watering can by the motion of the joint and make the flowers blossom.
       2. Ensure that when the seed moves, the watering follows according to the motion of the joint.
       3. Adjust the level of difficulty by the moving speed of the seed with respect to the accuracy and speed of motion.
11. Track the patients' upper extremity movements by the built-in sensors (accelerometer, gyroscope) of the smartphone, and transfer information about the movement to the tablet PC through the short-distance wireless connection.
12. Provide visual and auditory feedback for patients on their movement with the display on the tablet PC.
13. Demonstrate the real achievement of the game as compared with the expected achievement at the end of the game.

Representative Results

A total of 24 patients were enrolled and assigned to either the control or the experimental group (Table 1). A greater improvement in the FMA-UE, B-stage, and manual muscle testing was found after treatment with the mobile game-based VR upper extremity rehabilitation program than with conventional therapy (Figure 3). The effect was maintained until the one-month follow-up. This means MoU-Rehab was not inferior to the conventional therapy that is delivered one-on-one by an occupational therapist.

Patients in the experimental group completed the two-week treatment without adverse effects, and were generally satisfied with MoU-Rehab (Table 2), even though participants had various levels of IT (computer, tablet PC, and smartphone) familiarity (Figure 4). Patients in the experimental group responded positively about the display of the program (4.25 ± 0.62), the readability of the program (4.25 ± 0.62) and the convenience of program usage (4.08 ± 0.67) in 5-point Likert rating (0–5). In addition, they stated that they were willing to pay $22 ± 10 for the game applications. There was no relation between IT familiarity and the amount of improvement.

The MoU-Rehab is feasible and effective in promoting upper limb recovery after an ischemic stroke.

Figure 1: The Configuration of the mobile game-based upper extremity VR program. The system for the mobile game-based upper extremity VR program includes a mobile device (tablet PC) for display and a mobile device (smartphone) for obtaining information on the movement using built-in sensors. Please click here to view a larger version of this figure.

Figure 2: Game contents and applications of the mobile game-based upper extremity VR program. Each game application targets a specific movement of the hemiplegic arm. The level of difficulty is adjusted according to each patient's upper extremity function.The target movements in the target joints is represented by yellow lines and circles. Please click here to view a larger version of this figure.

Figure 3: Changes in the outcome measures at the end of the treatment and at the 1 month follow-up after treatment. (A) Changes in the Fugl-Meyer Assessment of the upper extremity. (B) Changes in the Brunnström stage (arm and hand). (C) Changes in muscle power measured by the manual muscle test (shoulder/ elbow/ wrist). Mo: month, pre: before treatment with mobile game-based upper extremity VR program, post: after the two-week treatment with the mobile game-based upper extremity VR program, F/U: follow-up. The error bars indicate standard deviations (SD). Please click here to view a larger version of this figure.

Figure 4: Results of the IT familiarity survey. Participants in the experimental group (n = 12) were asked about their IT familiarities such as their computer, tablet PC, or smartphone experience. Please click here to view a larger version of this figure.

	Experimental group (n=12)	Control group (n=12)	p-value
Sex (M/F)	7/5	6/6	0.5 a
Age (years)	61.0±15.2 (21-76)	72.1±9.9 (53-88)	0.046 b
Affected limb (L/R)	8/4	10/2	0.32 a
FMA-UE (range)	24.5±22.2 (4-63)	21.5±20.6 (4-57)	0.735 b
Brunnstrom-stage (arm)	2.7±1.5 (1-5)	2.7±1.5 (1-5)	1.00 b
Brunnstrom-stage (hand)	1.9±1.4 (1-5)	2.1±1.4 (1-4)	0.775 b
MMT (shoulder)	2.7±1.1 (1-4)	2.2±1.2 (0-4)	0.292 b
MMT (elbow)	1.9±1.4 (1-5)	2.1±1.4 (1-4)	0.775 b
MMT (wrist)	2.7±1.1 (1-4)	2.2±1.2 (0-4)	0.292 b

Table 1: Baseline demographic and clinical characteristics of the patients. There were no statistically significant differences between the two groups except with regard to age. ^a χ² test, ^b t-test. FMA-UE: Fugl-Meyer Assessment of the Upper Extremity, MMT: Manual Muscle Test, M: male; F: female, L: left, R: right. Data indicate mean ±SD (range).

Question	Experimental group (n = 12)	Control group (n = 12)
The upper extremity rehabilitation program provided sufficient treatment for me.	4.17 ± 0.72	4.00 ± 0.85
The upper extremity rehabilitation program provided on target services to you	4.42 ± 0.52	3.92 ± 1.00
In general, I am satisfied with the upper extremity rehabilitation program .	4.25 ± 0.75	3.92 ± 1.00
The upper extremity rehabilitation program helped me to better manage my health and medical needs.	4.33 ± 0.65	4.00 ± 0.74

Table 2: Results of the user satisfaction survey for mobile game-based upper extremity VR program. User satisfaction in the experimental group was not inferior to that in the control group. Participants in the experimental group responded positively to all items.

Discussion

Patients with stroke usually have disabilities related to motor impairments due to incomplete motor recovery. Such disabilities, lengthy travel time to a clinic, or socioeconomic difficulties can hinder patients' access to adequate rehabilitation therapy. A ubiquitous healthcare (u-Health) program can be a good option for removing those barriers. As a part of such a u-Health program, a mobile game-based VR rehabilitation program was developed for upper limb recovery after ischemic stroke in the present study. Our findings suggest that this program is feasible and more effective in promoting upper limb function recovery compared to conventional therapy. At the one-month follow-up evaluation, improvements were successfully maintained.

We speculate that the therapeutic effectiveness of MoU-Rehab was equal to or greater than that of conventional therapy, because of the game's effects, such as immediate feedback from the patients' movement, enjoyment, high motivation, and engagement. Participation games may facilitate motor learning¹⁹ while increasing interest in rehabilitation and promoting motivation. Moreover, auditory and visual feedback can facilitate patients' desire for interaction. These factors may increase the efficacy of rehabilitation therapy by achieving a high level of patient adherence to training and increasing engagement in therapy.

Most of the previous game-based studies used commercial off-the-shelf games¹⁴. However, these games were not specifically designed for patients with stroke. The game program developed in this study specifically targets patients with stroke. Various kinds of game applications were developed, from which applications that could be used for such patients were selected. We chose only game applications that could induce desired movements and avoided synergistic movements.

The mobile program was designed for use with a smartphone and tablet PC because smartphones and tablet PCs are increasingly used by the general population; thus, it is relatively easy to implement treatment programs using these handheld mobile devices at low cost. Considering the lightweight aspect and small size of these mobile devices, they are portable and easy to use, regardless of a person's location. Although the program was administered in the therapy room to ensure strict matching of the treatment time between the two groups, this program can also be used in home settings.

However, the study has a few limitations. The first limitation stems from the study's small sample size. Although attempts were made to allocate patients to the two groups randomly, patients were allocated by admission period to keep them unaware of their group allocation. Treatment time was strictly matched during the 2-week treatment period but was not possible during post-treatment and follow-up. The motor activity of the affected arms could not be monitored during the study period. Measuring the activity of the affected arms by accelerometers could indicate the influence of the program on the non-use phenomenon^20,21. In the present study, no kinematic data were available. By obtaining kinematic data such as linear and angular displacements, velocity, and acceleration, information on the characteristics of the patients' movements could be used to optimize individualized rehabilitation therapies through bidirectional feedback between patients and clinicians.

The findings of the present study suggest that this mobile game-based VR upper extremity rehabilitation program can be a substitute for some parts of conventional therapy that are delivered one-on-one by an occupational therapist. The program would be a good candidate tool for tele-rehabilitation for upper extremity recovery in patients with stroke.

Materials

Name	Company	Catalog Number	Comments
Galaxy Note 10.0	Samsung	Galaxy Note 10.0	Tablet PC
Galaxy S2	Samsung	Galaxy S2	Smartphone
Bluetooth	Bluetooth SIG	Bluetooth	short-distance wireless connection
Java	Oracle	Java	programming language