A Structured Rehabilitation Protocol for Improved Multifunctional Prosthetic Control: A Case Study

Summary

As prosthetic development moves towards the goal of natural control, harnessing amputees’ inherent ability to learn new motor skills may enable proficiency. This manuscript describes a structured rehabilitation protocol, which includes imitation, repetition, and reinforcement learning strategies, for improved multifunctional prosthetic control.

Abstract

Advances in robotic systems have resulted in prostheses for the upper limb that can produce multifunctional movements. However, these sophisticated systems require upper limb amputees to learn complex control schemes. Humans have the ability to learn new movements through imitation and other learning strategies. This protocol describes a structured rehabilitation method, which includes imitation, repetition, and reinforcement learning, and aims to assess if this method can improve multifunctional prosthetic control. A left below elbow amputee, with 4 years of experience in prosthetic use, took part in this case study. The prosthesis used was a Michelangelo hand with wrist rotation, and the added features of wrist flexion and extension, which allowed more combinations of hand movements. The participant’s Southampton Hand Assessment Procedure score improved from 58 to 71 following structured training. This suggests that a structured training protocol of imitation, repetition and reinforcement may have a role in learning to control a new prosthetic hand. A larger clinical study is however required to support these findings.

Introduction

Replacing hand function in amputees is a difficult endeavor. Coordinating highly skilled hand movements is not an innate ability, and takes humans years of learning to develop.^1–5 After the traumatic loss of a hand, replicating this capability by prosthetic means is not a trivial task and may require a period of sustained learning.

Prosthetic design and interfacing methods for their control are subject to rapid technological innovations, with the goal of multifunctional control in a natural manner.⁶ The complexity of these control systems increases substantially to provide more functions for amputees. To ensure accurate control of these systems, and to reduce abandonment of new technologies, adequate training needs to be established. This is likely to be more successful if it is based on the amputees’ inherent learning strategies.

Vision can play an important role during learning of hand movements. Behavioral studies have shown that by observing the actions of others⁷ or using visual cues⁸, able-bodied individuals learn and coordinate new movements. Through a process of observation, understanding and execution of an observed action, individuals are able to imitate the actions of others. Specific cortical networks, which may include a mirror-neuron system (MNS), are believed to underlie this capability, and may have a role in controlling prosthetic limbs.^9–11

The role of imitation might not just be limited to executing actions that have already been seen, but together with the MNS, allow the execution of movements that have not yet been observed but extrapolated from the observer’s motor repetoire.¹² Indeed, imitation may not necessarily be an innate ability, but an accruement of motor skills over time that lead to experienced and sophisticated actions.¹³ The strength of observing actions, over just simply imagining them, has been shown to improve learning new tasks.¹⁴ Thus, imitation may be a pragmatic approach to training amputees, as evidence suggests it a goal directed process¹⁵, with the target in the rehabilitation setting of enabling useful prosthetic hand function.

Rehabilitation studies have separately shown that visual cues, such as virtual simulations of a prosthetic hand, encourage amputees during rehabilitation training.¹⁶ In addition, the use of repetition when conducted in a blocked paradigm has been shown to enable rapid learning of upper-limb prosthetic control.¹⁷ While virtual simulations have been proven to be equally effective as real control of prosthetic hands in enabling abled-body users to control myoelectric devices,¹⁸ their effect on amputees using standardized outcome measures is not clear. Finally, where protocols for upper limb amputation training exist, the role of imitation in learning of prosthetic control is not explicitly discussed.^19,20

This study aims at understanding if the use of imitation, in combination with repetition and reinforcement, has a positive impact on the learning of multifunctional prosthetic control as part of a structured training program.

Presented herein is a case report of a transradial amputee who was trained to use a multifunctional prosthetic hand. The participant had previously become accustomed to operating traditional myoelectric prostheses. Using visual cues, both in the form of imitation of a healthy demonstrator and as simple computer visual feedback, the amputee quickly improved handling of his new device.

Protocol

This study was carried out in accordance with the Declaration of Helsinki, as approved by the local research ethics committee. The study was explained in full detail to the participant prior to commencement, allowing the participant the time to weigh up the decision to voluntarily take part in the study and confirm his participation by informed, written consent.

Note: One man, aged 27 years, took part in the study. The participant had normal vision, was a left below-elbow amputee, and was an experienced user (4 years total prosthesis use). Prior to commencing this study the prosthesis he used on a daily basis was a 4 channel myoelectric prosthetic hand with wrist rotation for 12-15 hr per day for 15 months. The participant’s right hand had previously been surgically reconstructed, but had no other physical or neurological impairment.

1. Study Design

1. Split the study over two sessions: naïve use, and use following structured training.
   Note: This is to allow intra-subject comparison before and after training respectively.
2. Ensure that these two sessions are at least three months apart, so as to be treated as independent from each other.
3. At the beginning of both sessions, fit a customized socket and prosthesis to the participant. Ensure that the prosthetic hardware and control algorithms match those detailed in the Materials section of this protocol. Ensure that the participant is not able to use the customized prosthesis in the intervening time between sessions.
4. Train the patient according the steps outlined in the Naïve Session and Structured Training Session sections of this protocol. At the beginning of each of these sessions, calibrate the prosthetic hardware. Use the collected calibration data for real-time prosthetic control.
5. Once the Naïve Session and Structured Training Session are complete, assess the participant’s performance using the Southampton Hand Assessment Procedure (SHAP) outcome measure.²³ Compare the SHAP scores to a baseline measure using the participant’s standard prosthesis (obtained before either training sessions).

2. Materials

1. Fit the participant with a custom-built socket. Attach a commercially available prosthesis according to manufacturer’s instructions. Equip the prosthetic hand with prototype components that allow actuated wrist flexion, extension and rotation. This enables the participant to control the hand with 3.5 degrees of freedom (DoFs) (Table 1).
   Note: In this experiment a Michaelangelo Hand (See Materials List) was used. Other terminal devices capable of wrist rotation, flexion and extension, together with standard grip functions would also be appropriate.
2. Record EMG signals using eight equidistantly placed raw signal electrodes around the stump, and an on-board decoding system at the sampling rate of 1,000 Hz and digitized with 10-bit depth. Perform the initial filtering and amplification within the electrodes themselves according to the vendor’s specifications. Use a personal computer (PC) to conduct the main processing, which communicates with acquisition hardware and controls the prosthesis via a wireless connection.
   Note: In this study the surface EMG electrodes and on-board decoding system (AxonBus) used were from Otto Bock. Other manufacturers of similar devices would also be appropriate. The wireless connection was via Bluetooth, and likewise other modalities could be applied.

3. Control Algorithm

1. Use a control algorithm that provides simultaneous and proportional prosthetic control across multiple DoFs.²¹ The algorithm used in this study was a two-stage decision making paradigm, so that context-dependent movement estimation was possible.
2. Upon system training, which contains all the controllable single DoF movements, record an incoming electromyogram (EMG).
3. In the first stage, assess intrinsic dimensional information of the intended motion based on the Mahalanobis distance of the newly calculated EMG feature vector from the training data. Make a decision as to whether the user’s intention was to perform a fine 1-DoF or a coarser simultaneous 2-DoF motion.
   Note: Mahalanobis distance of a feature vector x to class i with the class mean vector µ_i and covariance matrix Σ_i is calculated as:
   
   As described in Amsuess et al, the newly calculated feature vector is mapped to the high dimensional space and the Mahalanobis distance of the transformed point to any of the trained class points is taken as the measure for novelty.²¹ An empirically determined threshold to that distance gives the decision for novelty (2-DOF) or not (1-DOF).
4. In the second stage, based on the previous decision, use one of two parallel estimators — one dealing with the sequential movements (SEQ-E) and the other handling simultaneous motions (SIM-E) — to provide the control signals for the prosthesis.
   Note: SEQ-E is in essence a proportional estimator (i.e., the strength of muscular contractions) based on common spatial patterns (CSP)²¹, while SIM-E is a linear regressor, which simultaneously steers 2 DoFs of the wrist.

4. Software Framework

Note: The software framework used in this study allowed handling of the communication between the prosthetic hardware and the embedded control algorithm. It also offered visually supportive training tools needed for maximizing participant training.

1. Display the root mean square (RMS) of the EMG signals collected from the 8 equidistantly placed electrodes in a form of a polar plot of EMG amplitude as a function of electrode location. This visual feedback enables easy monitoring of the spatial distribution of the EMG in the transverse plane of the forearm. Using such a setup, each of the user’s motions can thereby elicit a distinct pattern²² in the polar plot, which can then be saved and used to train for the repeatability of the specific gesture.
   Note: The framework enables the collection of EMG data in the standard pattern recognition manner.²³ For each of the EMG channels the RMS over 40 msec is calculated as
   
   resulting in observations for every ensemble window.
2. For initial calibration collect the maximal long-term voluntary contraction (MLVC) values for each intended motion. Prompt the participant by using the demonstrator’s hand to perform the desired motion while giving vocal and visual instructions for 5 sec.
3. After calibration, present the participant with a set of trapezoidal cues. These force profiles contain plateaus set at 30%, 60% and 90% of the calibrated maximum.
4. Within each trial, instruct the participant to steer the red pointer along the cue by modulating the force level of the prompted motion (Figure 1). The vertical position of the pointer corresponds to the summed RMS values across all eight channels. Set the duration of the trial to 5 sec with plateau interval corresponding to the middle 3 sec.

5. Naïve Session

Note: During the naïve training session, the participant had no prior experience of the prosthetic control scheme used in this study.

1. Do not give the participant any formal clinical training, but only instruct that 8 actions of the residual limb, of which one is a resting state, will allow control of a visual target on a computer screen. These tasks are similar to those used in classic pattern recognition approaches for prosthetic control²³, and for those methods the participant in this study had approximately 60 hr of previous experience.
2. Display the required movements on screen in terms of text and a static picture while following a visual cue (Figure 1).
3. Show the participant his EMG activation patterns, which correspond to eight specific and unique polar plots (Figure 2).
4. Use audible instructions to encourage the participant to follow the visual cue. These audible instructions must be identical if used in the structured training session.
5. Repeat the tasks three times with different arm positions (relaxed, reaching in front, reaching across) to enhance system training. Keep in mind that there are 8 different actions and three force levels, once all arm positions are covered, system training input sums up to the total of 72 individual samples.
6. Once complete, allow the participant the opportunity to practice real-time control prior to completing the SHAP outcome assessment.
7. Ensure the participant does not have access to the customized prosthesis and control algorithms beyond the end of the naïve session.

6. Structured Training Session

1. Three months after the naïve session, perform a structured training session.
2. Structure the session in the following ordered steps (Figure 3):
   1. For imitation, instruct the participant to directly imitate the desired eight actions (Table 1) performed by the demonstrator in real-time. Execute each action for 3 sec.
   2. For repetition, ask the participant to repeat the action that has been imitated 10 times, so each action is performed for 30 sec.
   3. For reinforcement & computer system training, ask the participant to now engage with the computer’s visual feedback, that is exactly the same setup as the Naïve session. Ensure that there is no difference between these two sections.
   4. For prosthetic control, ask the participant to practice real-time control of the customized prosthesis before completing the outcome assessment.
3. During imitation, seat the participant at a 45° angle from the demonstrator and provide with a full and unobstructed view of the demonstrator’s hand matching the affected side of the participant during all actions (Figure 4). No visual cues from a computer screen should be available to the participant at this time.
4. For repetition, during the participant’s actions, have the demonstrator observe the corresponding EMG activity as represented by the polar plots of each movement (Figure 4). Once the demonstrator has determined that the participant can produce unique and repeatable EMG activation patterns for each movement, ask the participant to repeat the actions for 30 sec without any visual cues.
   Note: There are total of 8 unique actions — seven of them (wrist pronation/supination, wrist flexion/extension, hand open, key grip and fine pinch) requiring muscle activation, and the eighth being no action which represents a steady relaxed state.
5. After reinforcement & computer system training, present the participant with visual feedback of his eight actions, exactly as was seen in the naïve session, which correspond to the eight unique and specific polar plots on the computer screen (Figure 3). To tune performance, ask the participant to perform the actions while viewing the real-time polar plots with recorded motion overlays to reinforce learning, typically between 2-4 attempts for each movement. Once confident the participant can then complete the exact same tasks that were performed in the naïve session.

7. Prosthetic Control

1. Use the training data sets from each session to calibrate and adjust the prosthesis for real time control.
2. Initially, only allow the participant to control the prostheses by sequential proportional control, i.e., one movement at a time, with the speed of the device proportional to the levels of muscular contractions.
3. Once each of the eight actions are performed in a repeatable and reliable manner, switch the control scheme to proportional and simultaneous control, allowing more than one movement of the wrist at a time.
4. Have the participant practice simple tasks, such as picking up a bottle and laying it on its side (2 attempts is sufficient). Allow a period of rest before the outcome assessment is performed. In the case of this study, 2 hr of rest for the naïve session, and 24 hr of rest for the structured session.

8. Outcome Measurement

1. Evaluate global upper extremity function both in the naïve and structured training sessions using the SHAP, which monitors hand and upper extremity function closely related to activities of daily living (ADLs). The tasks performed in the SHAP include manipulating light and heavy objects, as well as tasks of ADL such as cutting an object with a knife or undoing buttons. The SHAP has been validated for assessment of pathological and prosthetic hand function.²⁴
   Note: This measurement was chosen as the participant in this study had been routinely followed up with this outcome measure by his clinical team.

Results

The baseline SHAP performance of the participant with his daily prosthesis was 81 when measured by the clinical staff 8 months prior to testing. A SHAP score of 100 represents able-bodied hand function.²⁴ The participant scored an overall SHAP score of 58 during the naïve session with the more advanced prosthesis control system. However, 3 months later and with no further interaction with the new system, aside from the structured training, the participant achieved a SHAP score of 71 with the same advanced syst...

Discussion

Our findings suggest for the participant in this study that structured training helped improve control of a multifunctional prosthetic hand during one single session. The structured program used here was a combination of imitation, repetition and reinforcement of hand movements that the participant was not able to complete with his traditional prosthetic hand.

Although the participant scored higher with his traditional prosthesis in the SHAP test, it is worth noting that he typically wore that...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Michelangelo Hand	Otto Bock Healthcare Products GmbH, A	8E500=L-M
AxonRotation	Otto Bock Healthcare Products GmbH, A	9S503
Wrist Flexor	Otto Bock Healthcare Products GmbH, A		prototype unit
AxonMaster	Otto Bock Healthcare Products GmbH, A	13E500
Electrode	Otto Bock Healthcare Products GmbH, A	13E200=50AC
ScissorFenceElectrodeCarrier	Otto Bock Healthcare Products GmbH, A		prototype unit
Acquisition Software	Otto Bock Healthcare Products GmbH, A		prototype unit
Carbon shaft	Otto Bock Healthcare Products GmbH, A		prototype unit