One of the hottest topics in rehabilitation robotics is that of proper control of prosthetic devices. Despite decades of research, the state of the art is dramatically behind the expectations. To shed light on this issue, in June, 2013 the first international workshop on Present and future of non-invasive peripheral nervous system (PNS)-Machine Interfaces (MI; PMI) was convened, hosted by the International Conference on Rehabilitation Robotics. The keyword PMI has been selected to denote human-machine interfaces targeted at the limb-deficient, mainly upper-limb amputees, dealing with signals gathered from the PNS in a non-invasive way, that is, from the surface of the residuum. The workshop was intended to provide an overview of the state of the art and future perspectives of such interfaces; this paper represents is a collection of opinions expressed by each and every researcher/group involved in it.
This study describes the first application of a support vector machine (SVM) based scheme for real-time simultaneous and proportional myoelectric control of multiple degrees of freedom (DOFs). Three DOFs including wrist flexion-extension, abduction-adduction and forearm pronation-supination were investigated with 10 able-bodied subjects and two individuals with transradial limb deficiency (LD). A Fitts' law test involving real-time target acquisition tasks was conducted to compare the usability of the SVM-based control system to that of an artificial neural network (ANN) based method. Performance was assessed using the Fitts' law throughput value as well as additional metrics including completion rate, path efficiency and overshoot. The SVM-based approach outperformed the ANN-based system in every performance measure for able-bodied subjects. The SVM outperformed the ANN in path efficiency and throughput with the first LD subject and in throughput with the second LD subject. The superior performance of the SVM-based system appears to be due to its higher estimation accuracy of all DOFs during inactive and low amplitude segments (these periods were frequent during real-time control). Another advantage of the SVM-based method was that it substantially reduced the processing time for both training and real time control.
Previous studies on intramuscular EMG based control used offline data analysis. The current study investigates the usability of intramuscular EMG in two degree-of-freedom using a Fitts' Law approach by combining classification and proportional control to perform a task, with real time feedback of user performance. Nine able-bodied subjects participated in the study. Intramuscular and surface EMG signals were recorded concurrently from the right forearm. Five performance metrics (Throughput,Path efficiency, Average Speed, Overshoot and Completion Rate) were used for quantification of usability. Intramuscular EMG based control performed significantly better than surface EMG for Path Efficiency (80.5±2.4% vs. 71.5±3.8%, P=0.004) and Overshoot (22.0±3.0% vs. 45.1±6.6%, P=0.01). No difference was found between Throughput and Completion Rate. However the Average Speed was significantly higher for surface (51.8±5.5%) than for intramuscular EMG (35.7±2.7%). The results obtained in this study imply that intramuscular EMG has great potential as control source for advanced myoelectric prosthetic devices.
The performance of pattern recognition based myoelectric control has seen significant interest in the research community for many years. Due to a recent surge in the development of dexterous prosthetic devices, determining the clinical viability of multifunction myoelectric control has become paramount. Several factors contribute to differences between offline classification accuracy and clinical usability, but the overriding theme is that the variability of the elicited patterns increases greatly during functional use. Proportional control has been shown to greatly improve the usability of conventional myoelectric control systems. Typically, a measure of the amplitude of the electromyogram (a rectified and smoothed version) is used to dictate the velocity of control of a device. The discriminatory power of myoelectric pattern classifiers, however, is also largely based on amplitude features of the electromyogram. This work presents an introductory look at the effect of contraction strength and proportional control on pattern recognition based control. These effects are investigated using typical pattern recognition data collection methods as well as a real-time position tracking test. Training with dynamically force varying contractions and appropriate gain selection is shown to significantly improve (p<0.001) the classifiers performance and tolerance to proportional control.
In this paper, the predictive capability of surface and untargeted intramuscular electromyography (EMG) was compared with respect to wrist-joint torque to quantify which type of measurement better represents joint torque during multiple degrees-of-freedom (DoF) movements for possible application in prosthetic control. Ten able-bodied subjects participated in the study. Surface and intramuscular EMG was recorded concurrently from the right forearm. The subjects were instructed to track continuous contraction profiles using single and combined DoF in two trials. The association between torque and EMG was assessed using an artificial neural network. Results showed a significant difference between the two types of EMG (P < 0.007) for all performance metrics: coefficient of determination (R(2)), Pearson correlation coefficient (PCC), and root mean square error (RMSE). The performance of surface EMG (R(2) = 0.93 ± 0.03; PCC = 0.98 ± 0.01; RMSE = 8.7 ± 2.1%) was found to be superior compared with intramuscular EMG (R(2) = 0.80 ± 0.07; PCC = 0.93 ± 0.03; RMSE = 14.5 ± 2.9%). The higher values of PCC compared with R(2) indicate that both methods are able to track the torque profile well but have some trouble (particularly intramuscular EMG) in estimating the exact amplitude. The possible cause for the difference, thus the low performance of intramuscular EMG, may be attributed to the very high selectivity of the recordings used in this study.
Time and frequency domain features of the surface electromyogram (EMG) signal acquired from multiple channels have frequently been investigated for use in controlling upper-limb prostheses. A common control method is EMG-based motion classification. We propose the use of EMG signal whitening as a preprocessing step in EMG-based motion classification. Whitening decorrelates the EMG signal and has been shown to be advantageous in other EMG applications including EMG amplitude estimation and EMG-force processing. In a study of ten intact subjects and five amputees with up to 11 motion classes and ten electrode channels, we found that the coefficient of variation of time domain features (mean absolute value, average signal length and normalized zero crossing rate) was significantly reduced due to whitening. When using these features along with autoregressive power spectrum coefficients, whitening added approximately five percentage points to classification accuracy when small window lengths were considered.
This study describes a novel myoelectric control scheme that is capable of motion rejection. As an extension of the commonly used linear discriminant analysis (LDA), this system generates a confidence score for each decision, providing the ability to reject those with a score below a selected threshold. The thresholds are class-specific and affect only the rejection characteristics of the associated class. Furthermore, because the rejection stage is implemented using the outputs of the LDA, the active motion classification accuracy of the proposed system is shown to outperform that of the LDA for all values of rejection threshold. The proposed scheme was compared to a baseline LDA-based pattern recognition system using a real-time Fitts law-based target acquisition task. The use of velocity-based myoelectric control using the rejection classifier is shown to obey Fitts law, producing linear regression fittings with high coefficients of determination (R(2) > 0.943). Significantly higher (p < 0.001) throughput, path efficiency, and completion rates were observed with the rejection-capable system for both able-bodied and amputee subjects.
Using electromyogram (EMG) signals to control upper-limb prostheses is an important clinical option, offering a person with amputation autonomy of control by contracting residual muscles. The dexterity with which one may control a prosthesis has progressed very little, especially when controlling multiple degrees of freedom. Using pattern recognition to discriminate multiple degrees of freedom has shown great promise in the research literature, but it has yet to transition to a clinically viable option. This article describes the pertinent issues and best practices in EMG pattern recognition, identifies the major challenges in deploying robust control, and advocates research directions that may have an effect in the near future.
Prosthetic wrists need to be compact. By minimizing space requirements, a wrist unit can be made for people with long residual limbs. This prosthetic wrist uses two motors arranged across the arm within the envelope of the hand. The drive is transmitted by a differential so that it produces wrist flexion and extension, pronation and supination, or a combination of both. As a case study, it was controlled by a single-prosthesis user with pattern recognition of the myoelectric signals from the forearm. The result is a compact, two-degree-of-freedom prosthetic wrist that has the potential to improve the functionality of any prosthetic hand by creating a hand orientation that more closely matches grasp requirements.
Reported studies on pattern recognition of electromyograms (EMG) for the control of prosthetic devices traditionally focus on classification accuracy of signals recorded in a laboratory. The difference between the constrained nature in which such data are often collected and the unpredictable nature of prosthetic use is an example of the semantic gap between research findings and a viable clinical implementation. In this paper, we demonstrate that the variations in limb position associated with normal use can have a substantial impact on the robustness of EMG pattern recognition, as illustrated by an increase in average classification error from 3.8% to 18%. We propose to solve this problem by: 1) collecting EMG data and training the classifier in multiple limb positions and by 2) measuring the limb position with accelerometers. Applying these two methods to data from ten normally limbed subjects, we reduce the average classification error from 18% to 5.7% and 5.0%, respectively. Our study shows how sensor fusion (using EMG and accelerometers) may be an efficient method to mitigate the effect of limb position and improve classification accuracy.
Recent literature in pattern recognition-based myoelectric control has highlighted a disparity between classification accuracy and the usability of upper limb prostheses. This paper suggests that the conventionally defined classification accuracy may be idealistic and may not reflect true clinical performance. Herein, a novel myoelectric control system based on a selective multiclass one-versus-one classification scheme, capable of rejecting unknown data patterns, is introduced. This scheme is shown to outperform nine other popular classifiers when compared using conventional classification accuracy as well as a form of leave-one-out analysis that may be more representative of real prosthetic use. Additionally, the classification scheme allows for real-time, independent adjustment of individual class-pair boundaries making it flexible and intuitive for clinical use.
A rhesus monkey was trained to perform individuated and combined finger flexions of the thumb, index, and middle finger. Nine implantable myoelectric sensors (IMES) were then surgically implanted into the finger muscles of the monkeys forearm, without any adverse effects over two years postimplantation. Using an inductive link, EMG was wirelessly recorded from the IMES as the monkey performed a finger flexion task. The EMG from the different IMES implants showed very little cross correlation. An offline parallel linear discriminant analysis (LDA) based algorithm was used to decode finger activity based on features extracted from continuously presented frames of recorded EMG. The offline parallel LDA was run on intraday sessions as well as on sessions where the algorithm was trained on one day and tested on following days. The performance of the algorithm was evaluated continuously by comparing classification output by the algorithm to the current state of the finger switches. The algorithm detected and classified seven different finger movements, including individual and combined finger flexions, and a no-movement state (chance performance = 12.5%) . When the algorithm was trained and tested on data collected the same day, the average performance was 43.8+/-3.6% n=10. When the training-testing separation period was five months, the average performance of the algorithm was 46.5+/-3.4% n=8. These results demonstrated that using EMG recorded and wirelessly transmitted by IMES offers a promising approach for providing intuitive, dexterous control of artificial limbs where human patients have sufficient, functional residual muscle following amputation.
This paper describes a novel pattern recognition based myoelectric control system that uses parallel binary classification and class specific thresholds. The system was designed with an intuitive configuration interface, similar to existing conventional myoelectric control systems. The system was assessed quantitatively with a classification error metric and functionally with a clothespin test implemented in a virtual environment. For each case, the proposed system was compared to a state-of-the-art pattern recognition system based on linear discriminant analysis and a conventional myoelectric control scheme with mode switching. These assessments showed that the proposed control system had a higher classification error ( p < 0.001) but yielded a more controllable myoelectric control system ( p < 0.001) as measured through a clothespin usability test implemented in a virtual environment. Furthermore, the system was computationally simple and applicable for real-time embedded implementation. This work provides the basis for a clinically viable pattern recognition based myoelectric control system which is robust, easily configured, and highly usable.
The electromyogram (EMG) signal has been used as the command input to myoelectric prostheses. A common control scheme is based on classifying the EMG signals from multiple electrodes into one of several distinct classes of user intent/function. In this work, we investigated the use of EMG whitening as a preprocessing step to EMG pattern recognition. Whitening is known to decorrelate the EMG signal, with improved performance shown in the related applications of EMG amplitude estimation and EMG-torque processing. We reanalyzed the EMG signals recorded from 10 electrodes placed circumferentially around the forearm of 10 intact subjects and 5 amputees. The coefficient of variation of two time-domain features--mean absolute value and signal length--was significantly reduced after whitening. Pre-whitened classification models using these features, along with autoregressive power spectrum coefficients, added approximately five percentage points to their classification accuracy. Improvement was best using smaller window durations (<100 ms).
When controlling a powered upper limb prosthesis it is important not only to know how to move the device, but also when not to move. A novel approach to pattern recognition control, using a selective multiclass one-versus-one classification scheme has been shown to be capable of rejecting unintended motions. This method was shown to outperform other popular classification schemes when presented with muscle contractions that did not correspond to desired actions. In this work, a 3-D Fitts Law test is proposed as a suitable alternative to using virtual limb environments for evaluating real-time myoelectric control performance. The test is used to compare the selective approach to a state-of-the-art linear discriminant analysis classification based scheme. The framework is shown to obey Fitts Law for both control schemes, producing linear regression fittings with high coefficients of determination (R(2) > 0.936). Additional performance metrics focused on quality of control are discussed and incorporated in the evaluation. Using this framework the selective classification based scheme is shown to produce significantly higher efficiency and completion rates, and significantly lower overshoot and stopping distances, with no significant difference in throughput.
In this work, the simultaneous real-time control of multiple degrees of freedom (DOF) for myoelectric systems is investigated. The goal of this study, in which ten able-bodied subjects participated, was to directly compare three control paradigms of constrained (force targeted), unconstrained (position targeted) and resisted unconstrained (position targeted) limb contractions. Artificial neural networks (ANNs) were trained for simultaneous myoelectric control of the three DOFs (wrist flexion-extension, abduction-adduction and pronationsupination) using mirrored bilateral contractions. In the resisted unconstrained experiment, some resistance to movement was provided using flexible wrist braces in order to increase the required level of muscle activation. The force, in constrained experiments, and position, in unconstrained and resisted unconstrained experiments, were measured. The three protocols were compared offline using estimation accuracies (R2) and online using a real-time computer-based target acquisition test. The constrained control paradigm outperformed the unconstrained method in the abduction-adduction DOF (R2 constrained = 90.8 ± 0.6, R2 unconstrained = 85.6 ± 1.6) and pronationsupination DOF (R2 constrained = 88.5 ± 0.9, R2 unconstrained = 82.3 ± 1.6), but no significant difference was found in the flexionextension DOF. The constrained control method outperformed unconstrained control in two real-time testing metrics including completion time and path efficiency. The constrained method results, however, were not significantly different than those of the resisted unconstrained method (with braces) in both offline and real-time tests. This suggests that the quality of control using constrained and unconstrained contractions based myoelectric schemes is not appreciably different when using comparable levels of muscle activation.
Related JoVE Video
Journal of Visualized Experiments
What is Visualize?
JoVE Visualize is a tool created to match the last 5 years of PubMed publications to methods in JoVE's video library.
How does it work?
We use abstracts found on PubMed and match them to JoVE videos to create a list of 10 to 30 related methods videos.
Video X seems to be unrelated to Abstract Y...
In developing our video relationships, we compare around 5 million PubMed articles to our library of over 4,500 methods videos. In some cases the language used in the PubMed abstracts makes matching that content to a JoVE video difficult. In other cases, there happens not to be any content in our video library that is relevant to the topic of a given abstract. In these cases, our algorithms are trying their best to display videos with relevant content, which can sometimes result in matched videos with only a slight relation.