Using a robotic isokinetic device with electromyography (EMG) measurements, this protocol illustrates that isokinetic motion itself can improve inter-rater reliability for the angle of catch measurements in stroke patients with mild elbow flexor spasticity.
Measuring spasticity is important in treatment planning and determining efficacy after treatment. However, the current tool used in clinical settings has been shown to be limited in inter-rater reliability. One factor in this poor inter-rater reliability is the variability of passive motion while measuring the angle of catch (AoC) measurements. Therefore, an isokinetic device has been proposed to standardize the manual joint motion; however, the benefits of isokinetic motion for AoC measurements has not been tested in a standardized manner. This protocol investigates whether isokinetic motion itself can improve inter-rater reliability for AoC measurements. For this purpose, a robotic isokinetic device was developed that is combined with surface electromyography (EMG). Two conditions, manual and isokinetic motions, are compared with the standardized method to measure the angle and subjective feeling of catch. It is shown that in 17 stroke patients with mild elbow flexor spasticity, isokinetic motion improved the intraclass correlation coefficient (ICC) for inter-rater reliability of AoC measurements to 0.890 [95% confidence interval (CI): 0.685–0.961] by the EMG criteria, and 0.931 (95% CI: 0.791–0.978) by the torque criteria, from 0.788 (95% CI: 0.493–0.920) by manual motion. In conclusion, isokinetic motion itself can improve inter-rater reliability of AoC measurements in stroke patients with mild spasticity. Given that this system may provide greater standardized angle measurements and catch of feeling, it may be a good option for the evaluation of spasticity in a clinical setting.
Spasticity after stroke is common and has been shown to induce complications, including pain and contractures, resulting in reduced quality of life1,2,3. Measurement of spasticity is important to properly plan the course of treatment and determine the efficacy of treatment. Commonly used tools in the clinical setting are the Modified Ashworth scale (MAS)4, which is a nominal measurement system for resistance to passive movement, and the modified Tardieu scale (MTS), which measures the angle of catch (AoC), representing the velocity-dependent characteristic of spasticity5. However, these measurement tools have been shown to have limited inter-rater reliability6,7, requiring the same rater to perform these tests to maintain satisfactory reliability8.
Three factors have been shown induce variability in AoC during MTS measurement, including (1) errors from angle measurements by a goniometry; (2) variability of manually moved joint motion profile between raters; and (3) variability in sensing the catch between raters9. A novel isokinetic robotic device with torque sensors is presented in this protocol. This device is applied to stroke patients with mild elbow flexor spasticity using surface electromyography (EMG) measurements10. It was hypothesized that the standardization of elbow joint motion will improve inter-rater reliability for AoC measurements elicited by the elbow flexor stretch reflex. To prove this, the reliability for AoC as measured by surface EMG was calculated and compared between the isokinetic passive and manual fast elbow extension, using this developed robotic device and EMG. Figure 1 shows an overview of the entire experimental procedure. In detail, the MTS measurement stage was conducted by two raters, and the order of experiments (manual vs. isokinetic motion) and order of raters were randomly determined, which required about 50 min for each subject (Figure 1).
1. Experimental set-up
2. Experimental set-up
NOTE: Two raters should participate in this experiment. In our case, the first rater was a physiatrist with more than 6 years of experience in rehabilitation, and the second rater was an occupational therapist with more than 3 years of experience in stroke rehabilitation.
3. MTS measurement
NOTE: The time required for each step is shown in Figure 1. The total time taken by one subject to perform all the experiment is about 50 min (including the experiment set-up step), but most of the time should be spent resting to maintain consistency of fatigue.
4. Quantifying the AoC
NOTE: AoC is determined based on two data: EMG and torque. AoC is determined by manual analysis due to the noisy characteristics of the EMG data and variability of individual characteristics. The AoC selection is carried out by a third rater, who is blind to the order of raters.
5. Data analysis
The reliability is divided into four grades according to the ICC value: extremely excellent (>0.90), excellent (0.75 < ICC ≤ 0.90), fair to good (0.40 < ICC ≤ 0.75), and poor (<0.40). The standard error of measurements (SEM) was calculated to determine the error component of the variance. The smallest detectable difference (SDD) was calculated from the SEM of test-retest data.
Normalized assessment motion index (NAMI): the NAMI score during an isokinetic motion was always 1, which means that the isokinetic device always generated a uniform constant input velocity. However, the test-retest reliability of the NAMI during a manual motion was poor for both rater 1 (ICC [95% CI] = -0.035 [-0.495–0.441]) and rater 2 (ICC [95%CI] = 0.438 [-0.038–0.752]). Moreover, the inter-rater reliability of the NAMI during manual motion was also poor (ICC [95% CI] = 0.148 [-0.344–0.576]). Conversely, the results of the two human raters showed almost equal averaged NAMI values (0.68 and 0.67 for each rater). The consistency error of the two human raters was larger than that of the isokinetic device, showing a large difference between the two raters. These results indicate that an assessment motion by a human rater is lacking in the isokinetic characteristics and that motion is inconsistent depending on the subject.
Test-retest reliability: Table 2 shows test-retest reliability for the AoC results in three conditions (isokinetic-EMG, isokinetic-torque, manual-EMG). The test-retest reliability for manual MTS was excellent (ICC = 0.804 and 0.840). However, the isokinetic MTS measurement improved test-retest reliability to the extremely excellent grade on both the EMG and torque criteria (Table 2)
Inter-rater reliability: Table 3 shows inter-rater reliability for the AoC measurement performance in three conditions. The ICC of the inter-rater reliability of the manual MTS was 0.788, which was near the lower limit of the excellent grade. The isokinetic MTS improved inter-rater reliability to the ICC of 0.890 based on EMG data and to the ICC of 0.931 based on torque data.
Correlations and consistency of timing of AoC between the EMG and torque criteria: the two AoC results calculated from the EMG data and torque data during the isokinetic MTS show a significantly high correlation in both rater 1 (Pearson correlation coefficient = 0.937, p < 0.001) and rater 2 (Pearson correlation coefficient = 0.957, p < 0.001). Moreover, the timing of AoC between the two results was highly consistent with an ICC of 1 (p < 0.001).
Figure 1: Experiment flow chart.
This figure is modified from Sin et al.10. Please click here to view a larger version of this figure.
Figure 2: Isokinetic MTS test robot.
(A) Configuration of the isokinetic robot device. (B) Inside configuration of the device. The control system includes a real-time processor and motor driver. (B) was previously published by Sin et al.10. Please click here to view a larger version of this figure.
Figure 3: Composition of the manipulandum.
Two cuffs for the wrist and forearm are connected to the linear slider through a fixation block, making the position of the cuff adjustable. A handle and hand strap are switchable from left-to-right. Please click here to view a larger version of this figure.
Figure 4: Control system Configuration.
The right three blocks show the hierarchy of the control system and arrows show the data flow between each unit. Please click here to view a larger version of this figure.
Figure 5: Graphic user interface (GUI).
The left side is the controller panel, which contains the various buttons or numeric controls required for robot control. The right side is a monitoring panel that shows the angle, interaction torque, and trigger signal in real-time. Please click here to view a larger version of this figure.
Figure 6: Example of inertia effect compensation.
The green line indicates the raw torque; the blue dotted line indicates the inertial force model; and the red line indicates the inertial torque compensation result. This figure was previously published by Sin et al.10. Please click here to view a larger version of this figure.
Figure 7: Example of AoC evaluation using EMG data (isokinetic MTS case).
An RMS EMG value of less than 0.1 is regarded as normal. Selection of the starting point of the clear EMG upsurge point is performed, and the angle value at that time is determined as AoC. This figure was previously published by Sin et al.10. Please click here to view a larger version of this figure.
Figure 8: Example of AoC evaluation using torque data (isokinetic MTS case).
Evaluation involves the following steps: draw two lines connecting the torque of the assessment starting point and the end point with an arbitrary torque data, respectively; find the point where the two lines become the regression line of the torque data before and after the selected point; if there is a significant difference between the gradient of two regression line, it is judged that a stretch reflex occurs at this point. This figure was previously published by Sin et al.10. Please click here to view a larger version of this figure.
Figure 9: Example of AoC evaluation using EMG data (manual MTS case).
As done in the isokinetic case (Figure 7), the AoC is determined as the angle when a clear upsurge of the EMG occurs. This figure was previously published by Sin et al.10. Please click here to view a larger version of this figure.
Figure 10: Variables for the normalized assessment motion index (NAMI).
Intuitively, the NAMI value is the ratio of the area under the velocity graph to the area of the gray box. More isokinetic movements show values closer to 1. This figure is previously published by Sin et al.10. Please click here to view a larger version of this figure.
Variable | Result |
Age, years, mean (SD) | 54.6 (12.2) |
Gender, n (%) | |
Men | 14 (82.4) |
Women | 3 (17.6)) |
Days from stroke onset, median (IQR) | 722 (1226) |
Hemiplegic side, n (%) | |
Right | 10 (58.8) |
Left | 7 (41.2) |
Stroke type, n (%) | |
Ischemic | 11 (64.7) |
Hemorrhagic | 6 (35.3) |
Stroke lesion, n (%) | |
Cortical | 4 (23.5) |
Subcortical | 13 (76.5) |
Brunnstrom stage, median (IQR) | |
Arm | 4 (1) |
Hand | 3 (1) |
Leg | 4 (1) |
Muscle Power, median (IQR) | |
Elbow flexor | 4 (1) |
Elbow extensor | 4 (1) |
MAS, elbow flexor, n (%) | |
1 | 7 (41.2) |
1+ | 5 (29.4) |
2 | 5 (29.4) |
Table 1: Subjects demographics and baseline characteristics.
Test | Retest | p | SEM | SDD | ICC (2,1) (95% CI) | |
Mean (SD) | Mean (SD) | |||||
Rater 1 | ||||||
Isokinetic (150°/s) motion with EMG | 93.74 (28.35) | 90.93 (25.44) | 0.216 | 12.12 | 33.59 | 0.948 (0.857-0.981) |
Isokinetic (150°/s) motion with torque | 90.30 (27.93) | 89.61 (27.25) | 0.201 | 3.02 | 8.37 | 0.997 (0.992-0.996) |
Manual motion with EMG | 82.67 (19.11) | 82.03 (21.73) | 0.838 | 17.21 | 47.7 | 0.804 (0.538-0924) |
Rater 2 | ||||||
Isokinetic (150°/s) motion with EMG | 90.77 (28.69) | 88.14 (28.34) | 0.123 | 15.1 | 41.86 | 0.929 (0.929-0.991) |
Isokinetic (150°/s) motion with torque | 97.06 (23.47) | 94.37 (25.86) | 0.192 | 9.9 | 27.44 | 0.959 (0.873-0.987) |
Manual motion with EMG | 80.96 (21.30) | 80.46 (22.81) | 0.875 | 16.94 | 46.96 | 0.840 (0.601-0.941) |
Table 2: Test-retest reliability results for the angle of catch measured with isokinetic robotic devices and robotic devices with manual motion.
This table was published by Sin et al.10 (p-values are calculated by paired sample t-test). SEM: standard error of measurement, SDD: smallest detectable difference, ICC: intraclass correlation coefficient, EMG: electromyography.
Rater 1 | Rater 2 | p | SEM | ICC (2,1) (95% CI) | |
Mean (SD) | Mean (SD) | ||||
Isokinetic (150°/s) motion with EMG | 88.16 (28.24) | 89.46 (28.33) | 0.973 | 17.81 | 0.890 (0.685-0.961) |
Isokinetic (150°/s) motion with torque | 94.32 (240.13) | 95.71 (24.44) | 0.775 | 12.54 | 0.931 (0.791-0.978) |
Manual motion with EMG | 80.81 (18.98) | 80.71 (21.17) | 0.586 | 17.5 | 0.788 (0.493-0.920) |
Table 3: Inter-rater reliability results for the angle of catch measured with isokinetic robotic devices and robotic devices with manual motion.
This table was published by Sin et al.10 (p-values are calculated by paired sample t-test). SEM: standard error of measurement, ICC: intraclass correlation coefficient, EMG: electromyography.
Appendix. Please click here to download this file.
This study attempted to standardize the MTS measurement using a robotic isokinetic device. It was investigated how the consistency of assessment motion affects the results of MTS measurement.
The NAMI value was proposed to represent the degree of variability in assessment motion. As expected, unlike the isokinetic motion method with no variability, the manual method showed variability between tests and between raters, resulting in poor reliability, which is consistent with results from previous studies7,8. The results on reliability for AoC measurement show that isokinetic motion itself can increase interrater reliability, compared to manual motion. Although, there have been concerns regarding the less stretch reflex provocation by the isokinetic motion11,12, subjects in this study with mild elbow flexor spasticity (MAS 1, 1+, 2) showed consistent stretch reflexes measured by surface EMG during isokinetic motion. This demonstrates that an isokinetic device can be used to measure AoC reliably, even in patients with mild elbow spasticity. AoC was also calculated by the torque criteria in this study. Interestingly, AoC measured by using both the EMG and torque criteria showed a high correlation, while the torque criteria alone showed a higher inter-rater reliability, which is consistent with the results provided by Lynn et al.13. Therefore, spasticity evaluation using the torque criteria is expected to be a better method with respect to reliability and convenience.
This new approach for quantifying the MTS measurement has some issues and limitations. First, the posture during AoC measurements in this study was different from conventional MTS measurements14. The conventional MTS was performed in the absence of shoulder abduction; in contrast, in this study, measurements were performed with the shoulder abducted 90 degrees. However, the purpose of this study was to verify the effects of consistency of the assessment motion on the AoC reliability. The posture used in this experiment makes it easy to measure AoC using the torque data by eliminating the influence of forearm weight, which is difficult to measure separately. Therefore, this experiment provides a perspective on how the assessment motion affects the reliability of AoC measurements.
Second, the AoC measurement using both the torque and EMG criteria was performed subjectively. However, this was conducted by a third rater who was blind to the subject information and order of raters to minimize potential bias. Third, the increase of the reaction torque due to passive mechanical properties was unexpected when designing the experiment initially. It was expected that the reaction torque is mainly caused by stretch reflex; however, in patients with mild spasticity, many cases showed that the reaction torque caused by passive stiffness was dominant. Therefore, AoC was obtained through post-experimental data analysis rather than real-time identification. Finally, there was relaxation of the elbow flexor during repetitive passive stretching. The experiment was designed to incorporate sufficient resting time to prevent fatigue throughout the experiment, and no subjects complained of fatigue. However, it is hard to prevent relaxation of the muscle due to repetitive passive stretching. To reduce this impact, the experiment was designed to randomize the order of raters, and the results showed no significant relaxation phenomenon between the two raters.
The goal of this study was to improve evaluation methods that rely on the subjective sense of the rater and hold them to more objective and quantitative standards. The results show the possibility of increasing assessment reliability using a robotic device. However, the method performed in this study is only half-automated, because the AoC evaluation is done by a human. It is expected that the further studies will enable real-time spasticity evaluation with high reliability and objectivity.
The authors have nothing to disclose.
This study was supported by the Seoul National University Bundang Hospital Research Fund (14- 2014 – 035) and Korea and National Research Foundation of Korea (NRF) Grant funded by the Korean Government (A100249). We would like to thank Seo Hyun Park and Hae-in Kim for helping to prepare and proceed with shooting video.
3D printer | Lokit | 3Dison+ | FDA type 3D printer |
Ball sprine shaft | Misumi | LBF15 | |
Bridge Analog Input module | National Instruments | NI 9237 | |
CAN communication module | National Instruments | NI 9853 | |
Caster | Misumi | AC-50F | |
Electromyography (EMG) device | Laxtha | WEMG-8 | |
EMG electrode | Bioprotech | 1.8×1.2 mm Ag–AgCl | |
Encoder | Maxon | HEDL 9140 | 500 CPT |
Gearbox | Maxon | GP 81 | 51:1 ratio |
Lab jack | Misumi | 99-1620-20 | |
Linear slider | Misumi | KSRLC16 | |
Motor | Maxon | EC-60 | brushless EC motor |
Motor driver | Elmo | DC Whistle | |
PLA | Lokit | 3D printer material | |
Real-time processor | National Instruments | sbRIO-9632 | |
Torque sensor | Transducer Techniques | TRS-1K |