Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity

Summary

Using a robotic isokinetic device with electromyography (EMG) measurements, this protocol illustrates that isokinetic motion itself can improve inter-rater reliability for the angle of catch measurements in stroke patients with mild elbow flexor spasticity.

Abstract

Measuring spasticity is important in treatment planning and determining efficacy after treatment. However, the current tool used in clinical settings has been shown to be limited in inter-rater reliability. One factor in this poor inter-rater reliability is the variability of passive motion while measuring the angle of catch (AoC) measurements. Therefore, an isokinetic device has been proposed to standardize the manual joint motion; however, the benefits of isokinetic motion for AoC measurements has not been tested in a standardized manner. This protocol investigates whether isokinetic motion itself can improve inter-rater reliability for AoC measurements. For this purpose, a robotic isokinetic device was developed that is combined with surface electromyography (EMG). Two conditions, manual and isokinetic motions, are compared with the standardized method to measure the angle and subjective feeling of catch. It is shown that in 17 stroke patients with mild elbow flexor spasticity, isokinetic motion improved the intraclass correlation coefficient (ICC) for inter-rater reliability of AoC measurements to 0.890 [95% confidence interval (CI): 0.685–0.961] by the EMG criteria, and 0.931 (95% CI: 0.791–0.978) by the torque criteria, from 0.788 (95% CI: 0.493–0.920) by manual motion. In conclusion, isokinetic motion itself can improve inter-rater reliability of AoC measurements in stroke patients with mild spasticity. Given that this system may provide greater standardized angle measurements and catch of feeling, it may be a good option for the evaluation of spasticity in a clinical setting.

Introduction

Spasticity after stroke is common and has been shown to induce complications, including pain and contractures, resulting in reduced quality of life^1,2,3. Measurement of spasticity is important to properly plan the course of treatment and determine the efficacy of treatment. Commonly used tools in the clinical setting are the Modified Ashworth scale (MAS)⁴, which is a nominal measurement system for resistance to passive movement, and the modified Tardieu scale (MTS), which measures the angle of catch (AoC), representing the velocity-dependent characteristic of spasticity⁵. However, these measurement tools have been shown to have limited inter-rater reliability^6,7, requiring the same rater to perform these tests to maintain satisfactory reliability⁸.

Three factors have been shown induce variability in AoC during MTS measurement, including (1) errors from angle measurements by a goniometry; (2) variability of manually moved joint motion profile between raters; and (3) variability in sensing the catch between raters⁹. A novel isokinetic robotic device with torque sensors is presented in this protocol. This device is applied to stroke patients with mild elbow flexor spasticity using surface electromyography (EMG) measurements¹⁰. It was hypothesized that the standardization of elbow joint motion will improve inter-rater reliability for AoC measurements elicited by the elbow flexor stretch reflex. To prove this, the reliability for AoC as measured by surface EMG was calculated and compared between the isokinetic passive and manual fast elbow extension, using this developed robotic device and EMG. Figure 1 shows an overview of the entire experimental procedure. In detail, the MTS measurement stage was conducted by two raters, and the order of experiments (manual vs. isokinetic motion) and order of raters were randomly determined, which required about 50 min for each subject (Figure 1).

Protocol

1. Experimental set-up

1. Patient recruitment
   NOTE: All procedures were reviewed and approved by the Seoul National University Bundang Hospital Institutional Review Board. These subjects were inpatients or outpatients with stroke diagnoses from four rehabilitation hospitals in the region.
   1. Perform the screening process using the following inclusion criteria: (1) upper extremity hemiparesis due to stroke; (2) over the age of 20 years; (3) mild elbow joint spasticity of MAS 1-2; (4) no previous disease affecting function of the hemiparetic arm, except for stroke; (5) free from hemodynamic instability; (6) no severe elbow contracture; (7) possibility for the shoulder to be abducted 90° and forearm to be in the neutral position without any joint pain; and (8) normal cognitive, language, visuospatial, or attention ability to follow experimental procedures.
      NOTE: The criteria are designed to screen patients who are able to participate in the experiment and regulate factors affecting the results.
   2. Recruit subjects who are provided with a detailed explanation of the entire study and expected clinical issues. Consent must be obtained prior to inclusion.
   3. Demographics and baseline characteristics of the recruited subjects are shown in Table 1.
2. Experimental system
   NOTE: A customized robotic device is used to produce standardized motion and measure the quantitative data simultaneously. The robotic system consists of a robotic part, control system, and measurement units. The overall configuration is shown in Figure 2.
   1. Robotic part
      1. For the robotic part, use a one-degree-of-freedom planar robot consisting of a motor and a forearm manipulandum, with three other components for adjusting the robot height and installing the device to various desks. The overall composition is shown in Figure 2A.
      2. For the forearm manipulandum, use an elbow joint connected to the motor, a linear slider with a fixation block to adjust the overall length and two cuff units for fixating the forearm and hand (as shown in Figure 3). The elbow joint has a rotating plate and a thrust bearing to prevent chafing during the experiment, and the cuff units were curved similar to that of a human forearm and were made using a 3D printer. The handle unit is designed to accommodate both left and right handed individuals, making it available to any subjects.
      3. Use a motor with a low gear ratio of 51:1, which should have back-drivable characteristics and the ability to produce a nominal speed of 315°/s and a continuous torque of 42.33 Nm.
      4. Use a lab jack unit that is attached to the bottom of the motor to adjust the height of the motor unit. The height of the robot will be able to be adjusted to the sitting height of various subjects.
      5. Place a fixation arm for mounting the device to the desk on the front part of the robot. The fixation arm will be movable up and down through a linear shaft and has clamps for securing to the desk.
      6. Place casters with stopper on the bottom of the robot, making the robot movable and grounded during the experiment.
   2. Control system
      1. Use a personal computer (PC), real-time processor, and motor driver for the central control system. The detailed control architecture block diagram is shown in Figure 4.
      2. Use a graphic user interface (GUI) to control the experiment mode (maximum ROM measurement, Isokinetic MTS, and manual MTS measurement modes) and store robot motion data. It contains a control panel and a monitoring panel (Figure 5). Details on the GUI configuration are included in the appendix.
      3. Implement the robot control algorithm using a real-time processor. The control algorithm consists of three control loops. The first loop is a data input/output loop that runs at 1 MHz from the FPGA module of the sbRIO. The second is a robot motion control loop that runs at 1 kHz from the real-time VI level. The last is a data communication loop that runs at 250 Hz. This loop transmits robot data (time, angle, torque and trigger signal for matching with EMG data).
         NOTE: The real-time processor has two communication modules: NI-9237 and NI-9853. The NI-9237 is an analog input device for receiving torque sensor data, and the NI-9853 is a CAN communication module for communicating with the motor driver.
   3. Measurement units
      1. Mount a torque sensor between the manipulandum and the motor to measure the reaction force. The torque data is transferred to the real-time processor via NI-9237. The NI-9237 has its own passband, stopband, and alias-free bandwidth filter. The filtered data enters the FPGA module and is processed again at 100 Hz with a low-pass filter to remove noises.
      2. Measure the joint angle by an encoder (HEDL 9140, Maxon, Switzerland) attached to the motor. The angle data is transferred to the real-time processor via the motor driver.
      3. Measure muscle activity with an eight-channel surface EMG device. The EMG data was collected at a sampling rate of 1024 Hz, and initially processed with a bandpass filter (20–450 Hz) and a notch filter (60 Hz). The measured EMG data is transferred to the PC directly.

2. Experimental set-up

NOTE: Two raters should participate in this experiment. In our case, the first rater was a physiatrist with more than 6 years of experience in rehabilitation, and the second rater was an occupational therapist with more than 3 years of experience in stroke rehabilitation.

1. Initial posture setting
   1. Place the patient in a chair with his/her back in a straight posture.
   2. Secure both sides of the shoulder and abdomen with seatbelts to keep the shoulder position stable throughout the experiment.
   3. Place the subject’s hemiparetic arm lightly on the robot manipulandum without fastening the strap.
   4. Unfasten the fixation block of the linear slider so that the cuff can be moved freely on the slider and allow the subject’s hemiparetic arm to be placed on the robot manipulandum without fastening the straps.
   5. Adjust the height of the robot using the lab jack until the patient’s shoulder is abducted 90°. Confirm the abduction angle using a goniometer.
   6. Instruct the subject to hold the handle and fasten the hand to the handle with straps. Align the rotation axis of the robot and anatomical axis of the elbow joint.
   7. Flex and extend the elbow joint so that the position of the cuff can be readjusted naturally in an optimal position without generating resistance during the elbow movement. Then, fasten the fixation block to fix the position of the cuff and fasten the straps of the forearm cuff.
   8. Attach the surface EMG electrodes on the biceps brachii muscle in the hemiparetic arm.
2. Passive ROM measurement
   NOTE: Passive ROM is used as a boundary ROM in the following experiments to prevent problems caused by movement outside the patient’s operation range.
   1. Input the patient's hemiparetic side information into the program GUI (right or left).
   2. Set the elbow 90° flexed using a goniometer. Press the 90 deg set button on the GUI panel. This process matches the angle recognized by the robot with the actual human joint angle.
   3. Press the Finish set button on the GUI to switch the robot to the actuating state.
   4. Click the buttons on the Motor run panel on the left side of the GUI in order from top to bottom.
   5. Turn on the Angle set button and set the speed to 1°/s. Then click the run button. The robot will extend the elbow slowly at 1°/s from a 90° flexed posture until the reaction torque reaches a certain threshold level or extends by 170°.
      NOTE: In this experiment, the torque threshold was set to 0.6 Nm. This value is determined experimentally via a pilot study.
   6. The maximum extended angle is automatically stored as the maximum ROM.
   7. Change the speed to -1°/s and click the run button again. The robot flexes the elbow slowly until the reaction torque reaches the threshold level.
   8. The maximum flexed angle is automatically stored as the minimum ROM.

3. MTS measurement

NOTE: The time required for each step is shown in Figure 1. The total time taken by one subject to perform all the experiment is about 50 min (including the experiment set-up step), but most of the time should be spent resting to maintain consistency of fatigue.

1. Inertia effect compensation
   NOTE: Theoretically, there should be no inertia effect during the isokinetic motion. However, there may be an inertia effect at the beginning of the motion. The inertial force should be compensated to measure only the reaction force generated by a stretch reflex. Since the magnitude of the inertial force is different for each subject, a preliminary test for the inertial force compensation should be performed before the actual MTS measurement. An example result is shown in Figure 6.
   1. Click the Back button on the control panel. The robot will flex the elbow to minimum angle posture (maximally flexed posture).
   2. Set the speed to 150°/s and turn on the Inertia test button then the Run button. The robot will apply a short perturbation of 5° to the patient at a rate of 150°/s. The peak torque and period value of each trial are automatically stacked and displayed on the GUI panel.
   3. Repeat steps 3.1.2–3.1.3 two more times. Determine a proper peak torque value and period value from the measured data and enter the value on the program GUI. The compensation torque profile (τ_comp) is automatically generated based on Equation 1 below, where: a, represents the determined amplitude and λ represents the period.
      
      NOTE: The shape of the inertial torque is modeled as a raised cosine shape to reduce the calculation load. The compensation torque, which is designed for two periods due to the inertia effect, almost disappears after the second period. The amplitude of the second period is designed to be 15% of the first period.
2. Familiarization step
   1. Prior to the actual experiment, perform three training operations to familiarize the patient with sudden movements.
   2. Click the Back button on the panel. The robot will flex the elbow to the minimum angle posture.
   3. Click the Run button after informing the subject. The robot will extend the patient’s elbow at a rate of 150°/s, until the angle reaches to a maximum angle or the reaction torque reaches the threshold level.
   4. Repeat steps 3.2.2–3.2.3 two more times and take a 5 min rest before starting the test.
3. Isokinetic MTS measurement
   NOTE: The isokinetic MTS measurement is designed to implement an ideal MTS measurement condition. The robot produces accurate constant velocity motion at a predetermined speed (150°/s) until reaching the maximum ROM or until reaching a certain threshold of the reaction torque. The maximum ROM value is determined in step 2.2, and the torque threshold value is determined as 0.6 via previous pilot studies, which is sufficient for detecting stretch reflexes.
   1. Click the Back button to flex the elbow to the minimum angle posture.
   2. Click the Run button without informing the subject. The robot will extend the patient’s elbow at a rate of 150°/s until the angle reaches the maximum angle or the reaction torque reaches a certain threshold level. Time, angle, reaction torque, and trigger signal data are stored during the test.
   3. Take a 2 min break between sets and repeat steps 3.3.1–3.3.3 two more times.
   4. Take a 5 min rest after performing three sets.
4. Manual MTS measurement
   NOTE: The manual MTS measurement is designed to simulate the MTS measurement normally performed at actual medical sites. To compare the results of isokinetic MTS, the robotic device is used only as a quantitative measuring tool that removes the measurement error, and actual measurement operation is performed by a human rater. For this purpose, the robot only compensates the friction of the robot itself. Details of the friction removal is in the appendix.
   1. Click the Back button to flex the elbow to the minimum angle posture.
   2. Click the Free run button, and the robot operation will change to manual operation mode.
   3. Hold the handle of the manipulandum and stretch the subject’s arm. During operation, the rater should generate a constant speed of 150°/s.
   4. Turn off the Free run mode and take a 2 min break.
   5. Repeat steps 3.4.1–3.4.4 two more times.
5. Repeat MTS measurement
   1. Take a 10 min rest after finishing the whole experiment with the first rater.
   2. Change the rater (to the second rater) and repeat steps 3.3–3.4.

4. Quantifying the AoC

NOTE: AoC is determined based on two data: EMG and torque. AoC is determined by manual analysis due to the noisy characteristics of the EMG data and variability of individual characteristics. The AoC selection is carried out by a third rater, who is blind to the order of raters.

1. Isokinetic MTS experiment data analysis
   1. AoC evaluation using EMG data
      NOTE: Generally, AoC is determined as the angle at which the maximum peak value of the EMG occurs. However, a stretch reflex duration is different for each patient; thus, using the EMG maximum peak point as an AoC is expected to have low reliability. The time difference may not be large; however, the AoC error can be significant due to the fast assessment speed of the MTS method. Therefore, the angle at the start of EMG upsurge point is selected as the AoC.
      1. Process the raw EMG data using the root mean square (RMS) to smooth the data and amplify it 50x.
      2. Synchronize the EMG data and robot angle data using the trigger signals of each data set.
         NOTE: In this system, the EMG data is measured by an independent device, unlike other data; therefore, the reference time may be different. The EMG device has a trigger interrupt marking function, which obtains the trigger signal from a real-time processor at the onset of MTS assessment.
      3. Determine the AoC manually as the starting point of the RMS EMG upsurge. The example is shown in Figure 7.
         NOTE: The RMS EMG of <0.1 is ignored here because it appears frequently even without the stretch reflex. Thus, a clear upsurge point at the start of the peak is selected as the AoC.
   2. AoC evaluation using the torque data
      NOTE: Muscles have passive mechanical characteristics that acts like a spring-damper system. Even if the muscle does not exert any force, the reaction force can increase as the muscles stretch. Because the intensity of the passive mechanical property and stretch reflex varies from patient to patient, it is difficult to identify the catch using only the absolute value of the reaction force. Instead, in this study, the catch is determined by changing the passive property due to spasticity rather than the absolute value of the reaction force. The change of the passive property is manually determined by the change in the slope of the regression line of the reaction torque.
      1. Draw one regression line from the point where the trigger signal goes up and draw another regression line from the point where the trigger signal goes down.
      2. Compare the slopes of the two regression lines. If the gradients of two regression lines show a significant difference, AoC can be determined at the intersection of two regression lines. The example is shown in Figure 8.
2. Manual MTS experiment data analysis
   NOTE: In the case of manual MTS, it is difficult to separate the force exerted by the subject and that applied by the rater using only one torque sensor. Therefore, in the case of manual MTS, only an AoC analysis using EMG data is performed without performing AoC analysis using the torque data.
   1. AoC evaluation using EMG data
      NOTE: The method to determining AoC evaluation using EMG is basically the same as for the isokinetic MTS case.
      1. Process the raw EMG data using the RMS method to smooth the data and amplify it 50x.
      2. Synchronize the EMG data and the robot angle data using the trigger signals of each data set.
      3. Determine the AoC manually as the starting point of the RMS EMG upsurge. An example is shown in Figure 9.

5. Data analysis

1. Normalized assessment motion index (NAMI)
   NOTE: The AoC of MTS can be affected by various motion factors, such as assessment speed, acceleration, etc. Therefore, the assessment motion should be as isokinetic as possible. The NAMI is proposed to evaluate the ideality of the assessment motion. The proposed index is a non-dimensional index that can be used to evaluate the consistency of the assessment motion assigned to the subjects in each trial.
   1. Calculate the ROM, maximum velocity and assessment time from each assessment trial.
      NOTE: The angle is measured by the encoder; thus, the calculated velocity is noisy. Therefore, the maximum velocity is determined as the maximum velocity of the trend line, not the peak point.
   2. Calculate the NAMI value for each trial during the whole experiment using Equation 3:
      Where: ϑ_max and ϑ_min represent maximum and minimum angles, respectively, measured during the experiment; ω_max is the maximum assessment speed; and Δt is the total time spent for one assessment. Figure 10 shows an example of each variable.
      NOTE: The proposed index gives a score close to 1 if the assessment motion is close to completely isokinetic and a score close to 0 if the velocity of the motion is inconsistent.
2. Statistical analysis
   NOTE: All statistical analyses are performed using the PASW statistical package (SPSS version 18.0). The intraclass correlation coefficient (ICC) method is used to identify the test-retest reliability and inter-rater reliability. Only results from the second and third tests are used to calculate the ICC.
   1. To verify test-retest reliability, calculate the ICC from the measured AoC data and NAMI result.
   2. To verify inter-rater reliability, calculate the ICC from the average of AoC and NAMI data.
   3. Calculate the p-value of the AoC results using paired sample t-tests to evaluate the differences between each rater or each assessment trial.
      NOTE: P-values of <0.05 are considered statistically significant.
   4. Calculate the Pearson correlation coefficient between AoC based on EMG criteria and torque criteria to verify a correlation between the two methods.

Results

The reliability is divided into four grades according to the ICC value: extremely excellent (>0.90), excellent (0.75 < ICC ≤ 0.90), fair to good (0.40 < ICC ≤ 0.75), and poor (<0.40). The standard error of measurements (SEM) was calculated to determine the error component of the variance. The smallest detectable difference (SDD) was calculated from the SEM of test-retest data.

Normalized assessment motion index (NAMI): the NAMI score during an isokinetic motion was always 1, which ...

Discussion

This study attempted to standardize the MTS measurement using a robotic isokinetic device. It was investigated how the consistency of assessment motion affects the results of MTS measurement.

The NAMI value was proposed to represent the degree of variability in assessment motion. As expected, unlike the isokinetic motion method with no variability, the manual method showed variability between tests and between raters, resulting in poor reliability, which is consistent with results from previou...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
3D printer	Lokit	3Dison+	FDA type 3D printer
Ball sprine shaft	Misumi	LBF15
Bridge Analog Input module	National Instruments	NI 9237
CAN communication module	National Instruments	NI 9853
Caster	Misumi	AC-50F
Electromyography (EMG) device	Laxtha	WEMG-8
EMG electrode	Bioprotech		1.8x1.2 mm Ag–AgCl
Encoder	Maxon	HEDL 9140	500 CPT
Gearbox	Maxon	GP 81	51:1 ratio
Lab jack	Misumi	99-1620-20
Linear slider	Misumi	KSRLC16
Motor	Maxon	EC-60	brushless EC motor
Motor driver	Elmo	DC Whistle
PLA	Lokit		3D printer material
Real-time processor	National Instruments	sbRIO-9632
Torque sensor	Transducer Techniques	TRS-1K