Setup for the Quantitative Assessment of Motion and Muscle Activity During a Virtual Modified Box and Block Test

Summary

The protocol described here aims to enhance the quantitative evaluation of upper limb deficits, with the goal of developing additional technology for remote assessment both in the clinic and at home. Virtual reality and biosensor technologies are combined with standard clinical techniques to provide insights into the functioning of the neuromuscular system.

Abstract

The ability to move allows us to interact with the world. When this ability is impaired, it can significantly reduce one's quality of life and independence and may lead to complications. The importance of remote patient evaluation and rehabilitation has recently grown due to limited access to in-person services. For example, the COVID-19 pandemic unexpectedly resulted in strict regulations, reducing access to non-emergent healthcare services. Additionally, remote care offers an opportunity to address healthcare disparities in rural, underserved, and low-income areas where access to services remains limited.

Improving accessibility through remote care options would limit the number of hospital or specialist visits and render routine care more affordable. Finally, the use of readily available commercial consumer electronics for at-home care can enhance patient outcomes due to improved quantitative observation of symptoms, treatment efficacy, and therapy dosage. While remote care is a promising means to address these issues, there is a crucial need to quantitatively characterize motor impairment for such applications. The following protocol seeks to address this knowledge gap to enable clinicians and researchers to obtain high-resolution data on complex movement and underlying muscle activity. The ultimate goal is to develop a protocol for remote administration of functional clinical tests.

Here, participants were instructed to perform a medically-inspired Box and Block task (BBT), which is frequently used to assess hand function. This task requires subjects to transport standardized cubes between two compartments separated by a barrier. We implemented a modified BBT in virtual reality to demonstrate the potential of developing remote assessment protocols. Muscle activation was captured for each subject using surface electromyography. This protocol allowed for the acquisition of high-quality data to better characterize movement impairment in a detailed and quantitative manner. Ultimately, these data have the potential to be used to develop protocols for virtual rehabilitation and remote patient monitoring.

Introduction

Movement is how we interact with the world. While everyday activities such as picking up a glass of water or walking to work may seem simple, even these movements rely on complex signaling between the central nervous system, muscles, and limbs¹. As such, personal independence and quality of life are highly correlated to the level of an individual's limb function^2,3. Neurological damage, as in spinal cord injury (SCI) or peripheral nerve injury, can result in permanent motor deficits, thereby diminishing one's ability to execute even simple activities of daily living^4,5. According to the National Institute of Neurological Disorders and Stroke, over 100 million people in the United States experience motor deficits, with stroke as one of the leading causes^6,7,8. Due to the nature of these injuries, patients often require prolonged care in which quantitative motor assessment and remote treatment may be beneficial.

Current practices for treating movement disorders often require both initial and ongoing clinical assessment of function through observation by trained experts such as physical or occupational therapists. Standard validated clinical tests often require trained professionals to administer them, with specific time constraints and subjective scoring of predefined movements or functional tasks. However, even in healthy individuals, identical movements can be accomplished with varying combinations of joint angles. This concept is termed musculoskeletal redundancy.

Functional clinical tests often do not account for the individual redundancy underlying inter-subject variability. For clinicians and researchers alike, distinguishing between normal variability caused by redundancy and pathological changes in movement remains a challenge. Standardized clinical assessments performed by well-trained raters utilize low-resolution scoring systems to reduce inter-rater variability and improve test validity. However, this introduces ceiling effects, thus lowering the sensitivity and predictive validity for subjects who may have mild movement deficits^9,10. Furthermore, these clinical tests cannot differentiate if deficits are caused by passive body mechanics or active muscle coordination, which may be of importance during initial diagnosis and when designing a patient-specific rehabilitation plan. Randomized clinical trials have revealed inconsistent efficacy of treatment plans formulated based on evidence provided by these clinical tests^11,12,13. Several studies have emphasized the need for quantitative, user-friendly clinical metrics that may be used to guide the design of future interventions^14,15.

In previous studies, we demonstrated the implementation of automated movement assessment using readily available consumer motion capture devices in post-stroke arm impairment, as well as the evaluation of shoulder function after chest surgery in breast cancer patients^16,17. Additionally, we have shown that using active joint moments to estimate muscle moments of specific active movements is a more sensitive measure of motor deficits after stroke compared to joint angles¹⁸. Motion capture and surface electromyography (EMG) may therefore be of critical importance in the assessment of patients who are diagnosed as asymptomatic by standard clinical tests, but who may still be experiencing movement difficulties, fatigue, or pain. This paper describes a system that may enable detailed and quantitative characterization of movement during standard clinical tests for the future development of methods for at-home evaluation and rehabilitation in movement-impaired patient populations.

Virtual reality (VR) can be used to construct an immersive user experience while modeling everyday tasks. Typically, VR systems track the hand movements of the user to allow for simulated interactions with the virtual environment. The protocol we describe here uses consumer VR products for motion capture to quantify the assessment of motor deficits, similar to other studies demonstrating the use of off-the-shelf video game controllers in quantitative evaluation of impairment after stroke or shoulder surgery^16,17. In addition, EMG is a non-invasive measure of the neural activity underlying muscular contraction¹⁹. As such, EMG may be used to indirectly evaluate the quality of the neural control of movement and provide a detailed assessment of motor function. Muscle and nerve damage may be detected by EMG, and disorders such as muscular dystrophy and cerebral palsy are commonly monitored using this technique^20,21. Furthermore, EMG may be used to track changes in muscle strength or spasticity, which may not be evident in kinematic assessments^22,23, as well as fatigue and muscle coactivation. Metrics such as these are critical in considering rehabilitation progress^23,24,25.

The experimental paradigm described here seeks to leverage a combination of VR and EMG to address the limitations of traditional clinical assessment tools. Here, participants were asked to perform a modified Box and Block task (BBT)²⁶ using real objects and in VR. The standard BBT is a clinical tool used in the general assessment of gross upper extremity function, in which subjects are asked to move as many 2.5 cm blocks as possible from one compartment, over a partition, to an adjoining compartment within one minute. While often used to reliably assess deficits in patients with stroke or other neuromuscular conditions (e.g., upper extremity paresis, spastic hemiplegia), normative data have also been reported for healthy children and adults, ages 6-89²⁶. A virtual movement assessment is used to simulate functional aspects of the validated clinical test performed in real life. VR is used here to decrease required hardware while allowing for the provision of standardized instructions and programmed, automated scoring. As such, constant supervision by trained professionals would no longer be necessary.

The BBT in this study has been simplified to focus on capturing the reaching and grasping of one block at a time that appears in the same location. This maximized the reproducibility of the movements and minimized the inter-subject variability in recorded data. Lastly, virtual reality headsets can be purchased for as little as $300 and have the potential to house multiple assessments. Once programmed, this would significantly decrease the cost associated with typical professional evaluation and allow for increased accessibility of these standard, validated clinical tests in both clinical and remote/at-home settings.

Protocol

Experimental procedures were approved by the West Virginia University Institutional Review Board (IRB), protocol # 1311129283, and adhered to the principles of the Declaration of Helsinki. Risks from this protocol are minor but it is necessary to explain all procedures and potential risks to participants and written, informed consent was acquired with documentation approved by the institutional ethical review board.

1. System characteristics and design

NOTE: The setup for this protocol consists of the following elements: (1) EMG sensors and base, (2) EMG data acquisition (DAQ) software, (3) a motion capture system, and (4) a VR headset with corresponding software. These components are visualized in Figure 1.

1. Setup of the system components
   1. Connect the EMG system.
      1. Plug in the EMG base station to power.
      2. Connect the EMG base station to a dedicated computer (Figure 1C) containing the DAQ scripts.
         NOTE: Example scripts can be found at: https://www.dropbox.com/sh/7se5lih4noxj584/AACqFDZytpDm-L8jAULFwfTHa?dl=0. Some commercial products may come with licensed DAQ software, which may also be used.
   2. Connect the motion capture system.
      1. Connect a second dedicated computer (Figure 1A) to a network router.
      2. Connect the network router to a motion capture server.
      3. Connect the motion capture server to a computer monitor for visualization.
      4. Connect the cameras to the motion capture server.
   3. Connect the VR system.
      1. Connect the VR headset to a third dedicated computer with the corresponding DAQ scripts (Figure 1B).
         NOTE: Example scripts can be found at: https://www.dropbox.com/sh/7se5lih4noxj584/AACqFDZytpDm-L8jAULFwfTHa?dl=0
      2. Load the VR gaming environment containing the intended tasks to the computer linked to the VR headset (Figure 1B).
   4. Prepare the area where the subject will complete the task.
      1. Use an arm-less, sturdy chair to ensure that there is no interference with the subject's normal reaching.
      2. For the safety and accuracy of the data collected, confirm that the testing area is clear of all obstacles.
   5. Synchronize the systems.
      1. Use a custom software function to synchronize systems in time to a one-time server.
      2. Alternatively, use one computer or a preferred message manager.

Figure 1: Experimental equipment setup. (A) The marker motion capture cameras are positioned on the floor and in the ceiling around the experimental space, establishing an optimal space for tracking motion. A dedicated computer is used to run the motion capture software and save the data. (B) The headset used to display the modified BBT in VR is connected to a dedicated computer where the virtual assessment and task data are saved. (C) The EMG base is connected to a dedicated computer where muscle activity data is recorded and saved during the task execution. EMG sensors and LED markers for motion capture are both placed on the subject's arm during the session (see Figure 2). Abbreviations: VR = virtual reality; EMG = electromyography. Please click here to view a larger version of this figure.

2. Experimental procedures

NOTE: a visual representation of the experimental flow described in this protocol is shown in Figure 2.

1. Setup EMG
   1. To determine EMG sensor placement for best signal quality, palpate the muscle belly while the participant contracts the relevant muscle²⁷. See Table 1 for the muscle selection used in this protocol.
   2. Using an alcohol wipe, carefully clean each electrode and the intended sensor placement site on the subject's arm.
      NOTE: Thorough cleaning of both the EMG sensors and the subject's skin will ensure a low electrode-to-skin impedance. This ensures that recorded EMG data has a high signal-to-noise ratio. Excess hair at the sensor placement site may cause low-quality data even if properly cleaned. In this case, it may be necessary to shave the hair.
   3. After preparing the EMG electrodes and the subject's skin, place the EMG sensors on the subject, ensuring good contact between the electrodes and skin. Bipolar electrodes should be positioned so that the sensors are parallel to the direction of the muscle fibers.
2. Setup motion capture system
   NOTE: This may not be necessary if only a VR headset is being used to track hand movements for kinematics.
   1. Calibrate the motion tracking cameras using the manufacturer's instructions. Move the calibration wand throughout the experimental space to calibrate the motion tracking cameras and set the 3D axis of the space.
      NOTE: The marker motion capture system used in this protocol includes a calibration wand with LED markers.
   2. Place the LED motion capture markers on the bony landmarks of the subject's upper extremity and trunk that are needed to construct the desired biomechanical models.
   3. Use the provided motion capture software to ensure that all markers are recognized and tracked by the motion capture cameras. Instruct the subject to perform several practice movements while study personnel visually monitor marker data in real time.
3. Setup VR skill assessment task
   1. Position a chair in the center of the motion capture experimental space. Use a chair that is similar to that used for the traditional, real-world test.
   2. Calibrate the VR headset in the chair where the subject will perform the assessment task. Once calibrated, instruct the subject to sit in the chair and place the VR headset on their head.
   3. While the subject is seated, measure the distance between the subject's shoulder and the ground, as well as the length of the subject's arm. Use these distances to set the location where the table and assessment task will spawn in VR.
   4. In the VR task control script, input the subject's measurements and program the desired number of block spawn repetitions.
      NOTE: In this Modified BBT, individual blocks will spawn one at a time to increase reproducibility.
4. Instruct the subject to perform the modified BBT assessment in VR.
   1. Provide a brief explanation of the task to the subject.
      1. The virtual block will spawn on the left or right side, as determined by the experimenter.
      2. Explain to the subject that they will need to pick up the virtual block, transport it over the partition, and place it onto the target in the opposing compartment (Figure 2).
         NOTE: In the virtual modified BBT used here, the block will automatically disappear and re-spawn at the starting position as many times as determined by the experimenter.
   2. Start collecting EMG data.
   3. Start collecting motion capture data.
   4. Start the VR skill assessment task.
      1. Let the task run for the preset number of repetitions before automatically ending.
   5. Save the EMG and kinematic data for post-hoc analysis.
   6. Determine a clinically relevant score for the Modified BBT post-hoc as the number of blocks successfully carried over the barrier or repetitions of the task in 60 s.

Figure 2: Experimental protocol, VR task, and subject setup. (A) Flow diagram describing the experimental protocol used here. (B) Example view of modified BBT implemented in VR environment. Anatomical measurements are used to calibrate the VR task, ensuring that the virtual table spawns at the correct relative location. (C) Placement of LED motion capture markers and EMG sensors on the subject. EMG sensors are placed on the muscles of interest and LED motion capture markers are positioned over bony landmarks. Abbreviations: VR = virtual reality; EMG = electromyography; LED = light-emitting diode. Please click here to view a larger version of this figure.

Results

EMG, kinematic, and force data obtained from subjects using this protocol can be used to characterize movements across repetitions of the same task, as well as during different tasks. Data shown here represent results from healthy control participants to demonstrate the feasibility of this setup. Representative EMG profiles recorded from a healthy subject performing the modified BBT in VR are shown in Figure 3. High muscle activation of the anterior deltoid (DELT_A) can be seen, suggesting that it is the primary mover of the arm for each reaching movement. Forearm and wrist extensors (ECU and ECR) were also notably activated, which suggests that these muscles are used to support the grasping of the block throughout the movement. Finally, increased activity of thumb muscles (EPB and APB) indicates their use in the grasp and release of the block. While EMG profiles recorded during the modified BBT in VR were similar to the real-world task, maximal muscle activity during VR assessment was reduced. These results suggest that both the standard real-world BBT and the modified BBT in VR engage similar motor control mechanisms.

VR hand-tracking data was used to obtain kinematic data that produced smooth profiles of joint angle trajectories throughout the modified BBT implemented in VR (Figure 4). The use of the thumb, middle, and index fingers in grasping the block is marked by joint angle changes at both the proximal and distal interphalangeal joints (Figure 4). Joint angles of fingers 2 (index) and 3 (middle) are seen to change along similar trajectories, signifying their concurrent use to grasp and then release the block. Comparatively, fingers 4 (ring) and 5 (pinky) show a distinct pattern in joint angle changes at the proximal and distal interphalangeal joints. This not only suggests that they were not used in grasping the block, but also shows that their movement opposing fingers 2 and 3 may mark a typical out-of-synergy motion in which these fingers remain flexed while the block is released. This fractionated control of digit extension versus flexion may be absent in someone with post-stroke motor deficits. Table 2 shows the average joint angle ranges during the task. Additionally, the timing of each movement or movement phase, such as grasp and reach, and the number of repetitions in a given period can also be measured from the kinematic data obtained from the VR headset.

The quality of data obtained from this protocol can be maximized through thorough consideration of several factors. As previously mentioned, acquiring high-quality EMG data relies on close contact between sensor electrodes and the skin. When sensors are positioned securely, EMG data will have a high signal-to-noise ratio with clear periods of bursting that indicate when muscles are actively contracting (Figure 5). Distinct bursts of electrical activity corresponding to muscular contraction can be visualized in both the unfiltered (Figure 5A) and filtered (Figure 5B) signals, with minimal noise. Aberrant or unexpected increases in signal amplitude may be indicative of motion artifacts that can result from insecure attachment of EMG sensors to the skin. If wired sensors are used, this could also result from wires that are dangling during motion. High-quality motion capture data requires the limbs to remain in clear camera view so that the tracking algorithm can precisely resolve the relative coordinates of tracked points. Large or abrupt changes in the coordinates of tracked points or missing coordinate values indicate low-quality motion capture data. While marker interpolation may be used to calculate missing coordinates, this method is only valid for brief periods relative to the duration of movement.

Figure 3: Representative filtered and rectified EMG profiles recorded from a healthy subject performing 10 repetitions of the modified BBT in VR. (A) Proximal arm flexors (biceps short head, red; biceps long head, gray; anterior deltoid, black). (B) Proximal arm extensors (triceps long head, green; lateral triceps, blue). (C) Distal arm flexors (flexor carpi ulnaris, red; flexor carpi radialis, black). (D) Distal arm extensors (extensor carpi ulnaris, green; extensor carpi radialis, blue). (E) Finger and thumb muscles (flexor digitorum superficialis, red; extensor digitorum communis, green; extensor pollicis brevis, blue; abductor pollicis brevis, brown). This figure is adapted from Yough²⁸. Abbreviations: BBT = Box and Block Task; EMG = electromyography; BIC_SH = biceps short head; BIC_LO = biceps long head; DELT_A = anterior deltoid); TRI_LO = triceps long head; TRI_LAT = lateral triceps; FCU = flexor carpi ulnaris; FCR = flexor carpi radialis; ECU = extensor carpi ulnaris; ECR = extensor carpi radialis; FDS = flexor digitorum superficialis; EDC = extensor digitorum communis; EPB = extensor pollicis brevis; APB = abductor pollicis brevis. Please click here to view a larger version of this figure.

Figure 4: Representative kinematic profiles recorded from a healthy subject performing 10 repetitions of the modified BBT in VR. (A) Joint angles of the thumb at the CMC, MCP, and IP joints. (B) Joint angles of the MCP joints of digits 2-5. (C) Joint angles of the PIP joints of digits 2-5. (D) Joint angles of the DIP joints of digits 2-5. Numbers following each joint name indicate the finger each joint corresponds to (thumb = 1, index = 2, middle = 3, ring = 4, pinky = 5). This figure is adapted from Yough²⁸. Abbreviations: CMC = carpometacarpal; Ad/Ab = adduction/abduction; E/F = extension/flexion; MCP = metacarpophalangeal; IP = interphalangeal; PIP = proximal interphalangeal; DIP = distal interphalangeal. Please click here to view a larger version of this figure.

Figure 5: Representative EMG sensor data. (A) Raw and (B) filtered EMG signals from an optimally placed, secure EMG sensor on the flexor carpi ulnaris. This figure is adapted from Yough²⁸. Abbreviations: EMG = electromyography. Please click here to view a larger version of this figure.

Delsys Sensor Number	Muscle
1	Flexor carpi ulnaris
2	Flexor carpi radialis
3	Extensor carpi ulnaris
4	Extensor carpi radialis longus
5	Anconeus
6	Flexor digitorum superficialis
7	Extensor digitorum communis
8	Biceps brachii - medial/short head
9	Biceps brachii - lateral/long head
10	Triceps brachii - long head
11	Triceps brachii - lateral head
12	Anterior deltoid
13	Lateral deltoid
14	Extensor pollicis brevis
15	Abductor pollicis brevis

Table 1: Muscle selection for EMG sensors for recording in this protocol.

Degree of Freedom			Minimum joint angle (rad)	Maximum joint angle (rad)	Magnitude (rad)
Wrist Supination-Pronation			0.24	0.68	0.44 ± 0.17
Wrist Extension-Flexion			-0.47	-0.4	0.1 ± 0.06
Wrist Adduction-Abduction			-0.18	-0.1	0.08 ± 0.03
Carpometacarpal Adduction-Abduction			0.35	0.55	0.2 ± 0.09
Carpometacarpal Flexion-Extension			0.05	0.24	0.18 ± 0.12
Thumb Metacarpal Flexion-Extension			-0.23	-0.15	0.08 ± 0.02
Thumb Interphalangeal Flexion-Extension			-0.56	-0.48	0.09 ± 0.03
Index Metacarpal Extension-Flexion			0.25	0.5	0.25 ± 0.13
Index Proximal Interphalangeal Extension-Flexion			0.74	0.94	0.0.2 ± 0.14
Index Distal Interphalangeal Extension-Flexion			0.67	0.81	0.15 ± 0.1
Middle Metacarpal Extension-Flexion			0.25	0.46	0.2 ± 0.11
Middle Proximal Interphalangeal Extension-Flexion			0.89	1.07	0.18 ± 0.11
Middle Distal Interphalangeal Extension-Flexion			0.74	0.9	0.15 ± 0.1
Ring Metacarpal Extension-Flexion			0.22	0.42	0.2 ± 0.14
Ring Proximal Interphalangeal Extension-Flexion			0.9	1.1	0.2 ± 0.15
Ring Distal Interphalangeal Extension-Flexion			0.78	0.93	0.15 ± 0.1
Little Metacarpal Extension-Flexion			0.32	0.48	0.16 ± 0.04
Little Proximal Interphalangeal Extension-Flexion			0.93	1.09	0.16 ± 0.09
Little Distal Interphalangeal Extension-Flexion			0.76	0.9	0.14 ± 0.09

Table 2: Average joint angle ranges during a modified BBT. Movement directions in DOF names correspond to negative-positive directions. For example, in "Wrist Supination-Pronation," supination is in the negative direction and pronation is in the positive direction.

Discussion

EMG system
The hardware of the EMG system consists of 15 EMG sensors used to obtain muscle activation data. A commercially available Application Programming Interface (API) was used to generate custom EMG recording software. The VR system hardware consists of a virtual reality headset used to display the immersive VR environment and a cable to link the headset to the dedicated computer where the virtual assessment task is stored. The software consists of 3D computer graphics software to create and run the VR task. Here, a modified Box and Block Test, adapted from the popular manual dexterity assessment²⁶, was used as the assessment task. Interaction with the VR environment was permitted through the use of a software program designed to calculate the location where the table would spawn, start the task, spawn blocks, and save the data.

Motion capture system
The motion capture system hardware consists of the VR headset, the marker motion capture system, and the VR motion controller. The VR headset is equipped with hand-tracking ability through four built-in infrared cameras (72-120 Hz). These data were extracted from the device for kinematic analysis. The marker motion capture system consists of a server, cameras with a 60° field of view, red light emitting diode (LED) as markers, and a calibration object. To accurately capture the space, cameras will need to slightly overlap. During the experiment, a given capture area was surrounded by eight cameras (480 Hz), where four were at the ceiling and four were at the floor. This system was calibrated and aligned before the experiment. Marker motion capture systems may be preferred as they provide high-definition kinematic data for both clinical and research applications. The VR motion controller contains an optical hand-tracking module that captures the hand and finger movement. The controller has two 640 x 240 pixels near-infrared cameras (120 Hz), which are capable of tracking movement up to 60 cm from the device and in a 140 x 120° field of view. This device was attached to the VR headset or secured above the head during movement. 3D computer graphics development software was used to visualize hand tracking and collect data from the VR environment. Data from the markerless and marker motion capture systems was acquired and saved using custom software program scripts. Example software program scripts can be found at: https://www.dropbox.com/sh/7se5lih4noxj584/AACqFDZytpDm-L8jAULFwfTHa?dl=0. 

Potential equipment substitutions
Wireless, bipolar, surface EMG electrodes are used in this protocol, however other EMG technologies (wired, etc.) could be substituted. If using standard commercially available sensors as described in the Table of Materials, the accompanying DAQ software can be used instead of the custom software scripts (sampling rate of 2,000 Hz). If alternative EMG sensors are used, then software compatible with that system should also be used.

Rather than linking the VR headset to the computer via cable, the VR headset has the capability to store the interactive environment within the headset. The implementation of the feature in future applications for take-home rehabilitation would be advantageous for the patient. The VR headset used here may be substituted for a different immersive VR headset. Prior to substitution, this would require validation of compatibility with the game development software.

Alternative development software may be used for the creation of the virtual environment and task described in this protocol. Additionally, other clinical assessments translated into VR may be used in place of the modified BBT illustrated here (e.g., the Action Research Arm Test (ARAT) or Graded Redefined Assessment of Strength, Sensation, and Prehension (GRASSP)). While the BBT can be used to assess a wide range of neurological injuries, the use of alternative clinical tasks would allow for evaluation of other movement disorders. The GRASSP, for example, is primarily used to assess patients with spinal cord injury, whereas the ARAT is most commonly used to assess hemiplegia due to cortical damage. Virtual assessments could also be administered without control by a software program script (i.e., standalone), but DAQ and control of the test administration would need to be accounted for. For skill assessment, a tangible Box and Block Test (BBT) could be used in place of the task developed in VR²⁶. Use of real-world equipment would require alternative motion capture technology, in place of hand tracking recorded from a VR headset.

Alternate reliable marker or markerless motion capture may be used to track hand kinematics. For example, VR-based motion capture may be used to simplify the setup, while still obtaining necessary hand-tracking data to extract kinematic information. The marker motion capture system used here includes software to calibrate the experimental motion capture space and run the cameras (sampling rate of 480 Hz). Alternative motion capture technology should be used with the appropriate corresponding software.

Separate computers are illustrated in Figure 1, each running a different hardware component. A single high-performance computer can run all the software for data collection and VR. In the protocol described here, the use of three separate computers requires data synchronization. Syncing the data in time across devices is critical for EMG, kinematic, and force data to yield meaningful conclusions. Custom software program scripts using a built-in datetime function were used to record the universal time on each system at the beginning of the task. Critically, all computers must be connected to the same network for these scripts to function properly. After the starting time was recorded using datetime, the DAQ scripts utilized the tic and toc functions to increment time during the trial. These functions have a precision of approximately 0.000001s, potentially allowing for data to be recorded at 1,000 kHz. In our setup, the highest sampling frequency is 30 kHz, which is well below the maximum capabilities of this method for recording and syncing time across devices. A transmission control protocol/internet protocol (TCP-IP) messaging application²⁹ developed in Python was used to transmit data between the VR environment and the DAQ application based on IP address.

In addition to ensuring time synchronization, it was of critical importance to sync data acquired at different sampling frequencies. Two of the computers were manually started to collect motion capture data (480 Hz) and record EMG data (2000 Hz). Prior studies have utilized custom synchronization circuits to send a common event between systems at trial landmarks (e.g., a sync pulse is sent when data recording begins, and again when recording ends)³⁰. Here, the use of the software program's datetime, tic, and toc functions allows for synchronization without the need for added hardware. The commercially available TCP-IP messaging application used here also supported continuous communication across systems. This framework for syncing time and data collection frequencies across systems also worked to consistently organize data for advanced post hoc analysis.

The experimental protocol described here is extremely customizable to address the variable needs of clinical or research groups, with the potential to be applied in both clinical and at-home/remote settings. While we have described an assessment of muscle activity and kinematics using a modified BBT in VR, a similar setup can be used for different clinical tests and may even be simplified for reduced data sets. For example, EMG and force sensors can be excluded if only kinematic data is needed. Acquisition of kinematic data only would simply require the use of a motion capture system. Additional hardware, such as electroencephalography or electrocardiography, could also be included to record extra data. In place of standard surface EMG electrodes, high-density surface or intramuscular EMG may be used to record more precise muscle activity data^31,32. Furthermore, other functional clinical tests, such as the ARAT or GRASSP, may be used to evaluate motor deficits in different patient populations. While the setup described here is primarily intended for research purposes to better quantify movement deficits in VR, we aim to develop a similar setup to allow for remote patient monitoring and rehabilitation. We expect that this protocol will be easily adapted using standard consumer VR goggles equipped with hand motion tracking, in combination with a wearable EMG recording device. In future studies, we aim to further develop this protocol for at-home use by incorporating a low-cost, wearable, high-density EMG sleeve that is currently being developed.

Compared to traditional physical or occupational therapy in a clinic or rehabilitation center, VR rehabilitation offers many potential benefits. While the protocol described here is intended to be used primarily for research purposes, virtual evaluation methods can potentially be implemented remotely and using off-the-shelf devices, without the need for expensive, specialized medical equipment. Additionally, VR systems are able to quantitatively track hand movements to provide insights into motion quality, which may not be feasible with standard assessment^33,34. The comparability of movements in VR versus real-world tasks remains a topic of intensive study and future insights will affect how VR can be used for rehabilitation. The procedure described here is a flexible setup that uses VR to guide standardized movements and obtain individual kinematic and EMG data. Data obtained from studies utilizing the described setup may also provide insights into how individuals move in the virtual environment, and potential mechanisms by which some subjects find VR to be more immersive. This is of critical importance when adapting real-life clinical assessments into a VR environment.

Several steps in the setup and execution of this protocol must be carefully monitored to obtain high-quality, accurate data. Of primary importance is proper EMG sensor placement. Clean, accurate EMG data with a high signal-to-noise requires the sensors to be placed on sanitized, hair-free skin above the belly of the selected muscle²⁷. Similarly, if a marker motion capture system is used, markers must be precisely placed on bony landmarks to capture reliable data for calculating the joint angles of interest. Compared to the EMG sensors, there is some flexibility in the orientation of motion capture markers, but a minimum of three non-colinear points should always be used to define the location of a rigid body in space³⁵. Certain techniques for calculating joint angles, such as model-based inverse kinematics, will require exact marker placement. If using this method, experimental marker locations should match that of the model³⁶. Musculoskeletal modeling software developed for the calculation of inverse kinematics usually includes built-in computational functions for matching the model and experimental marker locations. OpenSim, a platform developed by Stanford University³⁷, is commonly used for this type of analysis. Both EMG sensor and marker security should be monitored throughout the task to ensure high-quality data is recorded.

If the data obtained from this protocol is noisy, troubleshooting should focus on consideration of several steps. First, poor EMG sensor placement can produce noisy data and should be verified. To mitigate this problem, skin preparation prior to EMG sensor attachment is important, as previously discussed. Additionally, low-quality motion capture data may result if the markers move out of range of the cameras. A brief loss of tracked points can be resolved through interpolation, but if markers continue to fail to transmit accurate positional data, more severe underlying issues may exist. If a single marker is repeatedly dropping out, this may indicate that the marker is either obscured by the subject's movement or is moving out of the camera's range. Furthermore, faulty camera calibration may cause several markers to fail. In this scenario, it is best to fully recalibrate the motion capture system after removing the subject and test equipment from the experimental space. Additionally, sufficient lighting should be used, since low-light environments may impede the acquisition of high-quality motion capture data.

The described protocol has several limitations. First, the setup illustrated here requires the use of multiple motion capture cameras. However, the intent of this protocol is to collect high-resolution data to better quantify movement deficits in VR prior to the development of fully remote clinical assessments that may be completed by the patient alone. Additionally, tasks performed in VR lack tactile feedback. This is important to note since tactile feedback alone can alter the magnitude of grip force^38,39. This may affect muscle activation or kinematics when a task is executed in a virtual environment instead of using tangible equipment. If tactile feedback is believed to be of critical importance (e.g., if patients are also experiencing sensory deficits), the use of haptic robots may be included to mimic tactile feedback. Despite this limitation, we anticipate that data obtained from the setup described here will be crucial in quantifying changes in kinematics and muscle activity in VR-only tasks for patients with movement disorders. This will aid in adapting such clinical tests for at-home use. An additional limitation of this protocol is that a single block is presented at a time during the test. In the standard BBT, 150 wooden blocks are placed in one compartment of the box to start with. This modified BBT was designed to increase the reproducibility of the recorded motion capture and EMG data by simulating one block that respawns in the same virtual location. As such, the modified BBT implemented in VR is likely a more simplified task. In future studies, it will be important for the modified BBT in VR to be consistent with the setup of standard BBT to ensure content validity.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Armless Chair	N/A		A chair for subjects to sit in should be armless so that their arms are not interfered with.
Computer	Dell Technologies		Three computers were used to accompany the data acquisition equipment.
Leap Motion Controller	Ultraleap		Optical hand tracking module that captures the hand and finger movement. The controller has two 640 x 240-pixel near-infrared cameras (120 Hz), which are capable of tracking movement up to 60 cm from the device and in a 140 x 120° field of view. This device was attached to the VR headset or secured above the head during movement.
MATLAB	MathWorks, Inc.		Programming platform used to develop custom data acquisition software
Oculus Quest 2	Meta		Immersive virtual reality headset equipped with hand tracking ability through 4 infrared build-in cameras (72-120 Hz). Can be substituted with other similar devices (ex. HTC Vive, HP Reverb, Playstation VR).
Oculus Quest 2 Link cable	Meta		Used to connect the headset to the computer where the VR game was stored
PhaseSpace Motion Capture	PhaseSpace, Inc.		Markered motion capture system, consisting of a server, cameras with 60° field of view, red light emitting diode (LED) as markers, and a calibration object
Trigno Wireless System	Delsys, Inc.		By Delsys Inc., includes EMG, accelerometer, force sensors, a base station, and collection software. The Trigno-MATLAB Application Programming Interface (API) was used to develop custom recording software.
UnReal Engine 4	Epic Games		Software used to create and run the modified Box and Block Task in VR