Measuring the Kinematics of Daily Living Movements with Motion Capture Systems in Virtual Reality

The overall goal of this experimental procedure is to assess kinematic performance measures of daily living movements with motion capture systems in an immersive virtual environment. This method can help answer key questions in the medical field, such as how to diagnose dementia and mild cognitive impairment early through the ability to complete daily tasks. The main advantage of this technique is that it can quantitatively document detailed kinematic data, such as trajectory, moving distance, and timed completion while performing daily tasks in real time.

The implication of this technique extends towards early diagnosis of dementia in EMSCI because the ability to complete daily tasks such as using public transportation are closely intertwined with neuropsychological functions. We first had the idea for this method when we realized the potential of virtual reality as an assessment environment, where the participants can be tested under the same conditions which mimic the real world experiences. Generally, individuals new to this method will struggle, because conducting experiments in virtual reality and analyzing kinematic data are hard to perform.

Begin by setting up the hardware, including four computers, four stereoscopic three-dimensional projectors, and eight motion tracking cameras to track the position and orientation of the dominant hand and head during the two instrumental activities of daily living, IADL tasks. Ensure that all computers are equipped with the required software. For middleVR, check the website to obtain the latest versions of the libraries for the input devices, stereoscopy, clustering and interactions.

Next, create a Windows 10 HomeGroup to connect the four computers to a home network. On the primary computer, create a folder and share it with other HomeGroup computers. Next, mount eight motion tracking cameras, in a virtual environment to fully cover the capture volume.

Fix cameras securely so that they remain stationary during capture. Ensure objects in a virtual environment will be visible by at least two cameras at all times. Install motion capture software on the primary computer using the installation manual.

Connect the primary computer with the motion capture systems with category six Ethernet cables. Next, prepare the virtual environment by moving all reflective objects. Ensure that all computers, stereoscopic 3D projectors, and motion capture systems are turned on.

Once four computers are running, launch the VR software by double clicking on VRDaemon.exe. Finally, on the primary computer, initiate the motion capture software. Click the button near the top menu labeled open existing project, load the camera calibration file.

Begin by providing the participant with stereoscopic glasses, weighing around 50 grams, and ensure that they are placed comfortably over the eyes and ears. Next, attach reflective markers, weighing less than one gram, to the participant's dominant hand and head. Be careful to attach reflective markers tightly.

Inform the participant that they can freely move or rotate in the virtual environment using head movement, and can click virtual objects with the dominant hand. Let them know that a virtual hand will appear in the virtual environment to mimic the position of their index finger. Next, ask the participant to freely move in the virtual environment for five minutes to familiarize themselves with the VR environment.

Then, ask them to click virtual buttons for five minutes in order to become familiar with how to interact with virtual objects with the dominant hand. Lastly, check whether the participant is immune to VR sickness with a simulator sickness questionnaire. First, explain to the participant the details of the task, and provide the eight action steps to complete the task in the virtual environment.

Inform them the steps are one, insert the card into the ATM two, select the withdraw menu, three, select the amount to withdraw, four, select the bill type, five, enter the PIN or personal identification number, six, select the receipt option, seven, remove the card, and eight, take the money from the ATM. On the primary computer, initiate the middleware software. On the simulations tab, select a simulation file for task one, and a configuration file.

Then, press the run button. Observe the participant completing task one. Once the first task is finished, check the saved cinematic data and csv files for further analysis from a shared folder.

Provide the participant with a five minute break. Then start the second task, take a bus, by explaining the details. On the primary computer, initiate the middleware software.

On the simulations tab, select a simulation file for task two and a configuration file. Press the run button to start task two. Instruct the participant to wait in the bus stop.

Click the space bar key on the keyboard to make buses arrive at the bus stop. Finally, once the task is finished, check the saved kinematic data and csv files for further analysis from a shared folder. Help the participant remove the stereoscopic glasses and detach the reflective markers from the dominant hand and head.

Here, the trajectory of the hand is visualized from task one, withdraw money, in healthy controls, and patients with mild cognitive impairment. Further, the trajectory of the head movement is visualized from task two, take a bus, in healthy controls and patients with mild cognitive impairment. Once mastered, this technique can be done in ten minutes, if it is performed properly, while attempting this procedure, it's important to remember to fix motion tracking camera securely to fully cover the capture volume, and to familiarize the participant sufficiently with the virtual environment to prevent VR sickness.

Following this procedure, other non-invasive neuroimaging methods like electroencephalogram and functional near-infrared spectroscopy can be performed in order to answer additional questions like the relationship between kinematic performance measures and its underlying neurophysiological manifestation. After its development, this technique paved the way for researchers in the field of medicine to explore the kinematics of daily living movements in patients with neuropsychological disease. After watching this video, you should have a good understanding of how to set up a virtual environment, familiarize the participant, conduct tasks in VR, and measure the kinematics of daily tasks with motion capture system in the virtual environment.

Don't forget that performing tasks in virtual reality can cause VR sickness and precautions such as sufficient familiarization with the virtual environment, and check whether the participant is immune to VR sickness should always be taken while performing this procedure.