Multimodal Motion Capture Toolbox for Enhanced Analysis of Intersegmental Coordination in Children with Cerebral Palsy and Typically Developing

Summary

This study introduces a toolbox combining motion capture technologies to analyze kinematic signals and quantify full-body coordinative patterns with angle-angle plots. Designed to overcome challenges of traditional marker-based systems, the toolbox supports flexibility across research, clinical, and field settings, advancing individualized care and functional assessments for children with motor disabilities.

Abstract

Three-dimensional marker-based motion capture systems are the gold standard for evaluating kinematic patterns in human movement, offering precise quantification of segment and joint positions. However, traditional marker-based systems pose several challenges, particularly for children with neurological disabilities and sensory processing abnormalities, such as those observed with children with cerebral palsy. These challenges hinder the use of kinematic markers and limit detailed analyses of movement patterns. Recent advancements in markerless motion capture systems utilizing deep learning-based human pose estimation allowed us to explore cost-effective alternatives to traditional optical systems and the subsequent data processing approaches. An integrated toolbox was developed, combining multiple motion capture technologies: research-grade kinematic equipment, kinematic clusters, inertial measurement units, three-dimensional (3D) markerless systems, and two-dimensional (2D) markerless systems with commercially available cameras (via MediaPipe). For this current study, we present the outcomes of 3D marker-based versus 2D markerless motion capture, the major ongoing issue in human subjects' biomechanical studies, to describe coordinative patterns via hip-knee angle-angle plots. The cyclogram approach was selected because it offers a robust metric and readily interpretable framework for analyzing coordination via coupled motion between body segments. Two typically developing children and two children with cerebral palsy performed a functional movement pattern, the sit-to-stand task. The findings here demonstrated the feasibility of integrating multimodal systems for kinematic analyses, providing flexibility for research and clinical settings. Moreover, the novel open-source approach presented in this work addresses the challenges posed by many patient populations experiencing sensory processing issues, allowing for an advanced and individualized plan of care.

Introduction

Three-dimensional kinematic motion capture evaluations have offered reliable information in the quantification of expected and abnormal movement patterns. Such information is crucial when designing rehabilitation plans¹ and establishing safety approaches for patient engagement with clinical equipment (e.g.,^2,3). When considering children with neurologic-induced disabilities, these kinematic evaluations can provide enhanced care by addressing changes in coordination patterns in distinct functional tasks such as gait⁴, sit-to-stand task⁵, and upper extremity reaching⁶.

Although three-dimensional (3D) kinematic evaluations are considered the gold standard for accurate motion capture analysis⁷, placement of kinematic markers on the skin is a requirement to ensure exact quantification of segment and joint motions. This marker-based evaluation requires knowledge of specific anatomical landmarks for those performing this evaluation, along with the tolerance and acceptance of tapes secured to the skin of the individuals receiving the evaluation. While the former can be learned, the latter is an issue with several patient populations. As an example, children with brain injury or cerebral palsy may exhibit disorders with the processing of sensory information^8,9,10. This disorder may emerge as hypersensitivity, preventing clinicians and researchers from applying the kinematic markers on the children's skin, which, in turn, can hinder the quantification of changes in movement capabilities and an efficient individualized plan of care.

In addition to the issue of marker placement on the skin of patient populations, most measurement systems utilized in kinematics studies are not affordable, are confined to laboratory spaces, and are only available with highly qualified personnel, making them impractical for routine clinical use. However, current advances in motion capture technology support different approaches for recording kinematic data. For instance, automatic human pose estimation using deep learning techniques has attracted attention among computer vision researchers. These markerless motion capture techniques may offer a promising solution to technical and practical challenges associated with marker-based motion analysis¹¹. Instead of relying on the tracking of markers, this approach uses trained neural networks to estimate the positions of the entire landmark¹². This development enhances human posture-recognition technology's overall efficiency and adaptability.

It is well established that the kinematic relationship between body segments can reveal underlying control mechanisms supporting variability of movement patterns. While several methods can be used to measure coordination between segments¹³, angle-angle plots (cyclograms) allow for an inherently interpretable quantification of coupled motion between segments/joints over a stipulated timeframe^14,15. Since many children with cerebral palsy experience difficulty when performing whole-body movement patterns due to impaired selective motor control¹⁶, we selected to investigate the simple, yet with complex underlying control¹⁷, movement of sit-to-stand. The goal of this study was to demonstrate the feasibility of an open-source toolbox combining motion capture technologies to facilitate kinematic evaluations of coordinative patterns for children with disabilities.

In this work, we will utilize hip-knee cyclograms to demonstrate coordination between trunk/pelvis and thigh/shank segments, using the hip and knee joints as representative intersegmental couplings relevant to sit-to-stand biomechanics. Hip-knee coordination was selected since it captures the primary joint coupling that drives the vertical progression of the center of mass during the sit-to-stand task, a functionally critical movement for children with disabilities who often rely on compensatory lower-limb strategies due to impaired selective motor control¹⁶ affecting joint synergies. The integration of different technologies (i.e., from research-grade 3D kinematic equipment to commercially available cameras) into one processing toolbox allows for flexibility of use from research settings to field and clinical environments. The toolbox also allows for detailed kinematic evaluations when marker placement is not feasible due to sensory processing abnormalities for children with disabilities via markerless motion capture signal processing features.

Protocol

This study was conducted as a case series using representative results from four children to demonstrate protocol feasibility rather than statistical inference. Two children typically developing (male 4 years old, 100 cm, 14.5 kg; male 10 years old, 144 cm, 40.8 kg) and two children with cerebral palsy, Gross Motor Function Classification System III (female 4 years old, 93 cm, 13.8 kg; male 4 years old, 102 cm, 16.7 kg) participated in this study. Parents/legal guardians and children provided informed consent/assent prior to participation in accordance with our Institutional Review Board-approved study protocol (IRB-FY2024-33).

NOTE: While this protocol describes procedures for both 3D marker-based and 2D markerless motion capture systems, the focus of this study is on the implementation of the 2D markerless approach, which represents the core clinical and practical application of this work. The core implementation of the current protocol requires a marker-based motion capture system (minimum 8 cameras) for 3D kinematic data acquisition, and a single RGB camera (minimum resolution 1920 × 1080 pixels, 50 Hz) positioned perpendicular to the sagittal plane for 2D markerless capture. The method is suitable for controlled clinical or laboratory environments with adequate lighting (> 500 lux) and minimal background clutter. To improve robustness in non-ideal environments, the vailá toolbox integrates pre-processing tools (e.g., drawbox masking, video crop/resize) that allow users to isolate the participant from background distractions or adjust suboptimal camera framing. Key limitations include reduced accuracy for the single-camera, markerless approach for children exhibiting significant out-of-plane movements, as this approach cannot capture depth information or movements outside the camera's field of view. Users should ensure participants remain within the predefined capture volume and maintain sagittal plane alignment during task execution to optimize markerless tracking performance.

1. Pre-test checks

1. Power on and warm up the infrared motion capture camera system and the video camera. The infrared motion capture system requires approximately 30-60 min of warm-up to reach thermal stabilization before calibration and data collection. Verify the sampling rate at 100Hz (infrared cameras for marker-based system) and 50Hz (video camera for markerless system).
2. Clear and/or cover reflective objects from the capture volume.

2. Camera placement

1. Marker-based system (3D camera system)
   1. Position cameras approximately 1.5-2.5 m from the center of the movement area, mounted at heights ranging from 0.5-2.5 m. Ensure that the cameras are distributed evenly around the capture volume to optimize marker visibility and minimize occlusion during the sit-to-stand task. Alternate elevations of camera position to improve triangulation accuracy and reduce line-of-sight interference.
   2. Each point within the target volume (where the child will perform the task) must be visible to at least two cameras at all times, following standard recommendations for optimal 3D data capture. Define the capture volume based on the anthropometrics of pediatric participants and the task being evaluated.
      NOTE: The working space was centered at approximately 0.80 m above the floor.
   3. Use the marker-based system's software tools to refine the three main optical settings (zoom, aperture, focus) to ensure low calibration residuals and high marker tracking consistency across trials.
2. Markerless system (2D camera)
   1. Place the video camera laterally at 3.0 m from the participant (or closer, when possible, as long as the whole body of the subject can be seen during the entire task, from seating to standing), centered at the pelvis height when the participant is standing, orthogonal to the sagittal plane.
   2. Level the optical axis (no roll) and align the image midline with the hip joint center region.
   3. Configure the camera with fixed imaging parameters for all recordings.
      1. Set the acquisition region to an area of interest of 1920 × 1200 px to define the effective resolution. Use a 20 mm focal lens and keep the aperture (enough contrast between the environment and participant) constant throughout the experiment.
      2. Set the frame rate to 50 Hz and fix the shutter duration at 5 ms. Apply the following hardware settings and do not modify them during data collection: gain 9.998, gamma 0.4, black level 0, saturation 1.
      3. Disable all automatic exposure and white-balance adjustments to prevent temporal drift in image appearance.
         NOTE: To adjust the focus, a sign can be used with a short phrase written in font Calibri, style bold, size 48, placed where the participant will perform the task. Within the camera's field of view, the letters should be sharp, with no visible blur.
   4. When using any lens with noticeable distortion, prepare a Charuco/Checkerboard^18,19for calibration/undistortion (optional for strictly sagittal, stationary tasks).
      ​NOTE: Both marker-based and markerless video camera systems used are shown in Figure 1.

Figure 1: Close-up of the two camera systems used in this study. The 3D kinematic system's infrared camera (left) was used in which 12 cameras made up the system for the marker-based approach, and one video camera (right) was used for the markerless approach. Please click here to view a larger version of this figure.

3. Synchronization

NOTE: This step is only needed when using different systems concurrently. Synchronization between systems is not necessary when using only a single kinematic (2D or 3D) system.

1. Connect the LockLab (or equivalent control/synchronization unit) to send a transistor-transistor logic (TTL) trigger to both systems at the start of each trial.
2. If hardware triggers are unavailable, use a visible light-emitting diode (LED) flash (preferable method) or an audible clap captured by both systems for later alignment.

4. Capture volume calibration (marker-based system)

1. Visually inspect cameras' field of view (both infrared and video cameras) to remove unnecessary objects from each view. In the marker-based system's software, mask reflective objects that could not be removed from the camera's view (e.g., other cameras).
2. Perform wand calibration where the task will be performed. Move the wand throughout the full capture volume to ensure the entire space is covered. Accept calibration only if spatial accuracy ≤ 1 mm (mean residual).
   NOTE: Verify that at least three cameras simultaneously view the calibration wand during this procedure. Move the wand through the capture volume at a speed that approximates the motion expected in the experimental task: For high-velocity tasks, move the wand rapidly; for slower movements, sweep the wand more slowly. At the end of the calibration pass, check the software indicators for any cameras that have not captured sufficient views of the wand and perform additional targeted wand movements within those cameras' fields of view before concluding the calibration. Residual errors after wand calibration typically fall between 0.3 and 0.8 mm when the wand pass is smooth, the capture volume is uniformly illuminated, and all cameras have unobstructed views.
3. Set the volume origin by placing the calibration wand at the designated origin location within the defined testing space. Define the floor plane according to the manufacturer's instructions.

5. Marker placement (marker-based system)

1. Place retroreflective markers according to the Plug-in Gait^20,21,22 model (Figure 2) by a trained examiner.
2. Verify visibility of bilateral hip, knee, and trunk markers during a brief test movement prior to collecting static trial (required for 3D marker-based systems).

Figure 2: Plug-in Gait model adapted for the present study. The standard marker set was applied to define lower limb, pelvis, trunk, and head segments. Upper limb markers were removed to focus exclusively on lower-body and trunk kinematics. This adaptation preserves the integrity of the original Plug-in Gait full-body model while excluding arm motion from the analysis. Please click here to view a larger version of this figure.

6. Chair and environment

1. Use an armless chair and adjust the seat height so that the hip and knee angles are ~90°, with the feet flat on the floor. Record the seat height (cm) for each participant.
2. Provide uniform lighting; avoid backlight and occlusions in the sagittal view. Ensure a uniform, high-contrast background.

7. Participant instruction and safety

1. Confirm no identifiable information is included with recording files (only non-identifiable alphanumeric codes for each participant).
2. For sit-to-stand trials, ensure participants are aware of proper instructions and protocol.
   1. Instruct typically developing (TD) participants to keep their hands on their hips.
   2. For children with cerebral palsy (GMFCS III), provide manual support as needed to achieve an upright stance without compromising safety.
      NOTE: While pediatric patients may require manual support to ensure safety and task completion, care must be taken to minimize occlusion of body segments or markers, as this can compromise data quality.
   3. Cue the movement verbally ("up") and allow a self-selected comfortable speed during sit-to-stand trials.

8. Trial acquisition

1. Before capturing the dynamic trials, capture a static calibration trial with the participant standing in a neutral position with feet shoulder-width apart and arms at 90° of abduction.
   1. Ensure all markers are visible to at least two cameras; verify in the marker-based system's acquisition software that ≥ 95% of markers are detected across all cameras. Record for 2-3 s. One 3D perspective frame must have all markers.
   2. In the marker-based system's software, apply the Plug-in Gait static model to define joint centers and segment coordinate systems. Verify proper model fit by inspecting the 3D skeleton visualization and confirming anatomically plausible joint center positions.
      NOTE: This static trial provides the biomechanical model parameters required for subsequent 3D dynamic trial processing. In this step, one will need to input the participant's anthropometric measures (e.g., weight, height, leg length, etc.) in the marker-based system's software. When manual support is needed for pediatric patients during the static trial, care must be taken to avoid marker occlusion, which can compromise data quality.
2. Record three sit-to-stand²³ trials per participant. Resting period between trials can be as needed per participant, as this is not a physically challenging task.
3. Use a standardized file-naming convention including group (e.g., TD, CP), participant alphanumeric ID, and trial number.
4. In the marker-based system's software, reconstruct marker trajectories using the calibrated camera system.
   1. Label each marker according to the Plug-in Gait marker set nomenclature. Use automatic gap-filling (maximum gap size: 10 frames) for brief occlusions.
   2. Verify labeling accuracy by visually inspecting the skeleton in 3D perspective. Once all markers are correctly labeled and trajectories are continuous, export the trial as a .c3d file for analysis in the vailá toolbox.
5. Save optical data as .c3d and video frames or stream as lossless or high-bitrate compressed files.

9. Data processing

NOTE: The vailá toolbox (v0.10.21)²⁴ is cross-platform (Windows, macOS, Linux) and requires Python 3.12 or newer. The markerless analysis modules (e.g., MediaPipe²⁵) are optimized to run efficiently on standard CPUs, ensuring accessibility in clinical environments without specialized hardware. A CUDA-enabled GPU is optional and only utilized if the user selects alternative high-speed engines (e.g., YOLO-Pose) for faster processing. For detailed execution commands, step-by-step guides, and full documentation, users are directed to the toolbox's online resources and integrated help files, available at: https://github.com/vaila-multimodaltoolbox/vaila/tree/main/docs/modules/markerless-analysis.

1. Marker-based approach: In the marker-based system's software, process the static and dynamic trials. Apply the Dynamic Plug-in Gait model to the labeled marker trajectories (see step 7.4) to compute the joint angle time-series.
   1. Export the processed trial data as a .c3d file. Ensure that the computed joint angle time-series (e.g., hip, knee, ankle angles) are included in the export.
   2. Import the processed .c3d file into the vailá toolbox using the vaila.load_c3d() function. The toolbox automatically extracts the pre-calculated joint angle time-series data from the file.
   3. Apply the 4^th-order zero-lag Butterworth low-pass filter (6 Hz cutoff) to the joint angle data using the integrated filtering functions (as described in step 9.2.4 below) to ensure consistent processing with the markerless data.
   4. Subsequent analyses (e.g., cyclogram generation, coordination metrics computation) are then performed within vailá.
2. Markerless approach: Load MediaPipe Pose estimation module (v0.10.21) within vailá. Set the following parameters: static_image_mode = False, model_complexity = 1-2 (use level 2 when GPU/CPU resources allow higher accuracy), smooth_landmarks = True, min_detection_confidence = 0.6, min_tracking_confidence = 0.6.
   NOTE: Alternative pose estimation frameworks, such as OpenPose, can be used here.
   1. Process each video frame to extract the MediaPipe 33 landmarks (Figure 3). Discard frames with detection confidence below the threshold or apply interpolation for missing data.
      NOTE: MediaPipe detects 33 body landmarks, including face, torso, and limb keypoints (Figure 3). For the current protocol, only the trunk and lower-body landmarks relevant to sit-to-stand analysis was used: Shoulder [landmarks 11 (left shoulder) and 12 (right shoulder)], Hip [landmarks 23 (left hip) and 24 (right hip)], Knee [landmarks 25 (left knee) and 26 (right knee)], and Ankle [landmarks 27 (left ankle) and 28 (right ankle)]. These landmarks are automatically detected from the RGB video, and they are estimated from visual features (rather than retroreflective marker positions used in the Plug-in Gait model).
   2. Confirm all key landmarks (shoulder, hip, knee, ankle) are detected with confidence ≥ 0.7.
   3. (Optional) If lens distortion is relevant, undistort frames using calibration parameters (camera matrix and distortion coefficients) before running MediaPipe.
   4. YOLO-Pose: In parallel, load the YOLO-Pose module integrated in vailá.
      1. Configure the model with default weights for human skeleton detection (keypoint set consistent with COCO format). Each frame is processed to detect skeletal keypoints using the YOLO backbone, optimized for robust performance under motion blur, occlusion, or complex backgrounds.
      2. Apply temporal smoothing filters to reduce frame-to-frame jitter and align detected keypoints with the coordinate system used for MediaPipe.
      3. For the temporal smoothing filter, apply a low-pass 4^th-order Butterworth filter with a cutoff frequency of 6 Hz (sit-to-stand movements have dominant frequency content below 5 Hz) to all joint angle time-series data, both marker-based and markerless, to remove high-frequency noise while preserving the movement signal.
      4. Implement this filter by using the scipy.signal.butter() and scipy.signal.filtfilt() functions in Python, using zero-phase forward-backward filtering to avoid temporal shifts.
   5. Data handling in vailá: Export landmark trajectories from both pipelines (MediaPipe and YOLO-Pose) in a standardized .csv or .json format. Store metadata (frame number, timestamp, detection confidence) together with keypoint coordinates. This ensures comparability across methods and reproducibility of analyses.
3. Coordinate definitions and joint angles: Define sagittal-plane hip flexion and knee flexion angles with positive values for flexion.
   1. For markerless data, derive joint centers/segments from MediaPipe landmarks (e.g., hip: pelvis proxy; knee: femur-tibia; trunk: shoulder-hip segment) and compute sagittal angles using a consistent joint coordinate system.
      NOTE: Sagittal-plane joint angles are computed geometrically from the 2D landmark coordinates. The vailá toolbox calculates the angle (θ) between two segment vectors (vector a and vector b) using the arccosine of their dot product: θ = arccos((a · b) / (|a| |b|)). Hip Flexion is defined as the angle between the 'trunk' vector (vector connecting MediaPipe landmarks 11/12 [shoulder] and 23/24 [hip]) and the 'thigh' vector (vector connecting 23/24 [hip] and 25/26 [knee]). Knee Flexion is defined as the angle between the 'thigh' vector (vector connecting 23/24 [hip] and 25/26 [knee]) and the 'shank' vector (vector connecting 25/26 [knee] and 27/28 [ankle]).
   2. Apply the same sign convention and filtering (if needed) to both approaches to facilitate comparison.
4. Temporal synchronization: Align data from both approaches using the LockLab trigger time stamp; otherwise, align the first visible flash/clap frame across data streams.
   1. Resample time series, if necessary, to a common time base before comparisons.
5. Define movement cycle:
   1. Define the sit-to-stand cycle from onset of forward trunk lean (positive trunk angular velocity threshold) to stable upright stance for children typically developing (trunk angular velocity ~0 and knee full extension) and, for children with cerebral palsy, stable upright stance and highest position of cervical marker (marker-based system).
   2. For children with cerebral palsy, who may not achieve full extension (i.e., trunk aligned with pelvis, hip at 0° flexion, knees fully extended) as observed with children typically developing, define the end of the cycle as the most upright posture achieved consistent with the static standing trial, and confirmed by visual inspection to ensure a stable stance before the next trial.
   3. Do not time-normalize for the representative plots; report the actual task duration per trial. (Time-normalized analyses can be provided as supplementary, if desired.)
6. Cyclogram computation and metrics: Plot hip (x-axis) vs. knee (y-axis) angles for each trial to generate hip-knee cyclograms.
   1. Overlay trials and approaches (marker-based vs. markerless) for visual comparison.
   2. Compute optional summary metrics: hip/knee ranges of motion, loop area, loop orientation (principal axis angle), and path length.
   3. Export processed time series and metrics as .csv for statistical analysis.

Figure 3: MediaPipe's 33 landmarks. For this study, we utilized only the landmarks pertinent to the sit-to-stand task, specifically landmarks 11 and 12 (shoulders), 23 and 24 (hips), 25 and 26 (knees), and 27 and 28 (ankles). Please click here to view a larger version of this figure.

Results

The open-source Python-based toolbox successfully quantified hip-knee coordination patterns from both marker-based and markerless systems in pediatric participants during the sit-to-stand task. To provide a quantitative comparison alongside the visual cyclograms, key summary metrics (Hip and Knee Range of Motion) for the representative trials are presented in Table 1. Representative angle-angle plots from typically developing children (Figure 4A,B) exhibit c...

Discussion

The goal of this work was to demonstrate the feasibility and application of an open-source, multimodal toolbox capable of integrating marker-based and markerless motion capture systems to analyze whole-body coordination in pediatric populations. In representative applications, the protocol successfully produced hip-knee cyclograms that were comparable between marker-based and markerless systems in typically developing children, while also highlighting similarities and discrepancies in children with cerebral palsy when mo...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
FLIR Blackfly camera	FLIR Systems	Model: BFS-U3-200S6C-C	Used for 2D markerless motion capture (50 Hz, 1980×1200 px)
Hikvision PoE camera	Hikvision	N/A	Alternative video recording for 2D markerless capture
LockLab hardware trigger	Vicon	LockLab	Used for temporal synchronization
Logitech C920 camera	Logitech	HD Pro Webcam C920	Alternative consumer camera for markerless capture
MediaPipe	Google Research	v0.10.21	Used for real-time 2D pose estimation. Available online at https://ai.google.dev/edge/mediapipe/solutions/guide
OpenPose software	Carnegie Mellon University / OpenPose community	Version 1.7.0	Used for real-time 2D pose estimation
Plug-in Gait full body model	Vicon	Standard model	Used as the biomechanical model for anatomical marker placement
Vero infrared cameras	Vicon	Vero series	Used for 3D marker-based motion capture (100 Hz)