Deep Learning-Based Segmentation of Gravity-Loaded Human Spine

Summary

This article presents a protocol for segmenting the gravity-loaded human spine in weight-bearing CBCT images using a 3D U-Net convolutional neural network. The method provides a valuable tool for studying spinal alignment and supports 3D printing and biomechanics research on the spine under load-bearing conditions.

Abstract

Accurate analysis of spinal alignment under gravity-loaded conditions is essential for diagnosing and managing musculoskeletal disorders such as scoliosis and degenerative disc disease. Traditional imaging techniques, such as 2D radiographs and supine CT or MRI, do not capture the full 3D geometry of the spine in a load-bearing state. Weight-bearing cone-beam computed tomography (CBCT) provides high-resolution, 3D images in a standing position, allowing for a more accurate evaluation of spinal alignment. This study presents a deep learning-based protocol for segmenting these images using a U-Net convolutional neural network (CNN) with 3D convolutional layers and residual connections to segment the vertebral bodies, pelvis, and femoral head. The protocol includes image acquisition, manual annotation, preprocessing, and model training. Successful segmentation of the anatomical structures enables accurate measurement of clinically relevant spinal parameters such as Cobb angle and vertebral rotation. Additionally, the segmented 3D models can be exported in STL format for 3D printing, facilitating surgical planning and patient education. Furthermore, the protocol can be adapted to train segmentation models for other anatomical structures requiring assessment under gravity-loaded conditions, such as the lower limb. This method provides a reliable and adaptable tool for accurately segmenting spinal and other anatomical structures in gravity-loaded conditions, with practical applications in both clinical diagnostics and research studies.

Introduction

Accurate analysis of spinal alignment under gravity-loaded conditions is essential for diagnosing and managing musculoskeletal disorders, including scoliosis, degenerative disc disease, and spinal deformities^1,2,3,4. Traditional imaging techniques, such as 2D radiographs, do not capture the full 3D geometry of the spine. Although conventional CT and MRI can produce 3D images, these modalities require patients to be imaged in a supine position. In contrast, weight-bearing cone-beam computed tomography (CBCT) provides high-resolution, 3D images with the patient in a standing position, offering a more detailed evaluation of spinal alignment⁵.

Segmentation of the skeletal structure from 3D imaging is crucial for extracting clinically relevant spinal parameters, such as the Cobb angle, vertebral rotation, and pelvic incidence, which are necessary for diagnosing spinal deformities and planning surgeries⁶. Additionally, segmentation enables the creation of 3D models of anatomical structures, supporting patient education and surgical preparation⁷. Deep learning technologies, particularly convolutional neural networks (CNNs), have been widely used in medical imaging analysis, notably in the segmentation of complex anatomical structures from images^8,9. Deep learning-based segmentation reduces manual workload, minimizes operator variability, and improves accuracy and consistency in spinal imaging.

This article presents a deep learning-based protocol for segmenting gravity-loaded spines using weight-bearing CBCT. A 3D U-Net CNN is used to automatically segment the vertebrae, pelvis, and femoral head under load-bearing conditions. Integration of this method with routine weight-bearing CBCT provides a standardized, reproducible tool for more accurate spinal assessments, thereby supporting improved clinical decision-making and patient outcomes. This approach is replicable and adaptable for training segmentation models for other anatomical structures that require assessment under gravity-loaded conditions, such as the lower limb.

Protocol

This study was conducted in accordance with the ethical standards outlined in the Declaration of Helsinki and was approved by the Institutional Review Board of West China Hospital, Sichuan University (approval 834 / 2021). Participants included in the study were diagnosed with either degenerative scoliosis or adolescent idiopathic scoliosis (AIS). Exclusion criteria encompassed individuals with a history of spinal surgery, the presence of spinal implants, or those unable to maintain a standing position during the imaging procedure. All participants involved in the study provided informed consent prior to image acquisition and participation in the research. The privacy and confidentiality of all participants were strictly maintained throughout the study. The equipment and software used in this study are listed in the Table of Materials.

1. Imaging of the spine

1. Imaging system and patient positioning
   1. Use the weight-bearing CBCT system to acquire the image (Figure 1).
   2. Place the patient on the standing platform. Secure the patient with belts to prevent movement and falls.
   3. Instruct the patient to maintain a standing position, stable breathing, and minimize motion during the scanning process.
2. Image acquisition
   1. On the imaging system's software, define the imaging range.
      NOTE: For full-spine imaging, the range should include the first cervical vertebra down to both femoral heads.
   2. Choose the appropriate exposure parameters.
      NOTE: The default parameters are shown in Table 1. Adjust the parameters based on the patient's body size as necessary.
   3. Press the foot brake to initiate exposure. Continue until the software indicates that image acquisition is complete.
3. Image reconstruction
   1. Click the Reconstruction button on the software and adjust the reconstruction parameters as needed.
      NOTE: The reconstruction parameters used in this study are listed in Table 1.

2. Annotation using ITK-SNAP

1. Software launching and image importing
   1. Double click the software icon to launch ITK-SNAP, an open-source, multi-platform tool for 3D biomedical image segmentation and annotation¹⁰.
   2. Import the image for annotation. Click on File -> Open Main Image -> Browse. Select the target image file and click on Open to load it into the software.
2. Annotation
   NOTE: It is recommended to read the ITK-SNAP tutorial for manual segmentation¹¹ to familiarize yourself with the software's basic tools and functions. The essential steps for this experiment are outlined below.
   1. To improve image visibility, adjust the image contrast by selecting Tools -> Image Contrast -> Contrast Adjustment -> Curve-Based Image Contrast Adjustment. Adjust the curve as needed to enhance contrast.
   2. Create Labels for anatomical structures: Press the Edit Labels button on the toolbar. In the Description text box, type the names of the segmentation categories (e.g., spine, pelvic, femoral heads). Assign colors to each segmentation categories and press Close Window button to apply the changes.
   3. Manual segmentation using the polygon annotation tool: Ensure that the correct label is selected from the Active Drawing label drop-down menu in the Segmentation Label panel, matching the category you are about to segment.
   4. Select All labels in the Draw Over drop-down menu. Select the Polygon Tool in the IRIS toolbox.
   5. Use the polygon tool to manually segment the structures by drawing and filling polygons on the image. Zoom and pan as needed to ensure accuracy.
3. Saving the annotation
   1. Once the annotation is complete, click on Segmentation in the menu bar, then select Save Segmentation Image.
   2. In the dialog box, set the Image Filename (e.g., xxx.nii.gz), select File Format as NIfTI, and click on Finish to save the annotated file.

3. Image preprocessing

NOTE: In this protocol, image preprocessing for the input image and the corresponding target label data is implemented using PyCharm, an integrated development environment (IDE) for Python. The steps are executed through Python code, which is provided in Supplementary File 1 as pseudo-code to improve readability and clearly outline the key steps.

1. Import libraries and image loading
   1. Open PyCharm and create a new Python script.
   2. Import the required libraries for image processing and data manipulation, including NumPy, SimpleITK, and PyTorch (Supplementary File 1, Code line 1).
   3. Use SimpleITK to read the grayscale image and the corresponding label (segmentation mask) from their respective file paths (.nii.gz) (Supplementary File 1, Code lines 2-3).
2. Image resizing and normalization
   1. Set the input size for the z-axis to 16 slices, which balances memory usage and retains essential structural information for segmentation (Supplementary File 1, Code line 4).
   2. Add 1000 to the grayscale values of the image and clip values below 0 and above a threshold that is dynamically set between 1000 and 3000 (Supplementary File 1, Code line 5).
   3. Normalize the grayscale values of the image by dividing by the threshold (Supplementary File 1, Code line 5).
3. Image augmentation
   1. Apply Poisson noise to simulate real-world noise in the image (Supplementary File 1, Code line 6).
   2. Randomly adjust the brightness by a factor between 0.8 and 1.2 to account for varying imaging conditions (Supplementary File 1, Code line 7).
   3. Use a sliding window approach to extract 3D slices along the z-axis for both the image and the corresponding target label (Supplementary File 1, Code lines 8-10).
   4. Concatenate the image and target slices along the batch dimension to ensure they are processed together during augmentation (Supplementary File 1, Code line 11).
   5. Translate the image and target slices randomly along the x and y axes by a random value (Supplementary File 1, Code line 12).
   6. Apply random rotation on the concatenated image and target slices to simulate different imaging orientations (Supplementary File 1, Code line 13).
4. Return preprocessed data
   1. Return the preprocessed image and target label data in torch.tensor format for use in neural network training (Supplementary File 1, Code line 14).

4. Model training

1. Import libraries and define U-Net architecture
   1. Import the required libraries for model training, including PyTorch, torch.utils.data.DataLoader, and torch.optim.Adam (Supplementary File 1, Code line 1).
   2. Use the U-Net model with 3D convolutional layers and residual connections (Supplementary File 1, Code lines 2).
2. Initialize training parameters and data loading
   1. Set up the Dice loss function, Adam optimizer, and model. Set alpha to 0.8, the learning rate to 1e-5, and the number of epochs to 500 (Supplementary File 1, Code lines 3).
   2. Fetch the data and corresponding target labels from the torch.utils.data.DataLoader for each batch.
3. Training loop and forward pass
   1. Decrease the learning rate by multiplying it by 0.8 every 100 epochs to ensure the model fine-tunes its learning over time (Supplementary File 1, Code line 5).
   2. Reset the gradients of the optimizer to zero before processing each batch to ensure that previous gradients do not accumulate (Supplementary File 1, Code line 7).
   3. Input the data batch into the model to generate predictions for segmentation (Supplementary File 1, Code line 8).
   4. Calculate the Dice loss at four different depths (loss_0, loss_1, loss_2, loss_3) to capture information from different layers of the network (Supplementary File 1, Code lines 9-12).
4. Total loss, backpropagation, and optimization
   1. Calculate total loss using the formula: total loss = loss_3 + alpha × (loss_0 + loss_1 + loss_2) (Supplementary File 1, Code line 13).
   2. Compute the gradients by performing backpropagation on the total loss to update the network's weights (Supplementary File 1, Code line 14).
   3. Update the model weights using the optimizer to adjust the network parameters based on the computed gradients (Supplementary File 1, Code line 15).
5. Adjustments for alpha
   1. Every 30 epochs, reduce the value of alpha by multiplying it by 0.8 to reduce the influence of earlier layer losses as the training progresses (Supplementary File 1, Code line 16).

Results

Segmentation results
In this study, three anatomical structures-the spine, pelvis, and femoral heads-were segmented. The pelvis and femoral heads were included because certain spine parameters require landmarks from these structures for measurement. The dataset for each structure was divided into training and testing sets as outlined in Table 2.

The segmentation performance was evaluated using three standard metrics, including Dice coefficient, Volume Similarity (VS), and Recall:

Where TP denotes True Positive, FP denotes False Positive, FN denotes False Negative.

After training the network for 500 epochs, the segmentation results were evaluated based on the metrics. The performance for each anatomical structure is presented in Table 3. These results demonstrate the effectiveness of the proposed U-Net-based segmentation model across the spine, pelvis, and femoral heads. Figure 2 displays a representative segmentation result of the anatomical structures.

Post-segmentation applications

Measurement of 3D clinically relevant parameters
The 3D models generated through automatic segmentation can be used to measure a range of clinically important parameters, such as the Cobb angle, vertebral rotation, thoracic kyphosis, lumbar lordosis, pelvic incidence, pelvic tilt, and sacral slope. Here, we display the measurement of these parameters using a self-developed program, which processes the segmented data and provides accurate and automatic measurements of the spine and associated structures (Figure 3).

3D printing
The 3D model obtained by AI automatic segmentation and post-processing can be exported to STL format, a general 3D model file format, which is widely supported by 3D printing software. For this study, the 3D printer used was an industrial stereolithography (SLA) printer, which utilizes ultraviolet (UV)-curable resin as the printing material. Connections were applied among the isolated parts of the segmented model, ensuring a cohesive structure that can be printed as a single, unified model. This process enables the physical replication of the patient's anatomy, which can be used for surgical planning, patient communication, and educational tools. Figure 4 demonstrates a 3D-printed model derived from the AI-segmented data.

Figure 1: Patient positioning for weight-bearing cone-beam computed tomography (CBCT) imaging. The patient is placed on the standing platform and secured with belts to prevent movement and fall during the scan. The patient holds onto the overhead support to maintain a stable position while the CBCT system captures 3D images of the spine under gravity-loaded conditions. Please click here to view a larger version of this figure.

Figure 2: Segmentation results for spine, pelvis, and femoral head in axial, sagittal, and coronal views. The first column shows the CBCT images, followed by the ground truth, and then the segmentation results obtained from the deep learning model. The red overlays in the second and third columns indicate the segmented areas for the vertebral bodies, pelvis, and femoral head across all three views. The images demonstrate the high accuracy of the model in capturing anatomical structures, closely matching the ground truth. Please click here to view a larger version of this figure.

Figure 3: Measurement of clinically relevant parameters on segmented spine and pelvis structures. (A) Measurement of Cobb angle (8.9°) between vertebrae T11 and L3 in the coronal plane. (B) Measurement of thoracic kyphosis (T1-T12: 21.2°) in the sagittal plane. (C) Measurement of lumbar lordosis (L1-L5: 26.4°) in the sagittal plane. (D) Vertebral rotation (VR: 3.3°) of L2 relative to the pelvis in the axial plane. (E) Pelvic parameters, including pelvic tilt (PT: 21.0°), sacral slope (SS: 26.1°), and pelvic incidence (PI: 47.1°), measured on the sagittal view. Please click here to view a larger version of this figure.

Figure 4: STL model and 3D-printed physical model of the spine and pelvis. The image on the left shows the 3D model of the spine and pelvis in STL format, prepared for 3D printing. The image on the right displays the corresponding physical model printed using stereolithography (SLA) technology. Please click here to view a larger version of this figure.

Parameters	Default Settings
Tube voltage	110 kv
Tube current	6 mA
Frame rate	12 fps
Reconstruction FOV	350 mm
Slice thickness	2 mm
Spacing Between Slices	1 mm

Table 1: Default parameters for exposure and reconstruction. Default exposure and reconstruction parameters for weight-bearing cone-beam computed tomography imaging of the full spine.

Label	Training dataset	Testing dataset
Spine	100	20
Pelvis	57	6
Femoral head	47	5

Table 2: Dataset used for CBCT image model training and testing. The table shows the number of images used to train and test the model for segmenting the spine, pelvis, and femoral head structures.

Label	Dice Coefficient	Volume Similarity	Recall
Spine	0.93	0.954	0.89
Pelvis	0.925	0.965	0.93
Femoral head	0.96	0.971	0.959

Table 3: Segmentation performance of the deep learning model for different anatomical structures (spine, pelvis, and femoral head). Performance metrics include Dice Coefficient, Volume Similarity (VS), and Recall, all of which range from 0 to 1, where values closer to 1 indicate better segmentation accuracy.

Supplementary File 1: Pseudo-codes for Image preprocessing (step 3) and Model training (step 4). Please click here to download this File.

Discussion

This protocol introduces a deep learning-based method for segmenting gravity-loaded spinal images using weight-bearing CBCT. Key steps include image acquisition, manual annotation, preprocessing, and model training. Proper patient positioning and exposure settings are critical for obtaining high-quality images. Misalignment or poor positioning can lead to suboptimal images, making it difficult for the neural network to segment the structures accurately. Preprocessing is also an essential step, and in this protocol, it includes noise addition, brightness variation, rotation, and translation. These preprocessing techniques are designed to prepare the images for input into the neural network, enhancing the model's robustness by exposing it to various types of image distortions.

Manual annotation is another critical component of the protocol. The accuracy of manual annotations directly impacts the model's ability to learn and segment structures like the vertebral bodies, pelvis, and femoral head. This protocol focused on segmenting vertebral bodies, which is sufficient for measuring most clinically relevant parameters like the Cobb angle and vertebral rotation. For applications requiring the full vertebra (including lamina, transverse processes, and spinous processes), the model would need training with additional labeled data. Additionally, variability in manual annotations can also be a challenge, potentially impacting the reproducibility of results. To mitigate this, we recommend having multiple experienced annotators and implementing inter-observer reliability checks.

The U-Net architecture, with 3D convolutional layers and residual connections, is well-suited for handling 3D images^12,13,14. The multi-level loss function helps preserve details from deeper layers, ensuring segmentation accuracy. However, careful tuning of training parameters, such as learning rate and augmentation settings, is necessary to prevent overfitting and improve model robustness. While recent studies have proposed spine segmentation or detection models tailored to specific tasks using neural networks such as transformers or YOLO architectures^15,16, these typically rely on CT or MRI spine images. In contrast, this protocol adopts the U-Net architecture, which is more commonly used in pixel-level segmentation task¹⁷. More importantly, this method is designed to segment the spine under gravity-loaded conditions using upright CBCT, allowing for an assessment of the spine in a standing or weight-bearing state. This unique feature provides a more realistic evaluation of the spine's alignment and mechanical stresses, which is crucial for diagnosing spinal deformities and planning treatment strategies. Notably, the model's performance, as demonstrated by a Dice coefficient of 0.93 in spine segmentation, is comparable to or even exceeds the performance of other deep learning-based models that use CT and MRI data for spinal segmentation^9,18,19.

This method has some limitations. The dataset used in this study, while adequate for segmenting the vertebral bodies, pelvis, and femoral head, would benefit from expansion to better handle rare or complex spinal deformities. A larger, more diverse dataset would help the model generalize to a wider range of cases. Another limitation is that CBCT imaging is sensitive to metal artifacts, making it difficult for the model to segment post-operative spines with metal implants. Additionally, this method is limited by its high computational demand, requiring access to powerful GPUs, which may not be available in all clinical and research settings. The manual annotation process is time-consuming, and due to this, we were unable to involve multiple annotators to verify the accuracy of the annotations through consensus. Furthermore, the model has not yet been validated on external datasets, which limits its generalizability to other clinical settings.

The ability to accurately segment and measure spine-related parameters enhances diagnosis and treatment planning, especially in cases like scoliosis. The capability to export models in STL format for 3D printing also has significant applications in preoperative planning, allowing surgeons to practice or reference patient-specific models. These tangible models are valuable tools for patient education, helping patients better understand their condition and treatment²⁰. Beyond spine imaging, this protocol can also be adapted to segment other gravity-loaded anatomical structures, such as the lower limbs and joints, which is particularly valuable in cases like knee osteoarthritis or hip deformities. Furthermore, this protocol has potential applications in biomechanics studies, where detailed 3D models of the spine or other structures under load-bearing conditions are crucial for analyzing mechanics and stress distributions^21,22. This could provide valuable insights into the progression of deformities and guide the development of new treatment strategies.

In conclusion, this protocol offers a reliable and efficient method for segmenting and analyzing spinal images in gravity-loaded conditions. While there are some limitations, its potential applications in diagnosis, surgical planning, biomechanics studies, and education make it a valuable tool in both clinical and research scenarios.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
3D Printer (G1400)	UnionTech, Shanghai, China	G1400	Industrial stereolithography (SLA) 3D printer.
CUDA version 11.4	NVIDIA	11.4	GPU acceleration for deep learning model training.
ITK-SNAP	Open source	4.0.2	Software for 3D biomedical image segmentation and annotation.
Numpy	Open source	1.24.4	python library for image processing, website: https://numpy.org/about/
PyCharm	JetBrains	2023.2.1	an integrated development environment (IDE) used for Python programming.
Python version 3.8	Open source	3.8	Used in combination with PyTorch for deep learning model development.
PyTorch version 1.10	Open source	1.10	Deep learning framework used for training the segmentation model.
SimpleITK	Open source	2.3.1	python library for image processing, website: https://simpleitk.org/
UV-Curable Resin	UnionTech, Shanghai, China	UV-001	Material used in the SLA 3D printing process.
Weight-bearing cone-beam CT system  (WR3D)	Angell Technology, Shenzhen, China	DX6290	Used for image acquisition of the full spine in gravity-loaded condition
Windows 10 System with Intel i7-10700 CPU	Custom build (Various Manufacturers)	Windows 10 Professional,22H2	Running environment for model training; 32GB RAM, NVIDIA GeForce RTX 3090 (24GB).