Name: Automated Joint Space Detection Improves Bone Segmentation Accuracy (Video) | JoVE Journal
Uploaded: 2025-11-28T12:50:00.000+00:00
Duration: 6 min 45 s
Description: Learn an automated joint space detection workflow using deep learning and watershed segmentation in Amira to achieve over 98% accuracy in murine hindpaw bone segmentation. This computational method enables high-throughput quantitative assessment of bone health and osteoarthritis progression with minimal manual correction.

ai_agent_IP

ai_agent

maintenance

instantTrialDurationMinutes

isBiopharma

enableNewMyJoveSectionDesign

educationFeatures

seoMetaApi

ai_copilot_institution_blacklist

ai_chat_institution_blacklist

freeAccessWindow

ssoRoleCapture

scheduledMaintenanceBanner

joveLabsFeatures

disable_autoplay_for_bots

extended_trial_duration

5 minutes

Funding Sources:&#160;F30AG076326 (HMK), T32GM007356 (HMK), R01AR069000 (CTR), R01AR056702 (EMS), and P30AR069655 (LS, EMS, and HAA). HMK was a trainee in the Medical Scientist Training Program funded by NIH T32GM007356. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of General Medical Science or NIH. We would like to thank the faculty and staff of the Histology, Biochemistry, and Molecular Imaging core, the Biomechanics, Biomaterials, and Multimodal Tissue Imaging core, and the Center for Musculoskeletal Research at the University of Rochester Medical Center for their contributions to this work.

All animal experiments were performed in accordance with IACUC protocols approved by the University Committee for Animal Resources at the University of Rochester Medical Center.
Animal models 
The mice were housed in an AAALAC-accredited vivarium. A total of 19 mice were used for the described experiments, including 4 wildtype (WT) male, 4 WT female, 4 TNF transgenic (TNF-Tg) male, and 7 TNF-Tg female mice, with longitudinal monthly assessments. TNF-Tg mice (3647 line, C57BL/6 genetic background30) were initially obtained from Dr. George Kollias with continued maintenance at the University of Rochester. The TNF-Tg mice were bred as heterozygotes, with WT mice serving as littermate controls. Genotyping for the TNF transgene was performed using the following primer sequences: TNF-Tg Forward: 5-TAC-CCC-CTC-CTT-CAG-ACA-CC-3; TNF-Tg Reverse: 5-GCC-CTT-CAT-AAT-ATC-CCC-CA-3.
TNF-Tg mice develop chronic, progressive, and spontaneous inflammatory-erosive arthritis31 with more rapid onset of articular and extra-articular manifestations in female mice, leading to early mortality at approximately 5-6 months32. Thus, additional mice were allocated to the TNF-Tg female cohort. Indicated in previous descriptions of this study cohort23, n=2 TNF-Tg females died prior to completion of the study, n=1 before 4 months, and n=1 before 5 months. As the TNF-Tg mice exhibit well-established asymmetric disease progression in hindpaws33,34, individual limbs were considered the unit of measure (2 forepaws and 2 hindpaws per animal).
Micro-CT image collection 
Micro-CT datasets were collected as previously described12,23. Briefly, the mice were placed in a Derlin plastic and clear acrylic tube with 1%-3% isoflurane anesthesia for imaging with a micro-CT using the following parameters: 55 kV, 145 µA, 300 ms integration time, 2048 x 2048 pixels, 1000 projections over 180°, resolution 17.5 µm isotropic voxels. For safety when using isoflurane, proper personal protective equipment (nitrile gloves, lab coat or gown, safety glasses) was utilized, and the isoflurane vaporizer was maintained within a fume hood using a charcoal filter to capture gaseous waste with well-sealed tubing for continuity in the delivery system. The vaporized isoflurane was utilized for anesthesia induction in a sealed chamber, then anesthesia was maintained by continuous isoflurane flow into a murine nose cone in the micro-CT machine. Both the hindpaws and forepaws were taped together for stabilization during imaging sessions. Each dataset was collected in approximately 30-45 min (60-90 min total) with hindpaw and forepaw data derived from the same animals and timepoints. The mice were evaluated at monthly intervals starting at 2 months of age until 5 months (females, TNF-Tg with early mortality32) or 8 months (males). Portions of the hindpaw data utilized for this study were previously published (WT:12,35; WT and TNF-Tg:23) and are publicly available28. The forepaw data were made publicly available for the purposes of this study29.
Joint space segmentation algorithm with deep learning facilitation 
A high-throughput SA image processing algorithm to segment the individual bones of the complex hindpaw in mice (30-31 bones) was previously developed12, which provided a framework to investigate individual biomarkers of inflammatory-erosive arthritis progression using Amira software23. This baseline SA segmentation method utilized a marker-based watershed algorithm36 for separation at bone boundaries. The marker-based watershed algorithm separates different objects in an image by treating pixel values as local topography based on user-defined markers. These bone-specific markers were generated in an SA manner through a variety of image processing steps, including black top-hat (BTH), in an effort to highlight local regions of large density changes, such as bone edges and articulations. Together, this approach created an eroded version of each individual bone, which was then expanded to the bone borders through the application of a binary boundary mask. While the watershed method improved upon previous utilization of manual contouring, the accuracy (bones segmented correctly / total bones) was approximately 80% per dataset due to low contrast noise leading to bridged joint regions (over-connecting bones; 2+ bones as one material) or edge misidentification (over-splitting bones; 1 bone as 2+ materials). Thus, the SA marker generation for the watershed approach required consistent and frequent manual correction procedures12 to develop a resource of gold-standard labels23,28.
The development of the DL model for bone joint prediction was based on the architecture of a 3D U-Net with a ResNet-18 backbone. The training loss function was Dice, the validation metric was intersection over union (IoU), the gradient descent used Adam optimization with an initial learning rate of 0.0001, and weights were initialized randomly. The model was trained using 20 WT datasets (40 hindpaws) with equal sex and age (from 2-6 months) distribution, each split into 6 subvolumes (3 per hindpaw) of 200 x 200 x 200 voxels with a 25% randomized validation set of subvolumes (30 validation, 120 total) to avoid overfitting. These 3D tiles were positioned evenly on the tarsals, distal phalanges, and background regions. The training patch size was set at 96 x 96 x 96 voxels (Supplementary Figure 1). The model was trained over 500 epochs, taking approximately 6 h. Ground truth joint regions were obtained by an automatic recipe from ground truth labels, which expanded label interfaces with a 3D dilation size of 5 for both thickness and extent.
In conjunction with the joint space DL prediction, several image processing steps were implemented to augment joint space identification and bone segmentation. These strategies included the utilization of the BTH method combined with structure enhancement37, membrane enhancement, and tensor voting38 to reinforce joint space continuity by limiting membrane gaps. Together, these approaches fortify the joint spaces for bone separation to limit the segmentation leakage between adjacent bones that then generates over-connection errors as the watershed algorithm propagates across multiple bones. The final result separates and segments original micro-CT datasets into bone-specific labels.
A detailed step-by-step protocol of the DL facilitated segmentation method is provided below.
Step 1: Open Amira software (requires personal or institutional license). Step 2: Open Python tab, select Create New Python Environment with environment name: deep-learning-environment-2022_2. Check Install Deep Learning Packages. Step 3: Restart software, open Python tab, select user environment deep-learning-environment-2022_2. Step 4: Open data - DICOM stacks can be loaded by selecting all individual DICOM files or opening .am files that contain embedded DICOM stacks, if available. Step 5: Apply Deep Learning Prediction module to the imported data object with the following details: 
Data: Imported data object 
Architecture: .json file (Supplementary File 2) 
Weights: .hdf5 file (Supplementary File 3) 
Tiling: manual - optimization possible by reducing tiling pixel size and increasing tiling overlap as able, depending on computational hardware. The module will fail if processing requirements are insufficient. Ensure in Edit, Preferences, Large data that memory allocation for the software is maximized as much as possible. 
Tiling width, height, depth: 352 pixels 
Tiling overlap: 0 pixels 
Step 6: Apply Image Recipe Player, right-click in the Project Area without targeting a specific data object. Input/assess the following: 
Open recipe: .hxisp file (Supplementary File 4) 
Data: Imported data object 
Input joints: Result of Step 5 (Deep Learning Prediction) 
Step3 intensity range: 2500 - 20000 
Step 7: Evaluate the resulting processed data object that contains the final segmentation - ensure to adjust Colormap to Labels256 to appreciate the total individual resulting segmentations (default is only 8 colors). For 2D: Apply Ortho Slice and for 3D: Apply Volume Rendering. Details on embedded recipe (BTH+DL+SEF+MEF_D2.hxisp) applied in Image Recipe Player step provided below (Supplementary File 5). Note that in Image Recipe Designer, individual steps can be visualized and exported as needed to evaluate steps for optimization in particular datasets. In the protocol, particular steps are highlighted that will need adjustment for any unique application, as these are dependent on imaging outputs (i.e., density) and/or size of objects (i.e., bones): 
Step 1: Apply Median Filter with Data: Imported data object; Interpretation: 3D; Neighborhood: 26; Iterations: 3; Type: Iterative. 
Step 2: Apply Thresholding - this step will require optimization depending on datasets and the particular threshold that targets the object of interest, in this case, the bone with Data: Result of Step 1 (Median Filter); Intensity range: 2500 - 20000. 
Step 3: Apply Closing - this step will require optimization depending on size of joint spaces between bones with Data: Result of Step 1 (Median Filter); Type: Cube; Interpretation: 3D; Neighborhood: 26; Pixel size: 3. 
Step 4: Apply Image Arithmetic with Input A: Result of Step 3 (Closing); Input B: Result of Step 1 (Median Filter); Result channels: like input A; Expression: A-B. 
Step 5: Apply Thresholding with Data: Result of Step 4 (Image Arithmetic); Intensity range: 750 - 20000. 
Step 6: Apply Image Arithmetic with Input A: Result of Step 2 (Thresholding); Input B: Result of Step 5: (Thresholding); Result channels: like input A; Expression: A-(B&gt;0) 
Step 7: Apply Structure Enhancement Filter with Input image: Imported data object; Interpretation: 3D; Tensor type: Hessian; Standard deviation min/max: 1 - 3 pixels; Standard deviation step: 1 pixel; Contrast: Dark; Structure type: Plane. 
Step 8: Apply Auto Thresholding with Input image: Result of Step 7 (Structure Enhancement Filter); Type: Auto Threshold High; Interpretation: 3D; Mode: Min-max; Criterion: Factorisation. 
Step 9: Apply Membrane Enhancement Filter with Data: Imported data object; Output selection: Planeness Tensor Voting; Tensor voting scale: 3 pixels; Densification scale: 3 pixels; Type: Ridge 
Contrast: Dark; Scale: 1 pixel. 
Step 10: Apply Auto Thresholding with Input image: Result of Step 9 (Membrane Enhancement Filter); Type: Auto Threshold High; Interpretation: 3D; Mode: Min-max; Criterion: Factorisation. 
Step 11: Apply Dilation - this step will require optimization depending on the size of joint spaces - with Input image: Result of Step 10 (Auto Thresholding); Type: Ball; Interpretation: 3D; Size: 1 pixel; Precision: Faster. 
Step 12: Apply Image Arithmetic with Input A: Result of Step 11 (Dilation); Input B: Result of Step 8 (Auto Thresholding); Input C: Result of Deep Learning Prediction; Result channels: like input A; Expression: A||B||C. 
Step 13: Apply Remove Small Spots with Input image: Result of Step 12 (Image Arithmetic); Interpretation: 3D; Size: 500 pixels. 
Step 14: Apply Image Arithmetic with Input A: Result of Step 13 (Remove Small Spots); Input B: Result of Step 6 (Image Arithmetic); Result channels: like input A; Expression: !A*B. 
Step 15: Apply Remove Small Spots with Input image: Result of Step 14 (Image Arithmetic); Interpretation: 3D; Size: 500 pixels. 
Step 16: Apply Labeling with Input image: Result of Step 15 (Remove Small Spots); Interpretation: 3D; Neighborhood: 26. 
Step 17: Apply Convert Image Type with Data: Result of Step 1 (Median Filter); Output type: 16-bit unsigned; Normalization mode: Scaling; Scaling: Scale 3, Offset 2000. 
Step 18: Apply Marker Based Watershed Inside Mask with Data: Result of Step 17 (Convert Image Type); Markers: Result of Step 16 (Labeling); Binary Mask: Result of Step 6 (Image Arithmetic); Split type: Low Intensity.
Segmentation method testing and quantification 
The segmentation method was tested through the generation of a recipe that incorporated the DL joint prediction with the downstream image processing recipe on an intensity range of 2500 - 20000 Hounsfield units. Recipe generation allowed for batch processing (Apply a Recipe on a Batch of Files) of the original micro-CT datasets (.am file format as image stack saved after import of initial .dcm files into Amira). Computer hardware included 16 cores from an Intel Xeon Gold 5218 central processing unit (CPU) at 2.30 GHz, 128 GB of double data rate fourth generation (DDR4) error correction code (ECC) random-access memory (RAM) at 2666 megatransfers (MT)/s, and 24 GB of virtual graphics processing unit (vGPU/VRAM) on a 64-bit operating system running Windows 10 (operating system build: 19044.4780). Each hindpaw dataset (2 hindpaws) was segmented in approximately 32.7 ± 8.42 min (mean ± standard deviation) without user intervention. This is compared to the prior SA model, where segmentation time was dependent on user experience, with novice users at 40.5 ± 9.06 min per dataset and experienced users at 19.3 ± 5.34 min per dataset (WT datasets only, including correction of segmentation errors)12. Both DL and SA methods demonstrate remarkable improvements from prior gold standard manual contouring for segmentation at 190.6 ± 30.4 min per dataset by an experienced user (performed using conventional Scanco analysis)12. Forepaw datasets (2 forepaws) were segmented in approximately 53.4 ± 23.6 min without user intervention, where increased segmentation time can be attributed to additional structures within the original imaging datasets (i.e., spine and ribs), which are not present in the more distally isolated hindpaws and inflate the segmentation time in the absence of preceding volume editing steps.
Quantification of accuracy was performed by visual inspection (HMK) to identify correct segmentation or error type based on expected bone anatomy (Hindpaw:12,39; Forepaw:40). Accuracy was calculated as a percentage by:
<img alt="Accuracy formula, equation for model evaluation, diagram illustrating true/false positives/negatives." src="/files/ftp_upload/69252/69252eq1.jpg" style="margin: auto;" data-altupdatedat="1780977605000" data-alt="Equation 1">
where true positives were correctly segmented bones, true negatives equal 0 (there are no circumstances in which bones should be missing, and background was not relevant for quantification), false positives were over-split bones, and false negatives were over-connected bones. Accuracy was determined to be an appropriate quantitative metric given the single-class problem (i.e., identifying joint spaces), and true negatives (i.e., background) were non-contributory in accuracy calculations, thus reducing the risk of overestimating performance. The automated segmentation method does not include bone naming; the bone names are later associated with segmented materials manually by the user.
Evaluation of the hindpaws involved in this study has confirmed the fixed fusion in the tarsals of the navicular and lateral cuneiform (NAVLAT) in C57BL/6 mice41, and additionally determined that the adjacent intermediate cuneiform (INT) may also be variably fused with the NAVLAT structure (NAVLATINT)12,39. Similar variable fusion was appreciated in the carpal region of forepaws, where the trapezoid (ZOID; lesser multangular) and centrale (CENT) bones may present as either a single fused structure (CENTZOID) or subdivided into their individual bones, particularly in forepaws. Additional carpal bones investigated for segmentation accuracy included the trapezium (ZIUM; greater multangular), capitate (CAP), hamate (HAM), triquetrum (TRI; triangular), pisiform (PIS), scaphoid (navicular)/lunate (SCAPHATE; fixed fusion), and falciformis (FALC). The forepaw metacarpals (MET-F; 1-5), proximal phalanges (PP-F; 1-5), distal phalanges (DP-F; 2-5), and sesamoids (S-F; 1-10) are numbered lateral to medial, as opposed to hindpaw counterparts (metatarsals (MET-H), PP-H, DP-H, and S-H), which are numbered medial to lateral12. Along with NAVLATINT, additional tarsal bones were evaluated for hindpaws, as previously described12,23, including the calcaneus (CALC), cuboid (CUB), medial cuneiform (MED), talus (TAL), and tibiale (TIB). Note the overall cohort accuracy quantifications vary slightly when comparing the assessment of average accuracy per dataset with a variable number of bones due to anatomical fusions versus the accuracy calculated based on the total individual bones analyzed.
Statistical analysis 
Statistical analysis, including 3-way or 2-way mixed-effects analysis with interaction effects or Sidak's multiple comparisons and Fisher's exact test, was performed as appropriate in GraphPad Prism (v10.2.0; San Diego, CA, USA). Males (2-8 months) and females (2- 5 months) were analyzed separately, given the distinct timeframes of evaluation based on early TNF-Tg female mortality32. The sample sizes of the WT hindpaws used for training/validation and methodological testing are provided in Supplementary Table 1, along with sample size details for tested WT and TNF-Tg hindpaws and forepaws in Supplementary Table 2, Supplementary Table 3, and Supplementary Table 4. As certain timepoints for WT hindpaw testing incorporated accuracy evaluation for &lt;3 hindpaws, interaction effects were reported without post-hoc multiple comparisons in analyses that included WT hindpaws. Entire or portions of hindpaws were omitted from analysis if there were imaging errors with incomplete capture of the paw, considerable motion artifact rendering the scans uninterpretable, and/or if the animal died prior to a scheduled imaging session, as all data were collected in vivo.

Daniel Lichau and R&#233;mi Blanc are employees of ThermoFisher Scientific involved in the development and maintenance of the Amira software used to produce the methods described in this manuscript. All other authors have nothing to disclose.

To the end of fully automated analyses of bone volumes in mice, we generated further improvements in the segmentation of micro-CT data in complex structures, particularly murine hindpaws. The strategy was to target the articulating joint spaces to create boundaries for bone separation, where focusing on the negative space between bones enabled flexible implementation in alternative structures like the forepaw, since the approach was not particular to the shape and anatomy of the distinct hindpaw bones. Although the segmentation accuracy decreased when performed on forepaws, WT datasets still demonstrated bone accuracy of &#62;85%. Well-described corrective processes12 could be applied to create pre-annotated forepaw model datasets for DL training, dramatically lowering barriers for creation of structure-specific algorithms. This novel approach also allowed for application to TNF-Tg paws with severe and progressive inflammatory-erosive arthritis. In TNF-Tg paws, the decline in segmentation accuracy was striking over time, corresponding to the progressive increase in bone erosions and eventual pathologic bone-bone fusions from remodeling eroded surfaces with increased age. Thus, the remarkably successful application of an automated and highly accurate segmentation model in WT structures has the potential to guide future applications in disease models or other complex joints. Further investigation will focus on optimizing segmentation of arthritic joints that may quantify pathologic effects of bone erosions and fusions to identify disease biomarkers, as previously described23.
Despite the successful utilization of micro-CT imaging to monitor erosion of small bones in pre-clinical arthritis models12,23,25,35,42, there has been limited application of CT modalities in clinical evaluation. In particular for rheumatoid arthritis, scoring systems are primarily implemented for MRI43, ultrasound44,45, and/or conventional X-ray46 to generate semi-quantitative and user-dependent measures of disease severity, often in conjunction with clinical metrics47. As CT is considered the gold standard reference for evaluation of bone integrity48,49, further optimization of clinically translatable analytical approaches promises to provide tremendous benefit for reliable and longitudinal quantitative assessment of bone volumes, both to inform measures of disease severity and to evaluate treatment response. Although imaging modalities such as MRI provide a greater breadth of information, including regions of inflammation, bone marrow changes, and soft tissue pathology, novel CT imaging approaches with multi-energy inputs50 provide promise for extending CT utilization beyond the bone architecture. Despite these proposed benefits, we also acknowledge the immense challenges in clinical translation from developed pre-clinical analytical tools, considering the application to low-resolution clinical CT images and implementation in distinct human anatomy. Similar to our recent identification of bone-specific biomarkers in pre-clinical arthritis models23, a detailed clinical effort investigating purely quantitative metrics of bone erosion would be a major advancement in disease monitoring.
While our current work provides a foundation for clinical implementation, given the potential for flexible application to novel structures by targeting joint spaces, a primary limitation is the reliance on a well-documented pre-clinical, research-oriented software in Amira, not intended for clinical diagnosis. However, the underlying algorithms and strategic design can be readily implemented in alternative software environments through the detailed methodology provided. Regardless of the research software used, incorporation into clinical use (rather than investigation) requires translational efforts that meet the regulatory requirements for introduction to clinical practice. For the application of the novel segmentation strategy, it is also important to consider the potential limitations in differential image resolution, where we have previously described that image resolution (i.e., voxel/structure size) is a key determinant of segmentation accuracy using solely image processing algorithms12. In fact, this is possibly associated with the slight reduction in segmentation accuracy of forepaws, where the decreased size of the forepaw structures would inherently produce relatively reduced image quality compared to hindpaws. It is also important to acknowledge the discrepancy in the age range of training (2-6 months) and testing (including 7-8 months) datasets, which may impact application and accuracy with age-related changes, including continued bone growth or further onset of joint pathology. Our findings support maintained accuracy for WT hindpaws in the DL analysis beyond 6 months of age (Figure 2E), suggesting the deprecation in segmentation performance in TNF-Tg counterparts is likely more related to inflammatory-erosive progression independent of age itself. However, further studies with aged and elderly wild-type mice are needed to ensure consistent accuracy independent of the particular age range of the DL training cohort. Lastly, expanding the described methods beyond a single-class bone separation approach to a more robust multi-class analytical tool that includes predicted bone names based on structural architecture or coordinate location in fixed anatomy (i.e., akin to an atlas tree) will certainly provide essential improvements and likely enhance method adoption.
In conclusion, we have designed a novel image processing and DL facilitated micro-CT segmentation strategy to isolate individual bones within complex structures. This innovation demonstrates a remarkable improvement in both automaticity and segmentation accuracy compared to our recently created SA workflow12, which served here as the foundation for the production of numerous gold-standard segmentations to train DL models and optimize the current improvements. Although translation of the segmentation methods in forepaws and paws with inflammatory-erosive arthritis showed deprecated performance, implementation of this DL segmentation approach could reduce the manual efforts necessary to generate completely annotated datasets to allow for pathology- or structure-specific DL training models. Utilization of this DL method in future studies could allow for optimization of bone segmentation across different species and disease models in pre-clinical research to allow for detailed downstream quantitative analysis. We further urge the incorporation of such strategies into clinical research, as it promises eventual benefits for patient care.

High-quality image analysis not only enhances research endeavors but also has the potential to aid clinical radiologists as they seek to detect and quantify pathological changes, a task paramount to patient care. Image analysis is a detailed sequence of procedures, including feature extraction, translating an otherwise generic image into meaningful labels, and deriving quantitative metrics1. Much of this process is driven by prior knowledge, such as thresholding structures based on density or color, and then applying downstream image processing algorithms (i.e., dilation, erosion, smoothing, separation) to achieve the desired segmentation. Once optimized, the segmented images can provide inputs for supervised machine learning2, including deep learning (DL) that encodes and decodes complex features using neural networks,2,3 resulting in improved image segmentation accuracy and throughput.
In fact, the implementation of various 3D convolutional neural networks (CNNs) has provided critical advancements in automated bone segmentation algorithms for skeletal image analysis, where some models outperform human segmentation4. While 3D segmentation CNNs function through distinct architectures (i.e., AlexNet, ResNet, UNet), their outputs are fundamentally the same, an image mask denoting bone positive volume from background. Such deep learning models in musculoskeletal image analysis have been under rapid development, and the field has quickly moved from solving simple 2D fracture detection problems5 to complex multi-joint problems6 with the ability to deal with artifactual noise or anomalous features in datasets. For example, Woo et al noticed that structural anomalies (i.e., bone marrow lesions, bone cysts) on MRI were degrading their segmentation predictions for the articular cartilage in knee joints. Thus, they developed an anomaly-aware segmentation model to first identify unrelated anomalous structures, which dramatically improved the segmentation of bone and cartilage7. He et al trained 14 separate models, each on an ROI of key hand joints, to estimate skeletal age and then integrated their outputs to improve predictions from hand x-rays, rather than using the entire hand structure8. Similarly, including context into models, like regional segmentation features and global anatomical relationships, has been shown to improve predictions. Using multi-region CNNs to provide context on expected segmentation features also improves skeletal maturity assessment in hand X-rays by restraining the classification problem to anatomically appropriate locations9. Additional advances include SVTNet with CNN-based bone segmentation, followed by further processing by vision transformer models to capture global information regarding the spatial relationship between segmented regions of interest to estimate quantitative outcomes such as bone age10.
As demonstrated in these segmentation approaches, arthritis research imaging focuses on articulating surfaces between two or more bones, where methods for bone separation are critical for successful evaluation of differential pathologic processes in complex joints. Advances in image processing algorithms have demonstrated remarkable utility in increasing the analytical throughput of closely adjacent carpal or tarsal bones11,12. However, limited adoption due to inaccuracies requiring user intervention and difficulties in translation to distinct structures highlights the need for the implementation of optimized workflows. These multi-step processes can benefit tremendously from discrete tools for image enhancement (i.e., bone edge detection13). Beyond strictly morphologic operations, other studies have implemented registration-based techniques leveraging the typical reliability and consistency in anatomy for structure identification14,15,16,17. The alternative of manual inputs to generate ground truth labels is expensive and tedious, but can be similarly successful, where utilization may be essential in complicated, closely interlocking bones with less discrete boundaries (i.e., skull18). Similarly, alternative imaging approaches with multi-color/hue variability even within a discrete structure, such as magnetic resonance imaging (MRI19,20,21) or tissue histology22, also exhibit complexity that may benefit from initial manual segmentation to guide automated processes. Together, these methods can offer additional benefits by fueling further automation, where outcomes serve as training datasets to implement DL approaches. The benefits of segmentation automation are numerous, but particularly these methods allow for detailed spatially-relevant quantitative metrics, including regional/bone-specific erosive volume changes23,24,25,26 as well as identification of areas with high susceptibility to damage27.
Here, we build upon established semi-automated (SA) murine hindpaw segmentation methods12 with improvements in image processing algorithms in combination with ground truth bone segmentations28 to train DL models for joint space detection. This novel analytical strategy demonstrated significantly improved accuracy in individual bone segmentation of hindpaws, which reduced manual efforts for corrections of segmentation errors12 to expedite processing of downstream quantitative metrics. We also demonstrate the potential for implementation of these technical advances in novel structures, including forepaws and hindpaws with severe erosive arthritis. As manual segmentation is time-intensive and requires high levels of expertise7, similar strategies to sequentially utilize semi-automated and automated segmentation to produce improved input filters can lower the barrier to developing high-quality CNNs for specific applications. The associated methods developed in Amira software (Supplementary File 1) and the relevant datasets are provided publicly to support adoption and collaboration in further research endeavors28,29.

<table><tbody><tr><td>Computing system</td><td>Details provided in Protocol section&amp;nbsp;</td><td>Details provided in Protocol section&amp;nbsp;</td><td></td></tr><tr><td>Image visualization software</td><td>ThermoFisher Scientific</td><td>v2022.2 or later</td><td>Amira</td></tr><tr><td>Isoflurane</td><td>VetOne</td><td>13985-528-60</td><td>Fluriso, 1-3% for anesthesia</td></tr><tr><td>Mice</td><td>University of Rochester Medical Center</td><td>N/A</td><td>C57BL/6, TNF-transgenic</td></tr><tr><td>Micro-CT</td><td>Scanco Medical</td><td>N/A</td><td>VivaCT 40</td></tr><tr><td>Statistical software</td><td>GraphPad Software, Inc</td><td>v10.2.0 or later</td><td>GraphPad Prism</td></tr><tr><td>Tape</td><td>N/A</td><td>N/A</td><td>To secure animal paws for imaging</td></tr><tr><td>Tubing</td><td>N/A</td><td>N/A</td><td>Derlin plastic and clear acrylic for animal stabilization</td></tr></tbody></table>

automated joint space detection improves bone segmentation accuracy

Automated Joint Space Detection Improves Bone Segmentation Accuracy

Quantitative description of complex anatomical structures remains challenging due to the expertise necessary for manual segmentation, labor, and interobserver variability. To overcome this, automated detection of specific landmarks can be accomplished by digital image analysis techniques, including deep learning (DL) models. To this end, we undertook supervised automated analysis of micro-computed tomography (micro-CT) datasets of murine hindpaws and forepaws. Advancing beyond previously published semi-automated (SA) marker-based watershed algorithms, we added structure enhancement, tensor voting, and output dilation to identify joint spaces. Segmentation was enhanced by utilizing a DL joint space prediction model (3D U-Net architecture, ResNet-18 backbone) using wild-type (WT) hindpaw labels as ground truth. Prediction was extended to hindpaws and forepaws from WT and tumor necrosis factor transgenic (TNF-Tg) mice with inflammatory-erosive arthritis of both sexes across age. Segmentation accuracy improved dramatically using the DL methodology. Accuracy declined with increased disease severity and age in TNF-Tg mice. Subsequent testing in forepaws also displayed progressive reduction in accuracy with increasing arthritic severity. Overall, this supervised automated model outperforms recent SA approaches in healthy joints to enhance the investigation of complex bone anatomy. Although flexible application to novel and disease-modified datasets demonstrates deprecated performance, utilization may nonetheless catalyze structure-specific segmentation model development.

The development of an automated joint space detection workflow enabled high-throughput segmentation of distinct murine hindpaw bones with &#62;98% accuracy in wild-type animals. Flexible application to forepaws and paws with inflammatory-erosive arthritis was achieved, but with deprecated performance that warrants further optimization in future studies using publicly available data.

Quantitative description of complex anatomical structures remains challenging due to the expertise necessary for manual segmentation, labor, and ...

Recent high throughput image processing algorithms have provided numerous gold standard bone segmentations of complex joints with the potential to train deep learning models. The current challenge is the limited ability to utilize prior segmentation strategies in future experiments given structure-specific algorithms and significant manual corrective processes. We address the need for reproducible automated image analysis that enables quantitative assessment of bone health and osteoarthritis progression while minimizing manual labor and viability.To begin, view the micro computed tomography dataset to observe that bone and soft tissue are clearly distinguishable. Inspect regions where adjacent bones are closely packed to identify areas where limited image resolution and minimal gaps make their boundaries difficult to detect. Note the baseline where the supervised watershed-based segmentation method produces an overall accuracy of approximately 80%To improve segmentation accuracy, apply an updated workflow that combines a pre-trained deep learning joint space prediction model with a pre-built image processing and watershed-based bone separation recipe in the Amira software environment.Load the micro computed tomography dataset into Amira. Connect the deep learning prediction module. Select the trained model and press Apply to generate an initial joint space segmentation.Then load the recipe in the image recipe player module. Connect the micro computed tomography dataset and the preliminary joint segmentation, and apply the recipe to produce the final segmentation with fully individualized bones. Finally, evaluate the resulting output by adjusting the color map to labels 256 and inspecting orthogonal slices and volume renderings to confirm successful separation of individual bones throughout the hindpaw.To inspect or edit the image processing recipe, open the image recipe workroom in Amira and load the bone separation recipe. Connect the required input data and adjust parameters in the Properties window to customize the workflow for the current dataset. To review the workflow structure, sequentially visualize the intermediate outputs generated by the recipe, then apply a mathematical morphology black top-hat method implemented through closing, arithmetic subtraction and thresholding, and use this step to emphasize voxels that are darker than their surroundings and correspond to joint regions.To improve specificity, apply the Structure Enhancement filter to enhance dark, thin planar structures corresponding to joint spaces. Further reinforce joint continuity by applying the Membrane Enhancement filter, which incorporates a tensor voting stage. Then, integrate the joint mask generated by a convolutional neural network trained to identify periarticular negative space with the results from the previous image processing filters to generate a final joint mask.To complete bone separation, subtract the final joint mask from the bone mask and treat each remaining connected component as a marker for an individual bone. Apply the watershed algorithm to grow each marker within the original bone mask, reconstructing individual bones, and placing boundaries at optimal positions based on image intensities. To prepare training data for the deep learning model, select 20 wild-type datasets representing 40 hindpaws with balanced sex distribution and ages ranging from 2 to 6 months.Extract six sub-volumes from each dataset with three sub-volumes per hindpaw. Use sub volumes measuring 200 x 200 x 200 voxels to cover ankle regions, digits, and representative background areas. Configure a three-dimensional U-Net architecture with a ResNet-18 backbone.Optimize the model using the Adam algorithm with an initial learning rate of 0.0001 and randomly initialized weights. Train the model using the dice loss function and monitor validation performance using the intersection over union metric. Using deep learning to identify joint spaces in micro-CT datasets enabled clear and consistent separation of individual hindpaw bones.By focusing on the periarticular negative space, the model accurately delineated bone boundaries even in densely packed regions. Compared with the semi-automated workflow, which achieved about 80%accuracy, the deep learning approach delivered a marked improvement for wild-type mice, reaching 98-99%accuracy with minimal manual correction. In tumor necrosis factor or TNF transgenic mice, the deep learning method maintained high performance despite progressive inflammatory erosive disease, with the accuracy of around 98%during the early disease progression stages.Although the model was trained exclusively on hindpaws, it could be applied directly to forepaws without retraining to obtain anatomically coherent segmentations, demonstrating strong generalization to complex structures. It was found that the progression of severe erosive disease and joint destruction was accelerated in TNF transgenic female forepaws. Our protocol improves segmentation accuracy and automation, while still allowing manual corrections and continuous performance enhancements through added expert annotations.Our future research aims to incorporate multiple unique structures and disease models and automated algorithms to continue enhancing flexibility and application.

automated-joint-space-detection-improves-bone-segmentation-accuracy

Research

JoVE Journal

Biology

We present a workflow for segmenting and quantifying trabecular bones for 2D and 3D images based on the bone's outer boundary using an ImageJ plugin. This approach is more efficient and accurate than the current manual hand-contouring approach, and provides layer-by-layer quantifications, which are not available in current commercial software.

Outer-Boundary Assisted Segmentation and Quantification of Trabecular Bones by an Imagej Plugin

This protocol describes the process of applying seven different automated segmentation tools to structural T1-weighted MRI scans to delineate grey matter regions that can be used for the quantification of grey matter volume.

Automated Segmentation of Cortical Grey Matter from T1-Weighted MRI Images

We analyzed joint kinematics from four-dimensional computed tomography data. The sequential 3D-3D registration method semiautomatically provides the kinematics of the moving bone with respect to the subject bone from four-dimensional computed tomography data.

Four-Dimensional CT Analysis Using Sequential 3D-3D Registration

Dual fluoroscopy accurately captures in vivo dynamic motion of human joints, which can be visualized relative to reconstructed anatomy (e.g., arthrokinematics). Herein, a detailed protocol to quantify hip arthrokinematics during weight-bearing activities of daily living is presented, including the integration of dual fluoroscopy with traditional skin marker motion capture.

In Vivo Quantification of Hip Arthrokinematics during Dynamic Weight-bearing Activities using Dual Fluoroscopy

An object segmentation protocol for orbital computed tomography (CT) images is introduced. The methods of labeling the ground truth of orbital structures by using super-resolution, extracting the volume of interest from CT images, and modeling multi-label segmentation using 2D sequential U-Net for orbital CT images are explained for supervised learning.

Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography

The present protocol describes an efficient multi-organ segmentation method called Swin-PSAxialNet, which has achieved excellent accuracy compared to previous segmentation methods. The key steps of this procedure include dataset collection, environment configuration, data preprocessing, model training and comparison, and ablation experiments.

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique

We present a method for analyzing a user-defined region of interest (ROI) in a longitudinal in vivo rat radial defect model. This method enables comparative analysis between different scaffolds previously limited by variations in microcomputed tomography (µCT) scan field of view, specimen orientation, and baseline presence of scaffold.

Longitudinal Micro-Computed Tomography Image Analysis for User-Defined Region of Interest in Critical-Sized Bone Defects

We propose a protocol for assessing bone regeneration at various hierarchical levels. This multimodal approach allows for structural analysis across different length scales, effectively addressing challenges in sample preparation. It also provides a reliable workflow for evaluating the effectiveness of biomaterials in promoting bone regeneration.

Multimodal Approach to Assess Bone Regeneration and Scaffold Performance

The non-human primate is an important translational species for our understanding of the normal processing of the brain. The anatomical organization of the primate brain can provide important insights into normal and pathological conditions in humans.

Dissecting the Non-human Primate Brain in Stereotaxic Space

A suite of spatiotemporal processing methods are presented to analyze human trajectory data, such as that collected using a GPS device, for the purpose of modeling pedestrian space-time activities.

Trajectory Data Analyses for Pedestrian Space-time Activity Study

Learn an automated joint space detection workflow using deep learning and watershed segmentation in Amira to achieve over 98% accuracy in murine hindpaw bone segmentation. This computational method enables high-throughput quantitative assessment of bone health and osteoarthritis progression with minimal manual correction.

Automated Joint Space Detection Improves Bone Segmentation Accuracy

Explore More Videos