Erosion Identification in Metacarpophalangeal Joints in Rheumatoid Arthritis using High-Resolution Peripheral Quantitative Computed Tomography

Summary

Bone erosions are an important pathological feature of rheumatoid arthritis. The purpose of this work is to introduce a training tool to provide users with guidance on identifying pathological cortical breaks on high resolution peripheral quantitative computed tomography images for erosion analysis.

Abstract

Bone erosions are a pathological feature of several forms of inflammatory arthritis including rheumatoid arthritis (RA). The increased presence and size of erosions are associated with poor outcomes, joint function, and disease progression. High-resolution peripheral quantitative computed tomography (HR-pQCT) provides unparalleled in vivo visualization of bone erosions. However, at this resolution, discontinuities in the cortical shell (cortical breaks) that are associated with normal physiological processes and pathology are also visible. The Study grouP for xtrEme Computed Tomography in Rheumatoid Arthritis previously used a consensus process to develop a definition of pathological erosion in HR-pQCT: a cortical break detected in at least two consecutive slices, in at least two perpendicular planes, non-linear in shape, with underlying trabecular bone loss. However, despite the availability of a consensus definition, erosion identification is a demanding task with challenges in inter-rater variability. The purpose of this work is to introduce a training tool to provide users with guidance on identifying pathological cortical breaks on HR-pQCT images for erosion analysis. The protocol presented here uses a custom-built module (Bone Analysis Module (BAM) - Training), implemented as an extension to an open-source image processing software (3D Slicer). Using this module, users can practice identifying erosions and compare their results to erosions annotated by expert rheumatologists.

Introduction

Bone erosions occur when inflammation causes localized bone loss at the cortical bone surface. These erosions extend into the underlying trabecular bone region. They are a pathological feature of several forms of inflammatory arthritis, including rheumatoid arthritis (RA)¹. Erosion presence and size are associated with poor outcomes, patient function, and disease progression^2,3,4,5. While plain radiography remains the clinical standard for erosion assessment, high-resolution peripheral quantitative computed tomography (HR-pQCT) provides 3D images and superior sensitivity and specificity for erosion detection^6,7. For inflammatory arthritis, such as RA, HR-pQCT is commonly performed on the 2^nd and 3^rd metacarpophalangeal joints - the most affected joints of the hand⁸. Because HR-pQCT images have high spatial resolution, physiological interruptions in the cortical surface are also observed in healthy individuals without RA⁹. These cortical interruptions are often associated with vascular channels or nutrient foramen passing through the bone¹⁰. Thus, the challenge is to distinguish cortical interruptions associated with a disease process (i.e., pathological erosions) from non-pathological features.

The consensus definition of a pathological bone erosion was published by the Study grouP for xtrEme Computed Tomography in Rheumatoid Arthritis (SPECTRA) as the presence of a definite interruption in the cortical layer of the bone that extends over at least two consecutive slices and is detectable in two or more perpendicular planes¹¹. Further, the interruption must be non-linear in shape and accompanied by a loss in the trabecular region. Visual examples of cortical interruptions that do and do not meet the criteria of erosions are shown in Klose-Jensen et al.¹².

However, not all cortical interruptions that meet the above criteria are classified as erosions. Interruptions are sometimes caused by physiological processes such as vascular channels (Figure 1). These can be identified and differentiated from erosions due to their predictable anatomical locations, parallel and straight margins and sub-millimetric size¹³. Cysts are another form of cortical interruption that is not considered to be an erosion. They often have a rounded trabecular structure with a clear cystic wall ¹³. In contrast to the sharp edges and open trabecular structure displayed by erosions. However, it is possible for erosions to form within cystic sites, making it ambiguous to delineate the volume of bone loss caused by the erosions and not the cysts. While resolving this ambiguity with further criteria is not the purpose of this study, there is a need to provide comprehensive examples of pathological erosion and physiological cortical interruptions.

Figure 1: Example of cortical interruptions that were not caused solely by erosions. (A) A drawing illustrating the common location of vascular channels at the base of the metacarpal head. Examples of vascular channels in (B) coronal, (C) sagittal, (D) and (E) axial planes. (F) Example of a cortical interruption caused by a cyst. (G) Example of a void volume within trabecular region of the bone involving both cysts and erosions. Please click here to view a larger version of this figure.

Despite the challenges in erosion identification, there are currently no training tools in place to provide less experienced users with guidance on interpreting HR-pQCT images for erosion analysis. Recently, an open-source module for erosion analysis called bone analysis module (BAM) - Erosion Volume was developed, implemented as an extension to an open-source image processing software to enable erosion visualization and volumetric analyses¹⁴. The protocol presented here describes the use of a training module added to BAM (BAM - Training), which compares a users' erosion identification attempts by comparing the erosion identification with erosions annotated by expert rheumatologists. This training tool provides users with feedback on erosion identification in order to guide improvements in erosion analysis. Software installation instructions are provided in step 1. For new data acquisition, see steps 3 - 5.3. For training module use only, see step 2.

Protocol

All methods in this protocol follow the guidelines set by the Conjoint Health Research Ethics Board at the University of Calgary (REB19-0387).

1. Install 3D Slicer ¹⁵ and bone analysis modules

1. Download the installation file for a stable release of 3D Slicer that is relevant to the operating system used from https://download.slicer.org/.
2. Run the downloaded installation file and follow the instructions provided on the wizard.
3. Once the installation is complete, proceed with installing the bone analysis modules.
   1. Download the bone analysis modules from https://doi.org/10.5281/zenodo.7943007 as a compressed zip file and extract the compressed folder. Please note the directory where the extracted folder is located.
   2. Launch 3D Slicer. Load the modules onto 3D Slicer by clicking on Edit on the top left corner of the 3D Slicer window. Click on Edit > Application Settings to open a new window.
   3. Click on Modules which is a tab located on the left on the recently opened Settings window. Add the paths to the bone analysis modules under Additional module paths: (Figure 2).
      1. To do this, drag and drop the following list of folders onto the box located under Additional module paths:. These folders are located inside of the downloaded folder in step 1.3.1: AutoMask, CorticalBreakDetection, ErosionComparison, ErosionVolume, FileConverter, ImageRegistration, Training. Settings window should resemble Figure 2.
      2. Press OK on the bottom right of the Settings window. A restart is required to confirm the installation of the modules. Do this by closing 3D Slicer and relaunching it.
         ​NOTE: Loading the modules is only done once per installation of 3D Slicer. Upon further updates to the bone analysis GitHub repository, a simple terminal (or command line) git pull command at the directory containing the earlier BAM download will automatically update all the modules. Again, alternatively, the repository can be downloaded, and old modules exchanged with the new ones manually.

Figure 2: Example of settings window after adding bone analysis modules to an installation of 3D Slicer. The image shows a screenshot of the settings window with the modules highlighted in the red box. Please click here to view a larger version of this figure.

2. Training module

1. Launch the BAM-Training module.
   1. Click on the Drop-down Menu located on 3D Slicer's toolbar. Find bone analysis modules and hover the mouse above it (Figure 3). Click on Training.
2. Load files in the module.
   1. Upon launching the BAM training module, all the required files (greyscale images, masks, reference erosion segmentations) will be loaded automatically by clicking Proceed, assuming the BAM github repository was downloaded as mentioned above.
3. Choose a scan type for the files.
   1. Choose one by hovering over the drop-down menu labeled Input Volume: Select a Volume, this represents the master grey scale image.
   2. Choose the mask (i.e., the file that identifies the volume within the periosteal surface) in the drop-down menu labeled Input Mask: Select a Volume. Make sure this mask corresponds to the input volume above by verifying the measurement ID and the MCP joint is the same in both selections.
   3. If this is the first time running the training module on this image with this launch of 3D Slicer, then create a new output segmentation at the drop-down menu labeled Output Erosions: Select a Segmentation. To do this, click the drop-down menu and select Create New Segmentation. This will create a new output segmentation node labeled after the input mask label + _ER. To give the output a different label, select Create New Segmentation As… instead and input the desired label.
      NOTE: In 3D Slicer and this document, mask and segmentation identifies the volume inside the bone's periosteal surface. The mask is visualized as a binary image, while the segmentation refers to the visualization of the binary image overlaid with the grey scale image. These distinctions are made by 3D Slicer. An example is illustrated in Figure 4.
4. Place seed points as described below.
   1. To begin, create a new seed point list to add seed points. To do this, click on the drop-down menu labeled Seed Points: None and create a new list by selecting Create new Point List. Again, the default labeling standard is input image label + _SEEDS. To provide your own label, select Create New Point List….
   2. Scroll through the slices and identify erosion sites by placing a seed point in the region of interest. Press the Red Dot-Blue Drop button shown in Figure 5A to add a new seed point.
      1. Place the seed point as deep (inwards into trabecular bone) into the erosion volume as possible. Make sure the seed point is placed on the darkest region of the volume.
      2. To change the seed point size, modify the percentage size at the text box labeled Seed Point Size:. Other fields in the seed points table, such as bone and cortical interruption, are for the user's records and do not affect the erosion computation algorithm.
5. Obtain feedback as described below.
   1. Once the seed points have been placed. Press the button labeled Get Erosions, highlighted in Figure 5B, to run the erosion measurement algorithm on the given inputs. Upon completion of the erosion measurements, the module will provide feedback on the seed point placement. Each seed point's location is compared with the location of reference erosions to match seed points to the erosion it is trying to measure.
   2. Obtain reference erosions by calculating the erosion volume using seed points placed by rheumatologists with training, extensive publication records and more than 10 years of experience using HR-pQCT imaging and erosion analysis (SF and CF).

Figure 3: 3D Slicer's dropdown menu. The dropdown menu to find the bone analysis modules and select the training module. Please click here to view a larger version of this figure.

Figure 4: Identification of the volume inside the bone's periosteal surface. (A) Example of a mask. The mask is visualized as a binary image. (B) Example of a segmentation. The segmentation refers to the visualization of the binary image overlaid with the grey scale image. These distinctions are made by 3D Slicer. Please click here to view a larger version of this figure.

Figure 5: Example screen shot of training module within 3D Slicer. (A) Click to add new seed points. (B) Click to compute erosion volumes. (C) Click to import images. (D) Click to reveal seed points placed by experts. Please click here to view a larger version of this figure.

3. Image acquisition and export to use in erosion analysis tool

1. Acquire HR-pQCT images using a first or second-generation HR-pQCT scanner. A commercial scanner was used to acquire the images for this study (see Table of Materials).
   NOTE: The images used in this study are of the 2^nd and 3^rd metacarpophalangeal joints and were acquired using the protocol described in Barnabe et al.⁸, however images of any joints with erosions are compatible with BAM.
2. Inspect images for motion artifact^16,17. Do not use images with motion scores > 3 for erosion analysis.
3. Export AIM (proprietary image format) or digital imaging and communications in medicine (DICOM) files of each joint to a local disk using file transfer protocol (FTP). Files of individual joints may be generated using the joint space width (JSW) analysis pipeline provided by the manufacturer¹⁸. Rename files from default number format for easier use.

4. File conversion and bone mask generation

NOTE: Depending on the image format, follow step 4.1 for AIM (HR-pQCT proprietary image format), MHA (ITK MetaImage format), nii (NIfTI - Neuroimaging Informatics Technology Initiative), NRRD (Nearly Raw Raster Data) images or step 4.2 for DICOM images.

1. To import images with any of the following image file formats: MHA, nii, AIM or NRRD follow the steps described below.
   1. Click on the button labeled DATA located at the top left corner of the 3D Slicer window (Figure 5C).
   2. To add image files, click on Choose File(s) to Add, locate and add the images.
   3. To add an entire directory of images, click on Choose Directory to Add, locate and add the directory. This will load all images in that directory.
   4. If an image mask needs to be imported as a 3D Slicer segmentation for any reason, first convert to an NRRD or nii file. This conversion can be done automatically, see step 4.4.1 for details.
2. Import DICOM file(s) in 3D Slicer as described below.
   1. Click on the button labeled DCM located at the top left corner of the 3D Slicer window.
   2. Click on Import DICOM Files, locate and add the directory containing the DICOM files (extension is .dcm).
   3. Click on the button labeled Load located at the right side of the window.
3. Get image mask using Step 2 - Automatic Mask in the BAM - Automatic Mask module.
   1. Click on the drop-down menu located on 3D Slicer's toolbar. Find Bone Analysis Modules and hover the mouse cursor above it. Click on Automatic Mask.
   2. Under the tab Step 2 - Automatic Mask, select an input volume using the drop-down menu labeled Input Volume:. This is the input scan.
   3. Create a new output at the drop-down menu labeled output segmentation: and select Create New LabelMapVolume. This will create a new output node labeled after the input mask label + _MASK. To give the output a different label, instead select Create New LabelMapVolume as… and input the desired label.
   4. Input the number of bones to be masked at the text box with that label. Select Ormir in the drop-down menu labeled Algorithm for the optimal segmentation for this analysis¹⁹.
      NOTE: Other options for generating these masks are available and may be added in the future.
   5. Click on Get Mask. This will run the algorithm (~2-3 min) and output the result at the same directory of the input image. It will also save a separate mask for each bone if the image had multiple bones.
4. Perform manual correction of bone mask using Step 3 - Manual Correction in the BAM module. Often the mask generated is not accurate. Perform manual correction to add, delete, or edit specific components of the segmentations.
   1. To edit a mask that was generated through other means or generated in a previous run of 3D Slicer, use this module to load these masks to 3D Slicer from a file. The file extension can be any of the following, MHA, nii, NRRD, AIM.
      1. Copy the images to the LOAD_MASKS directory located in the BAM folder downloaded in step 1.3.
      2. Go back to 3D Slicer and press the button labeled Load in the manual correction stage.
   2. Select the segmentation to be corrected in the drop-down menu labeled Mask To Be Corrected:".
   3. Select the original gray-scale image belonging to this erosion segmentation in the drop-down menu labeled Master Volume:. Press Initialize.
   4. Each segmentation must have its own entry in the table below. Select the segmentation to be corrected based on the color of the segmentation.
   5. To add to a segmentation, click on the second button on the first row. This uses the paint function. Make additions to the volumes by drawing on the images (hold left mouse button down and move the mouse).
   6. To remove part of the segmentation, click on the button below the table labeled Erase Between Slices. This is the erase function and works like the paint function but erases instead.
   7. Draw additions as needed on roughly every 10-25 slices, but make sure to include the first slice and last slice where an addition was needed.
   8. If the paint function was used, the changes can be interpolated by clicking on the first button of the fifth row labeled Fill in Between Slices function. Click on the buttons Initialize > Apply.
   9. If the erase function was used, simply click on the button below the table labeled Apply Erase. Do not use both paint and erase functions at the same time. Apply one function first then apply the other.
   10. After the edits are completed, press Apply.

5. Identification of erosions

1. Use Step 4 - Erosions in the BAM - Erosion Volume module for identification of erosion. The erosion volume module is the tool responsible for identifying and measuring erosions within a scan.
   NOTE: This module is the focus of the training tool detailed above and has a nearly identical workflow. The differences are that the computed erosions are not compared against expertly annotated ones, exporting erosion statistics and manual correction of the volumes after their identification is possible here.
2. Place seed points and get erosion as done in step 2.4. In the end no feedback will be provided.
3. Manual Correction
   1. If the size and shape of the automatically detected erosion volumes are unsatisfactory, edit these under the tabs labeled Step 5 - Manual Correction and Export Segmentation. Follow steps outlined in step 4.4. However, there is no option to load external erosion volumes. Do not press Apply after finishing the corrections as the changes have already been saved.

6. Erosion Statistics

1. Export the computed data to a spreadsheet file (CSV format) using the tab labeled Step 6 - Statistics.
2. Provide the erosion volume computed in Step 4 and optionally corrected in Step 5 under the drop-down menu labeled Input Erosions.
3. Provide the greyscale image under the drop-down menu labeled Master Volume. Provide the voxel width, in mm, of the image in the text box. Press Get Statistics.
4. The spreadsheet file has been generated in the directory called EROSIONS_OUTPUT_DATA located in the BAM folder downloaded in step 1.3. See Table 1 for an example of the output table.

Representative Results

Using the training tool, users can practice identifying erosion sites while receiving feedback on their results. This feedback loop can improve the user's ability to identify erosions and potentially use the BAM modules to identify erosions on their own images. Feedback after seed point placement is based on the following criteria. 1) If the number of seed points placed does not match the number of reference erosions, the user is then prompted to delete or add the appropriate number of seed points. 2) If the seed point location cannot be matched with a reference erosion, feedback stating that no erosion exists at that seed point's location is displayed. 3) If a seed point is matched with a reference pathological/physiological cortical interruption such as a cyst or a vascular channel, the user is informed about the type of cortical interruption they attempted to identify as an erosion and is asked to remove the seed point. 4) If the location of the seed point overlaps a reference erosion, the algorithm may still not detect the erosion. This may occur when the seed point has not been centered in the erosion. In these cases, the user is prompted to adjust the position of the seed point. 5) If a seed point is placed too far from any erosion, the user is informed of their incorrect placement and encouraged to try again. 6) When a seed point location matches the reference erosion, a prompt is displayed informing the user of their successful attempt to identify the erosion at that specific seed point.

The following section illustrates examples of how the module works based on different inputs. Correct and incorrect inputs will be demonstrated in the following examples. Figure 6A shows the seed point location which is located within the erosion. Only one erosion exists within this image, therefore computing the erosions with the seed point will yield the expected results. Figure 6B shows the prompt displayed to users when their attempt to identify the erosions matches the expertly annotated image. The module also displays the results as segmentations on the grey-scale image (Figure 6C). If the user placed a seed point at a location without an erosion, such as Figure 7A, the module would display an error prompt (Figure 7B) stating that no erosion exists at this location and suggests that the user relocate/remove the seed points.

Figure 6: Example of correct erosion identification. (A) Example of a user correctly placing a seed point within the erosion site. (B) Example of feedback prompt when all erosions were identified correctly. (C) Example of displayed erosion segmentation when an erosion is computed correctly. Please click here to view a larger version of this figure.

Figure 7: Example of incorrect erosion identification. (A) Example of a seed point placed where no erosion exists. (B) Example of an error prompt when a seed point is placed at a location that has no erosion. Please click here to view a larger version of this figure.

The locations of all cysts and vascular channels on the training images provided have been identified by an expert. Therefore, it is possible to detect when a user attempts to incorrectly identify a cyst or a vascular channel. Figure 8A illustrates an attempt to identify a cyst by placing a seed point on it. Figure 8B is the subsequent displayed error prompt.

Figure 8: Example of cyst identification. (A) Example of a seed point placed on a cyst. (B) Example of an error prompt when a seed point is placed on a cyst. Please click here to view a larger version of this figure.

The module will also inform the user if they have the right amount of seed points. If the user placed an incorrect number of seed points, the module would inform the user of the exact amount of seed points that are missing or extra to identify all erosions on the image. The module also gives feedback for each placed seed point. Therefore, the user knows what actions to take for each individual seed point. Figure 9 demonstrates an example where a user only placed one seed point when two were expected.

Figure 9: Example of erosions computed while missing one seed point. The example demonstrates an example where the user only placed one seed point when two were expected. Please click here to view a larger version of this figure.

If a user has trouble finding any or all the erosions, they have an option to reveal the expertly annotated locations by pressing a button labeled Reveal Correct Seed Points (Figure 5D). When pressed, this button will load the correct seed points on to the current 3D Slicer window.

In summary, this shows that the software module can assess the correctness of the user's attempt to identify erosions in the select images by comparing the computed erosion with expertly annotated erosions. In addition, the module provides feedback based on each user-placed seed point to guide them towards the expected seed point location and input parameters.

Scan ID	Cortical Interruption	Bone	Label	Centroid Location	Volume (mm3)	Surface Area (mm2)	Roundness	Number of voxels (voxels)
3_Training.nii	Erosion	Metacarpal	SEEDS_28-1	210, 108, 242	3.321668853	14.46818378	0.74411491	14853
3_Training.nii	Erosion	Metacarpal	SEEDS_28-3	179, 100, 241	1.100739562	7.121231239	0.7239659	4922

Table 1: Example of a generated output file (csv format) describing computed erosions and their statistics.

Discussion

This training tool provides an opportunity to learn to identify erosions using the bone analysis module. Further use of this erosion analysis tool beyond training requires access to good quality images, with little or no motion artifact. The HR-pQCT erosion definition based on the literature describes anatomical features associated with pathological erosions that can be reported with reasonable reproducibility^11,20. However, this definition does not account for common anatomical locations of vascular channels, potentially resulting in their misclassification as bone erosions¹⁰.

The critical steps in this protocol are the generation of the bone mask, the placement of the seed points, and the generation of the erosion volume. While automated methods to generate the masks and erosion volume are implemented, the masks often require manual correction to ensure satisfactory results. A comprehensive description of the tools available to perform the manual corrections is provided. The placement of seed points is guided by the training examples provided by the BAM-Training module.

Based on the data utilized to date, this protocol provides suggestions for troubleshooting when the erosion analysis module does not produce the expected results. In future work, access to additional training data will be provided. A previous study showed that the erosion volumes assessed with this method are comparable with existing methods^14,21,22. Provision of training data will allow comparison to newer erosion analysis tools as they are developed²³.

The training tool introduced here primarily aids with erosion identification; however, the method is currently limited by the lack of a consensus on defining the extent of an erosion in the trabecular bone. Nonetheless, the BAM modules are open source, thus, as future definitions of erosion extent change, other researchers have access to modify the modules to meet their needs.

As the use of HR-pQCT in rheumatological research expands, the training tool provides inexperienced users with guidance on identifying pathological cortical interruptions on HR-pQCT images for erosion analysis. This tool will be applicable to researchers regardless of the method selected for erosion analysis. While completely automated erosion identification is desirable to improve reproducibility and speed of analysis, large reference/benchmark datasets with accurate annotations are required to train machine learning models. As an open-source tool, this module provides an opportunity to collectively develop large, annotated datasets for future use in machine learning. The use of this training tool will enable more researchers to include erosion analysis in their HR-pQCT research.

Materials

Name	Company	Catalog Number	Comments
3DSlicer	Open Source	N/A	Download at https://www.slicer.org/
BAM Erosion Analysis Modules	Open Source	N/A	Version used in manuscript: download at https://doi.org/10.5281/zenodo.7943007
XtremeCTII	Scanco Medical	N/A