Three-Dimensional Reconstruction for the Whole Lung with Early Multiple Pulmonary Nodules

Summary

This study introduces a three-dimensional (3D) reconstruction method for the entire lung in patients with early multiple pulmonary nodules. It offers a comprehensive visualization of nodule distribution and their interplay with lung tissue, simplifying the assessment of diagnosis and prognosis for these patients.

Abstract

For patients with early multiple pulmonary nodules, it is essential, from a diagnostic perspective, to determine the spatial distribution, size, location, and relationship with surrounding lung tissue of these nodules throughout the entire lung. This is crucial for identifying the primary lesion and developing more scientifically grounded treatment plans for doctors. However, pattern recognition methods based on machine vision are susceptible to false positives and false negatives and, therefore, cannot fully meet clinical demands in this regard. Visualization methods based on maximum intensity projection (MIP) can better illustrate local and individual pulmonary nodules but lack a macroscopic and holistic description of the distribution and spatial features of multiple pulmonary nodules.

Therefore, this study proposes a whole-lung 3D reconstruction method. It extracts the 3D contour of the lung using medical image processing technology against the background of the entire lung and performs 3D reconstruction of the lung, pulmonary artery, and multiple pulmonary nodules in 3D space. This method can comprehensively depict the spatial distribution and radiological features of multiple nodules throughout the entire lung, providing a simple and convenient means of evaluating the diagnosis and prognosis of multiple pulmonary nodules.

Introduction

Early multiple pulmonary nodules, which are small, round growths on the lung, can be benign or malignant^1,2,3. Although solitary pulmonary nodules are easier to diagnose and treat, patients with early multiple pulmonary nodules face significant diagnostic and treatment challenges. To develop effective treatment plans, it is essential to accurately identify the spatial distribution, size, location, and relationship with surrounding lung tissue of these nodules throughout the whole lung^4,5. Traditional diagnostic methods have limitations in accurately identifying early multiple pulmonary nodules.

Recent advancements in medical image processing technology and machine learning algorithms have the potential to improve the accuracy and efficiency of early pulmonary nodule detection and diagnosis. Various approaches have been proposed, such as pattern recognition methods based on machine vision and visualization methods based on maximum intensity projection (MIP)^6,7,8,9,10. However, these methods suffer from limitations such as false positives, false negatives^{11,12,13,14,15}, and lack of macroscopic and holistic descriptions of the distribution and spatial features of early multiple pulmonary nodules.

To address these limitations, this study proposes a whole-lung 3D reconstruction method that utilizes medical image processing technology to extract the 3D contour of the lung against the background of the whole chest scan. The method then performs 3D reconstruction of the lung, pulmonary artery, and early multiple pulmonary nodules in 3D space. This approach allows for a more comprehensive and accurate representation of the spatial distribution and radiological features of early multiple nodules throughout the whole lung.

The proposed method involves several key steps. Firstly, the medical images are imported into the 3D image processing software, and the lung region is extracted using a threshold-based segmentation technique. Subsequently, the extracted lung region is separated from the surrounding chest wall and the bony structures of the thoracic vertebrae. The early multiple pulmonary nodules and their relationship with surrounding blood vessels are then reconstructed in 3D space using maximum intensity projection (MIP) algorithms. Finally, the reconstructed 3D model of the lung, pulmonary artery, and nodules is displayed for further analysis.

This method has several advantages over existing methods. Unlike traditional methods that rely on 2D images, this method utilizes 3D volume to provide a more accurate and comprehensive representation of early multiple pulmonary nodules. The method also overcomes the limitations of false positives and false negatives associated with pattern recognition methods and MIP visualization methods. Furthermore, this method provides a macroscopic and holistic description of the distribution and spatial features of early multiple pulmonary nodules, which is essential for developing effective treatment plans.

The proposed method has several potential applications in the diagnosis and treatment of early multiple pulmonary nodules. The accurate identification of the spatial distribution and radiological features of early multiple nodules can aid in the early diagnosis and treatment of lung cancer. Furthermore, the method can be used to monitor the progression of the disease and evaluate the effectiveness of treatment plans.

Pattern recognition methods^6,7,8 based on machine vision have shown promise in identifying pulmonary nodules, but suffer from limitations such as false positives and false negatives. MIP visualization methods, on the other hand, provide a more accurate representation of individual nodules, but lack a macroscopic and holistic description of the distribution and spatial features of early multiple nodules. The proposed whole-lung 3D reconstruction method overcomes these limitations and provides a more accurate and comprehensive representation of early multiple pulmonary nodules.

Isovoxel transformation^16,17refers to the process of converting 3D images with different voxel sizes into 3D images with uniform voxel sizes. In the field of medical image processing, 3D volumes are often composed of voxels with varying sizes, which can lead to computational and visualization issues. The purpose of isovoxel transformation is to address these issues by resampling and interpolating the voxels in the original 3D volume, resulting in a new 3D image with consistent voxel sizes. This technique finds applications in various medical contexts, including image registration, segmentation, and visualization. Thus, this study proposed a whole-lung 3D reconstruction method that utilizes medical image processing technology to extract the 3D contour of the lung against the background of the whole chest scan. The method provides a more accurate and comprehensive representation of the spatial distribution and radiological features of early multiple nodules throughout the whole lung. This study contributes to the development of more accurate and effective diagnostic and treatment strategies for patients with early multiple pulmonary nodules.

Protocol

For the present study, ethical clearance was obtained from The Ethics Committee of Dongzhimen Hospital, affiliated with Beijing University of Chinese Medicine (DZMEC-KY-2019.90). In this specific case, a methodical description of the research approach is provided, outlining a case involving a 65-year-old female patient with multiple pulmonary nodules. This patient provided informed consent for her diagnosis through digital modeling and authorized the use of her data for scientific research purposes. The model reconstruction function is derived from a commercially available software tool (see Table of Materials).

1. Data preparation and isovoxel transformation

1. DICOM (Digital Imaging and Communications in Medicine) data preparation and data properties
   NOTE: The variation in parameters remains relatively unaffected by the research methodology.
   1. Copy the patient's DICOM data to a defined working directory.
   2. Using the file browser, examine each file directory to identify the image sequence with the highest number of scanning layers for analysis.
   3. Employ the Dicominfo function within MATLAB by providing DICOM files as input parameters. This will enable you to extract essential parameters, such as slice thickness and pixel spacing, directly within the MATLAB environment.
      NOTE: These parameters hold significant importance in configuring the display rate for the 3D volume. In the case of the dataset utilized in this study, the slice thickness measured 1 mm, the pixel spacing equated to 0.7188 mm, and a total of 387 layers were scanned.
2. Correct sorting of scanned data
   NOTE: The sequence of every image should be sorted for volume construction.
   1. Retrieve the location data for each image using the Dicominfo function. Access the location information by referencing info.SliceLocation within the MATLAB workspace.
   2. Save the location data into a variable using the SliceLocation function and generate a plot for it (Figure 1).
   3. Enhance the plot by adding a data point to it using the Data Tips button situated in the upper-right corner of the GUI. This data point should mark the maximum location of the normal sequence, which corresponds to the topmost location in the patient's imaging (Figure 1).
   4. Organize all the images by sorting them and then extract the images ranging from the first location to the maximum location. Achieve this by invoking the VolumeResort function.
   5. Safeguard the volume data, which consists of 512 pixels by 512 pixels by 340 layers, from the valid images along with their sorted index. This information will be valuable for future reference, particularly in the context of identifying important nodules.
3. Isovoxel transformation
   ​NOTE: 3D Isovoxel transformation allows subsequent processing to maintain the same display scale in all dimensions.
   1. Examine the three-dimensional scale of a 3D volume, which is 512 pixels x 512 pixels x 340 layers, using the size function in Matlab.
   2. To view the 3D-volume (Figure 2) using the Slice_View command function, record the sequence scan range containing the lungs from 60 to 340. Then, simply use the command V1=V0(:,:,60:340) to obtain a 3D-volume containing all the data of the entire lung. The size of V1 is 512 pixels x 512 pixels x 281 layers.
   3. Utilize the MATLAB command function dicominfo to obtain the slice thickness of the image sequence, which is 1 mm, and the pixel spacing is 0.7188. Calculate the number of z-axes for the isovoxel transformation with the command: round (281 x 1/0.7188). The number of layers for the isovoxel transformation should be 391.
   4. Use the Matlab command function imresize3 to perform isovoxel transformation on V1. Execute the script using the command V2=imresize3(V1, [512, 512, 391]). Then use the 3D_Slice_View function to view the isovoxel transformed 3D-Volume (Figure 3).

2. Removal of noise interference caused by Computed Tomography (CT) equipment

NOTE: In Figure 2, the high-intensity signal representing the CT equipment's patient couch is visible, which can interfere with image segmentation. To eliminate this interference, a spatial filter design is required.

1. Utilize the Data Tips button in Figure 2 to add continuous data points within the interactive interface. This will allow one to create a line connecting these points, effectively excluding the patient couch. Next, right-click on the Data Tips and select Export Cursor Data to Workspace to export the reference boundary for spatial filtering to the MATLAB workspace (Figure 3). The boundary scatter matrix in this case is named 'CI'.
2. Invoke the Noise_Clean function to apply spatial filtering to V2, using the input parameter 'CI' from the workspace. This operation will yield a 3D volume that removes the interference signal from the CT equipment. Finally, use the Slice_View command function to visualize the resulting volume, as demonstrated in Figure 4.

3. Extraction of lung contour

1. Begin by selecting a slice to serve as a template within the GUI displayed in Figure 4. For instance, choose the 232nd image for the image segmentation design and assign it to a variable 'I' using the command I=V2(:,:,232). Then, open the MATLAB Image Segmenter GUI by executing the command imageSegmenter(I), as depicted in Figure 5.
2. Figure 5 showcases an array of image segmentation tools. To start, select the Auto Cluster tool from the toolbar at the top and execute the command by clicking the left mouse button. The image will automatically be divided into two classes. Given the denoising process performed in step 2.2, image segmentation at this stage becomes relatively straightforward.
3. Next, click on the Show Binary button in the upper right corner to display the image in black and white binary. At this point, the lung region will appear black. To make the lung region white, select the Invert Mask button from the top toolbar and execute the command by clicking the left mouse button.
4. To eliminate the white color outside the lung area, select the Clear Borders button on the top toolbar and execute it by clicking with the left mouse button. After this step, only the white-colored lung area will remain. However, any black shadows remaining within the lung area at this point need to be filled. To achieve this, select the Fill Holes button in the toolbar, and the result after clicking the button is shown in Figure 6.
5. All the steps involved in lung image segmentation are presented in the GUI of Figure 6 at the lower left corner. By clicking the Export button in the upper right corner, save these automated steps as a function for batch-processing lung region segmentation. In the pop-up Script Editor, click the Save button to save the function in the current working directory.

4. 3D reconstruction for the whole lung with multiple pulmonary nodules

NOTE: Taking the dot product of the lung segmentation image of each image with the original image is equivalent to performing 3D spatial filtering on the volume, effectively filtering out interference signals outside the lungs and obtaining the 3D structure of the lungs.

1. Initiate the 3Dlung_Volume function within the MATLAB workspace.
   NOTE: This function conducts image segmentation on each image using the output from step 3.5. It then executes a dot product operation between the binary lung mask and the original image to generate a new 3D-volume exclusively containing lung tissue. In the GUI (Figure 7) that appears after the function completes, one can visualize and perform Maximum Intensity Projection (MIP) operations on the entire 3D lung volume.
2. Within the GUI, find the first drop-down menu in the top right corner. Select MIP Projection and then choose the jet colormap from the Built-in Colormaps options below. Next, in the drop-down menu located in the top right corner of the fourth view (3D Volume View), select Maximize. This action will yield a whole lung 3D volume (Figure 8) that can be observed from any angle, moved, and manipulated as needed.
   NOTE: In the human-computer interaction section illustrated in Figure 8, one can adjust the viewing angle freely by holding down the left mouse button and moving it. Scrolling the middle mouse button allows one to zoom in or out.
3. For advanced contrast and color enhancement operations, utilize the control panel on the right side of the GUI.

5. Focus on the examination of dominant pulmonary nodules

NOTE: In 3D space (Figure 8), the dominant lesion area among multiple pulmonary nodules becomes distinctly visible. The number, size, and concentration of these nodules are critical features of the dominant lesion, offering valuable insights into disease assessment.

1. Once more, invoke the Slice_View function, but this time input the entire lung's 3D volume obtained in step 4.2. Within the resulting GUI (Figure 9), use the bottom scroll bar to navigate to the region where the dominant lung nodules are situated, spanning scans 48 to 70.
2. Proceed by calling the 3Dlung_Horizon function to conduct 3D reconstruction of the Region of Interest (ROI) encompassing sections 48 to 70 from the whole lung's 3D volume. This action will generate a GUI interface tailored for visualizing pulmonary nodules, as illustrated in Figure 10. Within this GUI, one can explore the lesion's detailed features from various angles.

Representative Results

In the data preprocessing stage, DICOM data sorting should be the first step (Figure 1) to ensure the correct scan sequence for each layer during 3D reconstruction. Next, isotropic transformation is performed to ensure the correct aspect ratio of the 3D volume (Figure 2). Afterward, spatial filtering is applied to the original 3D volume (Figure 3) to eliminate interference signals from the patient couch of the CT equipment (Figure 4). To obtain the 3D contour of the entire lung, image segmentation is performed on each scan (Figure 5) to create a binary lung image (Figure 6). Based on the 3D contour of the lung, the entire lung 3D volume is reconstructed (Figure 7) and visualized in 3D (Figure 8). For the dominant lesion area (Figure 9), a separate 3D visualization (Figure 10) can be performed to carefully identify the detailed features of the lesion.

Isovoxel transformation ensures that the same scale is maintained in all dimensions during subsequent processing. Figure 2 displays the slice view after isovoxel transformation. In this graphical user interface (GUI), one can view the complete raw 3D volume data.

Figure 3 and Figure 4 demonstrate the spatial filtering process used to remove bed signal interference from the CT equipment. Without this, images with noisy signals cannot complete the segmentation of lung structures in subsequent steps.

Figure 5 and Figure 6 illustrate the lung contour extraction function, which can automatically extract lung contours, providing the basic conditions for the subsequent 3D reconstruction of lung structures.

Figure 7 and Figure 8 show the 3D reconstruction of the entire lung, revealing the spatial distribution of lung tissues and multiple lung nodules. By eliminating signal interference from tissues outside the lungs, the spatial location, size, and concentration of multiple pulmonary nodules can be accurately depicted.

Figure 9 and Figure 10 display the 3D visualization of the dominant lung nodules of interest. Due to the exclusion of signal interference from outside the lungs, the contrast of the images is improved. The ability to observe the 3D structure from any angle allows physicians to make more accurate judgments about the lesion features of the dominant pulmonary nodules.

Figure 1: Location plot of images. The plot displays the location of images based on their file name sequence. Please click here to view a larger version of this figure.

Figure 2: GUI for 3D-volume slice view. Graphical User Interface (GUI) for viewing slices of the 3D volume after Isovoxel transformation. Please click here to view a larger version of this figure.

Figure 3: Reference boundary scatter matrix. The matrix representing reference boundary scatter for spatial filtering. Please click here to view a larger version of this figure.

Figure 4: 3D volume slice view after spatial filtering. View of slices from the 3D volume after applying spatial filtering. Please click here to view a larger version of this figure.

Figure 5: Image segmenter GUI. Graphical User Interface (GUI) of the Image Segmenter tool. Please click here to view a larger version of this figure.

Figure 6: Result of lung area shadow filling. The resultant image after filling black shadows in the lung area using the "Fill Holes" button. Please click here to view a larger version of this figure.

Figure 7: 3D lung reconstruction with multiple pulmonary nodules. 3D reconstruction of the entire lung showing early multiple pulmonary nodules. Please click here to view a larger version of this figure.

Figure 8: Interactive GUI for 3D lung volume viewing. Interactive Graphical User Interface (GUI) for viewing and manipulating the entire 3D lung volume. Please click here to view a larger version of this figure.

Figure 9: Slice view for navigating dominant pulmonary nodule area. Slice view for navigating the region containing dominant pulmonary nodules within the entire 3D lung volume. Please click here to view a larger version of this figure.

Figure 10: 3D visualization of dominant pulmonary nodule. Three-dimensional visualization of the dominant pulmonary nodule within the lung volume. Please click here to view a larger version of this figure.

Discussion

This research introduces a unique approach for creating a complete three-dimensional (3D) reconstruction of the entire lung, employing advanced medical image processing techniques to delineate the lung's 3D shape amidst the context of a full chest scan. This technique offers a more precise and thorough depiction of the spatial arrangement and radiological characteristics of early multiple nodules across the entire lung. This study makes a valuable contribution to enhancing the accuracy and efficacy of diagnostic and treatment strategies for individuals with early multiple pulmonary nodules.

Critical steps
In this study, several critical steps were identified as essential for the success of the protocol: (1) Sorting and arranging DICOM scan sequence coordinates to generate an accurate 3D volume of the lung scan (step 1.2.2); (2) Isotropic transformation to ensure the correct aspect ratio of the 3D volume, which is crucial for subsequent 3D reconstruction (step 1.3.4); (3) Reconstruction of the entire lung using an early multiple pulmonary nodules model, enabling the identification of the dominant pulmonary nodule area (step 4.1); (4) Detailed visualization and examination of the local area containing the dominant lesion (step 5.2).

Modifications and troubleshooting
The segmentation of lung tissue structures may be affected by the grayscale threshold offset in the scanning sequence, potentially resulting in inaccurate image segmentation in some scans. In cases of inaccurate segmentation, a separate filter (repeating step 3) can be designed to obtain precise lung tissue contours. Maintaining the highest precision in isovoxel transformation^16,17is crucial to ensure the accurate utilization of data. These steps are expected to become more intelligent and automated in the future. With the advancement of large-scale medical imaging models, precise contour identification through computer vision is also an important direction for future development¹¹.

Limitations
Simplified implementation of lung contour extraction may lead to errors at the boundary of the lung's 3D contour, potentially affecting the visualization of small nodules near the lung's edge. However, the impact of this limitation is minimal when visualizing the dominant lesion area in cases of multiple pulmonary nodules.

Significance with respect to existing methods
Compared to computer vision approaches, this method offers a comprehensive representation of lung tissue structure, including the relationships between multiple pulmonary nodules and lung tissue, while avoiding the issues of false positives and false negatives. Additionally, it effectively filters out signal interference from other tissue structures, leading to more precise and accurate diagnoses with enhanced contrast and clarity.

Future applications
This 3D visualization method holds substantial potential for various clinical applications, such as facilitating doctor-patient communication, enabling precise diagnosis, supporting data-driven evidence-based classification, aiding in treatment planning, and evaluating prognosis. It can assist in preoperative planning, provide intraoperative navigation for the surgical removal of multiple lung nodules, and monitor changes in nodule size and shape over time to assess treatment effectiveness. Overall, it has the capacity to enhance clinical decision-making in the diagnosis and treatment of multiple pulmonary nodules.

Materials

Name	Company	Catalog Number	Comments
MATLAB	MathWorks	2022B	Computing and visualization
Tools for Modeling	Intelligent Entropy	PulmonaryNodule V1.0	Beijing Intelligent Entropy Science & Technology Co Ltd. Modeling for CT/MRI fusion