A subscription to JoVE is required to view this content. Sign in or start your free trial.

Summary

A process of registering cone-beam computed tomography scans and digital dental images has been presented using artificial intelligence (AI) -assisted identification of landmarks and merging. A comparison with surface-based registration shows that AI-based digitization and integration are reliable and reproducible.

Abstract

This study aimed to introduce cone-beam computed tomography (CBCT) digitization and integration of digital dental images (DDI) based on artificial intelligence (AI)-based registration (ABR) and to evaluate the reliability and reproducibility using this method compared with those of surface-based registration (SBR). This retrospective study consisted of CBCT images and DDI of 17 patients who had undergone computer-aided bimaxillary orthognathic surgery. The digitization of CBCT images and their integration with DDI were repeated using an AI-based program. CBCT images and DDI were integrated using a point-to-point registration. In contrast, with the SBR method, the three landmarks were identified manually on the CBCT and DDI, which were integrated with the iterative closest points method.

After two repeated integrations of each method, the three-dimensional coordinate values of the first maxillary molars and central incisors and their differences were obtained. Intraclass coefficient (ICC) testing was performed to evaluate intra-observer reliability with each method's coordinates and compare their reliability between the ABR and SBR. The intra-observer reliability showed significant and almost perfect ICC in each method. There was no significance in the mean difference between the first and second registrations in each ABR and SBR and between both methods; however, their ranges were narrower with ABR than with the SBR method. This study shows that AI-based digitization and integration are reliable and reproducible.

Introduction

Three-dimensional (3D) digital technology has broadened the scope of diagnosis and planning for orthodontic or surgical-orthodontic treatment. A virtual head constructed from a facial cone-beam computed tomography (CBCT) image can be used to evaluate dentofacial and dental abnormalities, plan orthognathic surgery, fabricate dental wafers and implant surgical guides using computer-aided design and manufacturing^1,2,3,4. However, CBCT scans have a low representation of dentition, including dental morphology and interocclusal relationship, which are due to their limited resolution and streak artifacts from dental restoration or orthodontic brackets⁵. Therefore, the dental features have been substituted on CBCT images with digital dental images (DDI), such as scanned casts or intraoral scan images.

For reliable integration of DDI on CBCT images, numerous studies reported various methods such as the use of fiducial markers^6,7, voxel-based⁸, and surface-based registrations (SBRs)^9,10. These procedures have their methods of using extraoral markers, multiple CBCT scans, and extra process steps such as cleaning metal artifacts on CBCT images. Regarding SBR accuracy, several previous studies reported errors ranging from 0.10 to 0.43mm^9,11. In addition, Zou et al. evaluated intra-/inter-observer reliability and errors between a digital engineer and an orthodontist using SBR and reported the need for clinical experience and repeated learning¹⁰.

Artificial intelligence (AI) has been used to predict treatment outcomes¹² and digitize landmarks on cephalometric radiographs¹³ or CBCT images^14,15,16, and some commercial software is currently available to assist in this process¹⁷. Accurate identification of anatomical landmarks on 3D images is challenging because of the ambiguity of flat surfaces or curved structures, areas of low density, and the wide variability of the anatomical structures.

AI-based, machine-learned automation can be applied not only for digitization but also for the integration of DDI and dentofacial CBCT. However, there is little research on the accuracy of an AI-based registration (ABR) compared to the existing surface-based method. To achieve more accurate outcomes of 3D skeletal and dental changes through bimaxillary orthognathic surgery, it is necessary to evaluate the accuracy of AI-based programs when merging CBCT and DDI. Therefore, this article presents a step-by-step protocol for digitizing and integrating CBCT and DDI with an AI-based registration (ABR) and to evaluate its reliability and reproducibility compared to that of SBR.

Protocol

This retrospective study was reviewed and approved by the Institutional Review Board of Seoul National University Bundang Hospital (B-2205-759-101) and complied with the principles of the Declaration of Helsinki. Digital Imaging and Communications in Medicine (DICOM) files from CBCT and DDI in Standard Tessellation Language (STL) format from the dental cast were utilized in the study. The need for informed consent was waived due to the retrospective nature of the study.

1. CBCT and Digital Dental Images (DDI) acquisition

1. Select patients based on the following inclusion criteria: skeletal Class III malocclusion; bimaxillary surgery via computer-aided planning; and orthodontic treatment with fixed edgewise appliances.
2. Exclude patients with craniofacial syndromes, cleft lip/palate, or missing maxillary first molars or right central incisor.
3. Obtain CBCT scans with a field of view of 200 mm x 180 mm, a voxel size of 0.2 mm, and exposure conditions of 80 kVp, 15 mA, and 10.8 s. Ensure that patients are in an upright position with their teeth in maximum intercuspation. Save the scans as Digital Imaging and Communications in Medicine (DICOM) data files.
4. Acquire DDIs from dental stone casts or direct intraoral scanning and save them in the Standard Tessellation Language (STL) format as separate maxillary and mandibular dentition.

2. AI-based Registration Protocol (ABR)

1. CBCT reorientation and digitization
   1. Open the software and click the Load DICOM File button to import CBCT DICOM files into the software.
   2. Select any one of the DICOM files in the DICOM data folder and click open.
      NOTE: When DICOM files are loaded, the software automatically reconstructs them into a CBCT craniofacial volume.
   3. Click on the Reorientation button in the Landmark panel (Figure 1).
   4. N (Nasion): click the V notch of the frontal bone in the 3D view (Figure 2). Immediately after the click, observe that the blue dot (activated) turns into a red cross that will appear in the axial, sagittal, and coronal views as well. Click the blue triangular arrows back and forth to identify the landmark.
      1. In the sagittal view, scroll the mouse wheel up and down to find the most anterior point where the frontonasal suture meets the nasal and frontal bones and click to determine the vertical and anteroposterior position of the landmark.
      2. In the coronal view, scroll the mouse wheel up and down to find the moment just before the nasal bone disappears to ensure the most anterior point and click to determine the horizontal position of the Nasion.
      3. In the axial view, adjust the anteroposterior position as it is on the most anterior point.
   5. R Or (Orbitale): click the most inferior point on the margin of the right orbital contour in the 3D model (Figure 2).
      1. In the coronal view, scroll the mouse wheel up and down to find the lowest point on the inferior margin of the right orbit and click.
      2. In the sagittal view, click the most superior point of the right maxilla or zygomatic bone structure that constitutes the lower boundary of the orbit.
      3. In the axial view, scroll through the mouse and click so that the red cross is positioned where the eye orbit rim meets.
   6. L Or (Orbitale): click the most inferior point on the margin of the left orbital contour in the 3D model (Figure 2) and modify the point on the three views as in the process for R Or.
   7. R Po (Porion): click the most superior point of the outline of the right external auditory meatus in the 3D model (Figure 2).
      1. In the coronal view, click the lowest point of the right temporal bone to determine the horizontal and vertical positions.
      2. In the sagittal view, click the most superior point of the outline of the right external auditory meatus to adjust the vertical and anterior-posterior positions.
      3. In the axial view, scroll the mouse wheel to click where the external auditory canal appears, in which the line of the temporal bone disappears.
   8. L Po (Porion): click the most superior point of the outline of the left external auditory meatus in the 3D model (Figure 2) and modify the point in the three multiplanar views as in the process for R Po.
      NOTE: The five basic skeletal landmarks, including Nasion, right and left orbitales, and right and left porions in the reconstructed craniofacial model (Figure 2), are now identified.
   9. Click on the Done button to complete the reorientation of the reconstructed craniofacial model.
   10. Click on the Preliminary Landmark Picking button in the Landmark panel and select the Dentition I landmark group.
       NOTE: Landmark groups of cranial base, TMJ, Maxillary Skeletal, Mandibular Skeletal, Dentition I, and Soft tissue are already selected for craniofacial analysis.
   11. Click on the Execute button in the Preliminary Landmark Picking panel and let the software automatically pick preliminary landmarks and determine their coordinates.
   12. When modifying the landmarks, press the Manual Landmark Picking button in the Volume tab, make the necessary adjustments, and click on the Done button to confirm (Figure 3).

3. DDI merging procedure

1. Click on the Registration of Dentition Scan button in the Tools panel (Figure 4).
2. Select maxilla dentition and click on the Load button in the Dentition Registration panel.
3. Select the STL files of the same patient with the CBCT model in the folder to load maxilla dentition STL files. Once the STL files are open, look for DDIs on the right side of the screen and four views (3D, axial, sagittal, and coronal) of the CBCT on the left side of the screen.
4. Pick the registration landmarks on the loaded DDI: the mesiobuccal cusps of the right maxillary first molar (R U6CP), the right maxillary central incisor midpoint on incisal edge (R U1CP), and the mesiobuccal cusp of the left maxillary first molar (L U6CP) (Figure 5) by switching the blue triangular arrows back and forth.
   NOTE: Left-click and drag the mouse to rotate the DDI and right-click and drag to zoom in and out. The registration landmarks are simultaneously calibrated by machine-learned automation after being digitized manually.
5. Click on the Done button in the Dentition Registration panel.
6. Click on the Yes button to confirm the automatic registration (Figure 6).
7. For mandibular dentition merging, select mandible dentition and click on the Load button in the Dentition Registration panel. Repeat steps 3.2 to 3.6. Pick the registration landmarks on the mandibular dentition: the mesiobuccal cusp of the right/left lower first molar (R-/L- L6CP), right lower first incisor midpoint on the incisal edge (R L1CP).
8. The DDI is now merged with the reconstructed CBCT model (Figure 7).
   1. When modifying the merging, click on the Pick Registration Landmark button in the Dentition Registration panel (Figure 8).

4. Obtaining the 3D coordinate values (x, y, and z) of each landmark

1. Click on the Manual Landmark Picking button in the Volume tab or click the Analysis tab to obtain the 3D coordinate values of the landmarks. For data export, go to analysis tab→data export panel, and click on the Landmark button to save the data as a file.
   NOTE: The X-plane (horizontal) is the plane that passes through the Nasion, parallel to the Frankfort horizontal (FH) plane which passes through the left and right Orbitales and right Porion. The Y-plane (midsagittal) is perpendicular to the X-plane, passing through the Nasion and basion. The Z-plane (coronal) sets the plane perpendicular to the horizontal and midsagittal planes via the Nasion (zero point; 0, 0, and 0) (Figure 9).

Representative Results

Here we described the integration process of CBCT and DDI using an AI-based program. To evaluate its reliability and reproducibility, a comparative study with surface-based registration (SBR) was conducted. It was determined that a minimum sample size of ten was required after a power analysis under correlation ρ H1 = 0.77, α = 0.05, and power (1−β) = 0.80¹⁸. A total of 17 sets of CBCT scans and digital dental images from orthognathic patients at Seoul National University Bundang Hospital from March 2016 to October 2019 were studied. The entire SBR and ABR processes for the same population were repeated twice by the same examiner, an orthodontic resident who had trained in landmark identification for more than 1.5 years. SBR was performed through a protocol similar to that of some previous studies^9,10 (Figure 10). The mean differences in x, y, and z coordinate values of R-/L-U6CP, and R U1CP after repeated integrations with each program were evaluated. All data were statistically analyzed with SPSS 22.0 software. Reliability in the coordinates of the landmarks was analyzed in each ABR, SBR, and between them to evaluate reproducibility using intraclass correlation (ICC)¹⁹.

The intra-observer reliability of x-, y-, and z-coordinate values of R-/L-U6CP, and R U1CP was significant and almost perfect for ABR (0.950 ≤ ICC ≤ 0.998) and SBR (0.886 ≤ ICC ≤ 0.997), respectively (Table 1). The reliability difference in the y- and z-coordinate values in most landmarks was significant and showed almost perfect to substantial agreement between the SBR and ABR. However, the x-coordinate values of R-/L-U6CP and R U1CP presented moderate, mediocre, and low agreement, respectively, and were insignificant.

As shown in Table 2, the mean differences of all coordinate values from the repeated integrations were not significantly different in each method. These differences on the x-coordinates ranged from -0.005 to -0.098 mm for ABR and from -0.212 to 0.013 mm for SBR. They ranged from -0.084 to -0.314 mm on the y-coordinates for ABR, and from−0.007 to 0.084 mm for SBR, and ranged from -0.005 to 0.045 mm on the z-coordinates for ABR and from−0.567 to 0.074 mm for SBR. However, there was no significance in the mean difference between the first and second registrations between the ABR and SBR.

Figure 1: Reorienting a craniofacial model. This is initiated by clicking on the Reorientation button in the Landmark panel. Please click here to view a larger version of this figure.

Figure 2: The five basic landmarks for reorientation of the reconstructed craniofacial model; nasion, right and left orbitales, and right and left porions. Please click here to view a larger version of this figure.

Figure 3: Landmarks and their coordinates after preliminary automatic landmark selection. Reviews and modifications of the landmarks can be done by clicking on the Manual Landmark Picking button in the Volume tab. Please click here to view a larger version of this figure.

Figure 4: Initiation of merging digital dental images with the reoriented craniofacial model. This is done by clicking on the Registration of Dentition Scan button in the Tools panel. Please click here to view a larger version of this figure.

Figure 5: Location of the three registration landmarks on the loaded digital dental images. The mesiobuccal cusps of the right maxillary first molar (R U6CP), the right maxillary central incisor midpoint on incisal edge (R U1CP), and the mesiobuccal cusp of the left maxillary first molar (L U6CP). These landmarks were simultaneously calibrated by machine-learned automation. Please click here to view a larger version of this figure.

Figure 6: Confirmation of the three registration landmarks on the loaded digital dental images and CBCT. The right and left mesiobuccal cusps of the maxillary first molars (R U6CP, L U6CP) and right upper central incisor midpoint (R U1CP). Clicking on the Yes button performs the automatic registration. Abbreviation: CBCT = cone-beam computed tomography. Please click here to view a larger version of this figure.

Figure 7: The reconstructed craniofacial model with the digital dental image merged. Please click here to view a larger version of this figure.

Figure 8: Modifying the merging. When modifying the merging, click on the Pick Registration Landmark button in the Dentition Registration panel. Please click here to view a larger version of this figure.

Figure 9: Reference planes of the program. The X-plane (horizontal) is the plane that passes through the Nasion, parallel to the Frankfort horizontal (FH) plane that passes through the left and right Orbitales and right Porion. The Y-plane (midsagittal) is perpendicular to the X-plane, passing through the Nasion and basion. The Z-plane (coronal) sets the plane perpendicular to the horizontal and midsagittal planes via Nasion (zero point; 0, 0, and 0). Please click here to view a larger version of this figure.

Figure 10: Surface-based registration of the maxillary digital dental images into the dental portions of reconstructed CBCT images. (A) Before and (B) after merging. First, the initial points were registered using the mesiobuccal cusps of the maxillary first molars and the contact point of the central incisors in the CBCT and DDI. Subsequently, the surface was registered to achieve a more accurate integration using the iterative closest points algorithm. Abbreviation: CBCT = cone-beam computed tomography; DDI = digital dental images. Please click here to view a larger version of this figure.

Table 1: Reliability in three coordinates of each landmark when integrating facial CBCTs and digital dental images in each ABR and SBR and between them. *paired t test; †independent t test. ICC > 0.8/0.6/0.4/0.2 or ≤ 0.2 represent very good, good, moderate, fair, or poor strength of agreement, respectively. Abbreviations: CBCT = cone-beam computed tomography; AI = artificial intelligence; ABR = AI-based registration; SBR = surface-based registration; CI = confidence interval; ICC= intraclass coefficient. Please click here to download this Table.

Table 2: The mean differences in the three coordinates of each landmark from repeated registrations of facial CBCTs and digital dental images with the ABR and SBR. Δ (1st-2nd), the mean difference in x, y, and z coordinates of each landmark between the first registration (1st) and second registration (2nd) of DDI and facial CBCT images. *paired t test; †independent t test; ^bWilcoxon Signed-rank test. Significance was set at P < 0.05. Abbreviations: CBCT = cone-beam computed tomography; AI = artificial intelligence; ABR = AI-based registration; SBR = surface-based registration; S.D. = standard deviation. Please click here to download this Table.

Discussion

Using the presented protocol, digitization of landmarks and integrating CBCT and DDI can be easily accomplished using machine-learned software. This protocol requires the following critical steps: i) reorientation of the head in the CBCT scan, ii) digitization of CBCT and DDI, and iii) merging CBCT images with the DDI. The digitization of five landmarks for the reorientation of the head is critical because it determines the 3D position of the head with reference planes in spatial areas. Three landmarks (R-/L-U6CP and R U1CP) on the DDI were calibrated by machine-learned automation after being digitized manually. The only manual process was locating the basic five skeletal landmarks in the reconstructed CBCT model, including Nasion, right and left orbitales, and porions (Figure 2), and the three dental landmarks in the DDI, including R-/L-U6CP, and R U1CP (Figure 5). Therefore, the user needs to be experienced in digitizing these eight landmarks, which may influence errors of registration. The average consuming time of SBR was 3-4 min for CBCT and DDI merging by a program expert. In the ABR program, an average of 50 s was consumed for picking five landmarks for reorientation, 40 s for picking three landmarks in DDI, and 2-3 s for the program to merge CBCT and DDI. In addition, the time for automatic landmark picking in the whole CBCT varies from 30 s to 2 min according to the landmark group selection.

When the digitization of some landmarks is imprecise, they can be modified by manual digitization and clicking on Manual Registration. Suppose there is any anatomical or morphological variance (e.g., missing central incisors or first molars), a clinician can identify specific landmarks by customizing certain points in the CBCT and DDI to match.

Regarding the mean errors of various integration methods with CBCT and DDI, previous studies using markers reported the range of registration errors to be from 0.1 to 0.5 mm²⁰. In an artifact-resistant surface-based registration, Lin et al. reported accuracy errors from 0.10 to 0.43 mm¹¹. However, in our study, the range of mean difference in ABR was less than that with SBR (0.001 to 0.314 mm; Table 2). This means that ABR can have more accuracy than SBR. Interestingly, the z-coordinate of the maxillary incisor in the ABR and the x-coordinate in the SBR showed relatively less average errors. It may be derived from different landmarks of the maxillary incisor between ABR and SBR, which is the midpoint and contact point of the maxillary incisor, respectively.

In addition, metal artifacts and the operator's level of experience during integration can influence the accuracy when merging CBCT and DDI. Nkenke et al. reported 0.13 mm and 0.27 mm without and with metal artifact correction, respectively²¹. Another study found that maxillary teeth presented poor-to-moderate reliability in x-coordinate values with SBR between different operator groups¹⁰. Consistently, in our study, the reliability of x-coordinate values of maxillary first molars and incisors presented moderate-to-poor agreement in comparison between ABR and SBR. In addition, the reliability in the y-/z- coordinates in most landmarks was almost perfect to substantial agreement, while the x-coordinates showed moderate to low agreement (Table 1). This variability in x-coordinates might be derived from the ambiguity of the landmarks due to occlusal wear in the first molars and crowding or spacing in maxillary central incisors.

Regarding AI-identification of CBCT, landmarks on crests, edges, apices, and between areas with distinctive densities are easier to locate and, therefore, tend to present the highest accuracy²². Guillot et al. found that landmarks in the cranial base showed higher accuracy than those in the maxilla and mandible¹⁴. However, these studies did not merge CBCT with DDI and evaluated the identification of anatomical landmarks in only CBCT by AI.

This study had a small sample size used to evaluate the reliability of ABR; further evaluation with a larger sample size is needed. Considering this study was only conducted by one examiner, inter-examiner differences might affect the reliability, which can be further studied. In addition, as this protocol was based on a machine-learned algorithm in which convolution neural networks were developed with a sample database, the database should be updated periodically. It should be understood that anatomical diversity of teeth and facial bone, especially in dentofacial deformities, differences in radiographic density, and resolution of CBCT and DDI could result in a compromised data representation. This ABR protocol can be applied to predesign an implant or periodontal surgery and simulate computer-aided orthognathic surgery and orthodontic treatment.

Materials

G*Power	Heinrich Heine Universität, Dϋsseldorf, Germany	v. 3.1.9.7	A sample size calculuation software
Geomagic Qualify®	3D Systems, Morrisville, NC, USA	v 2013	3D metrology feature and automation software, which transform scan and probe data into 3D to be used in design, manufacturing and metrology applications
KODAK 9500	Carestream Health Inc., Rochester, NY, USA	5159538	Cone Beam Computed Tomograph (CBCT)
MD-ID0300	Medit Co, Seoul, South Korea Seoul, Korea	61010-1	Desktop model scanner
ON3D	3D ONS Inc., Seoul, Korea	v 1.3.0	Software for 3D CBCT evaluation; AI-based landmark identification, craniofacial and TMJ analysis, superimposition, and virtual orthognathic surgery
SPSS	IBM, Armonk, NY, USA	v 22.0	A statistic analysis program