Quantitative Super-Resolution Ultrasound Microvascular Features for Machine Learning-Based Classification of Thyroid Nodules

Summary

This study compared five machine learning algorithms for thyroid nodule classification using super-resolution ultrasound features. SVM performed best (accuracy 68.6%, Area Under the Curve 0.690), identifying microcalcification and post-contrast enlargement as key contributors, but the findings require external validation before clinical use.

Abstract

This study compared five machine learning algorithms—Random Forest (RF), Support Vector Machine (SVM), Decision Tree (DT), eXtreme Gradient Boosting (XGBoost), and Gradient Boosting (GB)—for classifying thyroid nodules using quantitative features derived from conventional ultrasound, contrast-enhanced ultrasound (CEUS), and super-resolution ultrasound (SRUS). The retrospective study included 68 thyroid nodules from 63 patients (30 benign and 38 confirmed as papillary thyroid carcinoma [PTC]) and was analyzed using 25 quantitative features via five-fold patient-grouped cross-validation, ensuring that all nodules from the same patient were assigned to the same fold. Model performance was summarized across folds and from pooled out-of-fold (OOF) predictions, and a focused OOF SHapley Additive exPlanations (SHAP) analysis using a permutation-based explainer was performed for the best-performing SVM. SVM achieved the highest mean accuracy (0.686 ± 0.120) and mean Receiver Operating Characteristic-Area Under Curve (ROC-AUC) (0.690 ± 0.164), with a pooled OOF sensitivity of 0.842 and specificity of 0.500, while microcalcification and post-contrast enlargement were identified as the most significant model-specific SHAP contributors. RF offered balanced performance (accuracy 55.9%, sensitivity 55.3%, specificity 56.7%), whereas DT performed poorly (accuracy 0.429, ROC-AUC 0.447), approaching random guessing. The study demonstrated that quantitative SRUS measurements can be incorporated into conventional classifiers; however, the findings remain exploratory and require prospective, multicenter validation with larger cohorts before clinical implementation.

Introduction

Thyroid nodules represent a highly prevalent clinical finding, with detection rates approaching 68% in asymptomatic populations when examined by ultrasound examination¹. Recent meta-analyses have revealed that the global prevalence of thyroid nodules has increased from 21.53% during 2000-2011 to 29.29% during 2012-2022, affecting approximately one in every four individuals in the general population^2,3. Among these nodules, 7–15% are ultimately diagnosed as thyroid cancer, of which papillary thyroid carcinoma (PTC) represents the predominant subtype^4,5. PTC accounts for 80–90% of all thyroid malignancies and has become one of the fastest-growing cancer types, with incidence rates increasing from 4.8 to 14.9 per 100,000 between 1975 and 2012⁶.

Conventional ultrasound remains the cornerstone for thyroid nodule evaluation, offering a non-invasive, cost-effective, and widely accessible diagnostic modality⁷. Traditional sonographic features suggestive of malignancy include marked hypo-echogenicity, irregular margins, microcalcifications, and taller-than-wide shape^8,9. However, it is fundamentally limited by the acoustic diffraction barrier, which restricts spatial resolution to approximately 100 micrometers¹⁰. This limitation significantly constrains the ability to visualize microvasculature and assess fine vascular morphology, which are critical parameters for distinguishing between benign and malignant thyroid lesions¹¹. Color Doppler flow imaging (CDFI) and contrast-enhanced ultrasound (CEUS), while improving blood flow detection, can only visualize vessels with flow velocities exceeding 1 cm/s and remain constrained by diffraction-limited resolution^12,13.

Super-resolution ultrasound (SRUS), as a transformative technology, overcomes the traditional diffraction limit of conventional ultrasound systems^14,15. By leveraging contrast microbubbles as point targets for localization and tracking, SRUS achieves micron-scale spatial resolution, enabling visualization and quantification of microvasculature at an unprecedented level of detail^16,17. This technique can measure both microvascular density (MVD) and microvascular flow rate (MFR), providing structural and functional assessments precisely unattainable through non-invasive imaging^17,18. Pilot studies have demonstrated that benign thyroid nodules exhibit significantly higher MFR than malignant ones, with mean values of 16.76 ± 6.82 mm/s versus 9.86 ± 4.54 mm/s, respectively¹⁹. The introduction of "ultrasound microvasculomics"—a high-throughput extraction of quantitative features from SRUS images—further enhances the potential for precise and personalized microvascular assessment.

Machine learning (ML) has revolutionized medical image analysis, particularly in thyroid nodule characterization^20,21. Deep learning models, particularly convolutional neural networks (CNNs), have achieved Receiver Operating Characteristic-Area Under Curve (ROC-AUC) values exceeding 0.90 in previous studies^22,23,24. Ensemble machine learning approaches combining multiple algorithms—including Random Forest (RF), Support Vector Machines (SVM), Decision Trees (DT), Gradient Boosting (GB), and eXtreme Gradient Boosting (XGBoost)—have shown promise in improving diagnostic accuracy and robustness across heterogeneous datasets²¹. As Habchi et al.²⁵ recently reviewed comprehensively, the field of artificial intelligence (AI) for thyroid cancer has rapidly expanded to include visual converters, large language models, and hybrid architectures, reflecting the development of the field towards increasingly complex image-based diagnostic frameworks.

While conventional ultrasound features have been extensively studied in conjunction with AI algorithms, the combination of super-resolution microvascular parameters with ML models remains largely unexplored. Despite the promising potential of SRUS technology and the proven efficacy of machine learning algorithms in medical imaging, several critical questions remain unanswered. First, which machine learning architecture—whether traditional algorithms such as RF and SVM, or advanced ensemble methods like XGBoost and GB—performs optimally when applied to SRUS-derived microvascular features? Second, how do these models compare in terms of sensitivity, specificity, and overall diagnostic accuracy for differentiating benign from malignant thyroid nodules? Third, which microvascular parameters extracted from SRUS imaging contribute most significantly to classification performance? Addressing these questions is essential for establishing evidence-based frameworks for the clinical translation of SRUS-assisted thyroid nodule diagnosis.

Accordingly, this study aims to comprehensively evaluate the performance of five ML algorithms (RF, SVM, DT, XGBoost, and GB) for classifying thyroid nodules using super-resolution ultrasound microvascular imaging data. By systematically comparing these approaches and identifying the most informative microvascular biomarkers, this investigation seeks to advance the diagnostic paradigm for thyroid nodule characterization and contribute to the growing body of evidence supporting the clinical utility of SRUS technology in oncological imaging. We hypothesized that the prespecified 25 variable feature set would provide above-chance discrimination between benign nodules and nodules reported as papillary thyroid carcinoma. Patient-grouped cross-validation was used to reduce information leakage, and a focused SHAP analysis was added to describe how the selected classifier used the measured variables. Both analyses were exploratory.

Protocol

Study design and patient population

This retrospective study analyzed thyroid ultrasound examinations obtained between 13 June 2024 and 13 January 2025. The study protocol was approved by the Ethics Committee of Beijing Friendship Hospital, Capital Medical University (approval No. BFHHZS20240300) and was conducted in accordance with the ethical principles outlined in the Declaration of Helsinki, and informed consent was waived because of the retrospective design. The dataset comprised 68 thyroid nodules from 63 patients (30 benign and 38 malignant), and the nodule was the analytic unit. Reference diagnoses were based on ultrasound-guided fine-needle aspiration cytology (FNAC). The malignant group comprised 38 nodules reported as papillary thyroid carcinoma on FNAC.

Inclusion criteria

Eligible patients were required to meet all of the following criteria: (1) underwent conventional grayscale and Doppler ultrasonography, followed by CEUS with satisfactory image quality permitting subsequent microvascular flow imaging analysis; (2) had a cytopathological diagnosis of PTC confirmed by fine-needle aspiration biopsy, with or without concomitant positive BRAFV600E mutation testing; or had a Bethesda Category III (atypia of undetermined significance) cytology result, but with concurrent positive BRAFV600E mutation testing; (3) had a cytopathological diagnosis of benign, proliferative nodules, adenomatous nodules, or benign follicular nodules in Bethesda Category II without concomitant positive BRAFV600E mutation testing.

Exclusion criteria

Patients were excluded from the study if any of the following conditions were present: (1) poor-quality CEUS images or insufficient microbubble signal precluding reliable microvascular flow assessment; (2) absence of cytological or pathological results; (3) histopathological diagnosis of a rare or special subtype of thyroid carcinoma (e.g. anaplastic variants); or (4) cytological findings suggestive of follicular neoplasm (Bethesda Category IV) or any lesion with indeterminate follicular origin, regardless of mutational status; (5) Hashimoto's thyroiditis

Ultrasound, CEUS, and SRUS acquisition

All examinations were performed with the referenced ultrasound system and a linear-array transducer. Patients were positioned supine with the neck extended. The target nodule was localized and measured on B-mode ultrasound; microcalcifications were recorded, and the same lesion-centered imaging plane was used to assess intranodular vascularity with Color Doppler.

After intravenous access was established, the system was switched to a low-mechanical-index CEUS/URM mode (URM means Ultra-Resolution Microscopy imaging). CEUS was performed with a SonoVue intravenous bolus injection of 1.2 mL immediately followed by a saline flush of 5ml; the onscreen timer and continuous cine acquisition were started with bolus administration. The probe was kept in a fixed plane with minimal pressure, and the patient was asked to avoid swallowing during wash-in and wash-out. 

For quantitative CEUS, a lesion-restricted region of interest was used to derive the time-intensity parameters. For SRUS, the URM workflow localized and tracked microbubble signals after motion control and generated vessel-ratio, complexity, microvascular density, perfusion index, and flow velocity measurements. The representative image exports showed settings of VSP 4, RES 2, CTR 3, SM 2, VEN 3, and CPT 10 s; corresponding settings were unavailable for the remaining examinations.

The 25 model-input variables are listed in Table 1 and grouped by acquisition modality: age and sex; B-mode microcalcification; Color Doppler intranodular vascularity; qualitative CEUS; quantitative CEUS; and 11 SRUS microvascular measurements.

Feature selection

All 25 quantitative features were retained in the machine learning models. Only numeric features were used; patient names, registration numbers, and lesion-size fields were excluded. No data-driven feature selection was performed, and the same prespecified predictors were entered into each classifier.

Feature standardization

No transformation, imputation, or global scaling was applied before cross-validation. Standardization was applied only to the radial-basis-function SVM via a StandardScaler pipeline. The scaler was fitted on each fold's training nodules and then applied to that fold's validation nodules. Tree-based classifiers received the original numeric scales:

where x is the original feature value, µ is the training-fold mean, and σ is the training-fold standard deviation. Fold-wise preprocessing prevented validation observations from contributing to the SVM scaling parameters.

Data partitioning

Primary performance evaluation used five-fold StratifiedGroupKFold cross-validation with shuffle = True and random_state = 42. A patient identifier defined 63 groups, and all nodules from the same patient were assigned to the same fold. No patient contributed nodules to both the training and validation subsets of a fold.

Model training and validation

Training protocol

Five classifiers were evaluated: random forest (100 trees; random_state = 42), radial-basis-function SVM (C = 1.0; gamma = scale; probability = True; StandardScaler pipeline; random_state = 42), decision tree (Gini criterion; unrestricted depth; random_state = 42), XGBoost (100 estimators; learning_rate = 0.3; max_depth = 6; subsample = 1.0; colsample_bytree = 1.0; random_state = 42), and gradient boosting (100 estimators; learning_rate = 0.1; max_depth = 3; random_state = 42). No grid search, Bayesian optimization, threshold tuning, or nested model selection was performed.

Cross-validation

Within each of five patient-grouped folds, models were trained on the remaining patient groups and evaluated on the held-out groups. Accuracy, sensitivity, specificity, precision, F1-score, and ROC-AUC were calculated for each fold and summarized as mean ± SD. OOF predictions were pooled across all 68 nodules to generate one cross-validated ROC curve and one confusion matrix for each model.

Performance evaluation

Evaluation metrics

Model performance was assessed within each patient-grouped validation fold and from pooled OOF predictions using the following metrics:

Accuracy: Overall proportion of correct predictions

Sensitivity (Recall): Proportion of actual malignant nodules correctly identified

  

Specificity: Proportion of actual benign nodules correctly identified

Precision (Positive Predictive Value): Proportion of predicted malignant cases that were truly malignant

F1-Score: Harmonic mean of precision and recall

Area Under the Receiver Operating Characteristic Curve (ROC-AUC): Measure of the model's ability to discriminate between benign and malignant nodules across all classification thresholds

where TP = true positives (correctly identified malignant nodules), TN = true negatives (correctly identified benign nodules), FP = false positives (benign nodules incorrectly classified as malignant), and FN = false negatives (malignant nodules incorrectly classified as benign).

Confusion matrix analysis

OOF confusion matrices were generated by pooling the predictions made for each nodule only in the fold in which that patient was held out. Thus, each nodule received a prediction from a model trained on nodules from other patients.

Feature importance analysis

For the Random Forest model, feature importance scores were calculated based on the mean decrease in Gini impurity across all decision trees. The top 15 most important features were identified and ranked to determine which microvascular parameters contributed most significantly to classification performance.

Exploratory SHAP analysis

A focused OOF SHAP analysis using a permutation-based explainer was performed for the SVM. For each held-out nodule, only the corresponding training-fold observations were used as the background distribution (128 antithetic permutations; random seed = 20260716). Mean absolute SHAP values summarized the magnitude of contributions, and signed values indicated the direction. The analysis was exploratory and not used to infer causality, identify independent biomarkers, or define clinical thresholds.

Statistical analysis

Comparative analysis

Model performance was summarized descriptively across the five patient-grouped validation folds and in pooled OOF predictions. No formal between-model hypothesis test or P-value-based ranking was performed because the folds are related and the cohort is small.

For baseline group comparisons, normality of continuous variables was assessed within each outcome group using the Shapiro-Wilk test. Welch's t test was used when both groups were compatible with normality; otherwise, a two-sided Mann-Whitney U test was used. Categorical variables were assessed with Pearson's chi-square test, with Fisher's exact test for sparse 2 x 2 tables. P values were two-sided, exploratory, and unadjusted (alpha = 0.05).

Reproducibility

Reproducibility was supported by a specified patient-group definition, fixed random seed (42), fixed classifier configurations, and fold-wise preprocessing. Identifiers were used only for grouping and were not entered as predictors or exported with model outputs.

Results

Patient characteristics and study population

A total of 68 thyroid nodules from 63 patients were included, comprising 30 benign nodules and 38 PTCs. The nodule was the analytic unit, and patient grouping prevented nodules from the same patient from appearing in both the training and validation subsets. Baseline characteristics are summarized in Table 2.

Figure 1 shows technically successful lesion-restr...

Discussion

Current diagnostic approaches for thyroid nodules rely primarily on conventional ultrasound, which remains inherently operator-dependent and subject to inter-observer variability⁵ and the inability to visualize microvasculature below the diffraction limit. To our knowledge, our study is the first comprehensive evaluation of multiple machine learning algorithms applied to super-resolution ultrasound (SRUS) microvascular imaging data, achieving an objective quantitative assessment of the risk of PTC...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Decision Tree classifier	scikit-learn	Version 1.5.2	Machine learning classifier
Fine-needle aspiration biopsy needle			Used for ultrasound-guided FNAC
Gradient Boosting classifier	scikit-learn	Version 1.5.2	Machine learning classifier
Intravenous catheter			Used for contrast administration
Python	Python Software Foundation	Version 3.11.9	Programming language used for the machine-learning analysis
Random Forest classifier	scikit-learn	Version 1.5.2	Machine learning classifier
SHAP (permutation explainer)	SHAP Developers	Version 0.46.0	Model explainability analysis
SonoVue contrast agent	Bracco		Ultrasound contrast agent for CEUS/SRUS imaging
StandardScaler	scikit-learn	Version 1.5.2	Feature standardization for the SVM pipeline
Sterile normal saline			Used for intravenous flush following contrast injection
StratifiedGroupKFold	scikit-learn	Version 1.5.2	Patient-grouped cross-validation
Support Vector Machine (SVM) classifier	scikit-learn	Version 1.5.2	Radial-basis-function kernel classifier
U5-15LE linear-array transducer	VINNO Technology	U5-15LE	Linear ultrasound probe
ULTIMUS 9E ultrasound system	VINNO Technology	ULTIMUS 9E	Ultrasound imaging system used for B-mode, Color Doppler, CEUS, and SRUS imaging
XGBoost	XGBoost Developers	Version 2.1.1	Gradient boosting framework