Multimodel Interpretable Machine Learning for Osteoporosis Prediction in Diabetes Using a Large Critical Care Electronic Health Record Database

Summary

This protocol describes the development and evaluation of an interpretable ensemble tree-based machine learning model using routinely collected clinical data to predict osteoporosis risk in patients with diabetes.

Abstract

Diabetes mellitus is associated with increased skeletal fragility and elevated fracture risk; however, early identification of osteoporosis (OP) in diabetic populations remains challenging. This study aimed to develop and compare multiple machine-learning (ML) models for predicting OP among patients with diabetes, while evaluating their clinical utility and enhancing model transparency using SHapley Additive exPlanations (SHAP). Using routinely collected demographic and laboratory data from the MIMIC-IV database, multiple ML algorithms, including tree ensembles, gradient boosting, and linear baselines, were trained and validated. Model performance was comprehensively evaluated using discrimination metrics, including the area under the receiver operating characteristic curve (AUC) and precision–recall analysis, as well as probability calibration curves and decision curve analysis (DCA) to assess net clinical benefit. The best-performing model was further interpreted using SHAP to quantify and visualize feature contributions at both global and individual levels. Among all evaluated models, ensemble tree-based methods showed improved performance. The ExtraTrees classifier achieved the highest validation performance (AUC = 0.862; average precision = 0.866). Furthermore, the selected model exhibited excellent probability calibration (Brier score = 0.154) and demonstrated substantial net clinical benefit across a wide range of risk thresholds in the DCA. SHAP analysis identified gender and age as the most influential predictors, followed by routine hematologic and metabolic laboratory indicators. Local explanations provided clinically interpretable insights into individual predictions. An interpretable ensemble tree-based model effectively predicts OP risk in patients with diabetes using routinely available clinical variables. The integration of SHAP improves clinical transparency, and strong calibration coupled with positive net clinical benefit supports its potential implementation as a reliable decision-support tool for early OP risk stratification in diabetic populations.

Introduction

Diabetes mellitus and osteoporosis (OP) are highly prevalent chronic conditions that increasingly coexist, creating a substantial burden through fragility fractures, disability, and healthcare costs. In patients with type 2 diabetes (T2D), fracture risk is paradoxically elevated even when areal bone mineral density (BMD) measured by dual-energy X-ray absorptiometry (DXA) is preserved or higher than expected, highlighting that bone “quality” and systemic metabolic factors contribute importantly to skeletal fragility^1,2. Despite DXA being the clinical standard for diagnosing OP, its accessibility and....

Protocol

Data source and ethics statement
This retrospective study used data from the MIMIC-IV (version 3.1), a publicly available, de-identified critical care EHR database hosted on PhysioNet. The database includes demographic information, vital signs, laboratory tests, diagnoses, procedures, and medications.

Access to MIMIC-IV requires credentialed authorization and adherence to the PhysioNet data use agreement and training requirements. The investigators completed the required training and were granted access (certificate number: 68351548). Because the database is de-identified, this study analyzed anonymized data and did not....

Results

Model performance comparison
To identify the optimal predictive approach for OP in patients with diabetes, multiple ML models were systematically compared. A regularized linear classifier (logistic regression) was included as a baseline control model. The overall performance comparison is summarized in Figures 3–6. Across all evaluated models, ensemble tree-based methods outperformed the linear baseline in terms of discriminative performance.

Discussion

In this study, multiple ML algorithms were systematically compared to identify an optimal predictive model for OP among patients with diabetes. Ensemble tree–based methods, particularly the ExtraTrees model, showed improved discrimination, reliable probability calibration, and greater net clinical benefit compared to linear baselines and other algorithm families. These findings indicate that nonlinear interactions and complex feature relationships are important for accurately capturing OP risk in diabetic populatio.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
CatBoost	Yandex	v1.2 (or applicable)	Gradient boosting algorithm optimized for categorical features
imbalanced-learn (SMOTE)	Scikit-learn Contrib	v0.11 (or applicable)	Library used to perform Synthetic Minority Over-sampling Technique for class balancing
LightGBM	Microsoft Corporation	v4.0 (or applicable)	Gradient boosting framework based on decision trees
Matplotlib	Matplotlib Development Team	v3.7 (or applicable)	Visualization library used for plotting figures and performance curves
MIMIC-IV Clinical Database	PhysioNet	v3.1	Publicly available critical care electronic health record dataset used as the data source
Navicat Premium	PremiumSoft CyberTech Ltd.	v17.0	Database management tool used for SQL-based data extraction
NumPy	NumPy Developers	v1.24 (or applicable)	Numerical computing library used for array operations and data processing
OpenPyXL	Eric Gazoni, Charlie Clark	v3.1 (or applicable)	Library used for reading and writing Excel files
Pandas	Pandas Development Team	v2.0 (or applicable)	Data manipulation and analysis library for structured datasets
PostgreSQL	PostgreSQL Global Development Group	v14 (or applicable)	Relational database system used to query and manage MIMIC-IV data
Python	Python Software Foundation	v3.8+ (or applicable)	Programming language used for data processing, modeling, and analysis
Scikit-learn	Scikit-learn Developers	v1.3 (or applicable)	Machine learning library used for model development, preprocessing, and evaluation
SciPy	SciPy Developers	v1.10 (or applicable)	Scientific computing library used for statistical analysis
Seaborn	Michael Waskom	v0.12 (or applicable)	Statistical data visualization library for enhanced graphical outputs
SHAP (SHapley Additive exPlanations)	Scott Lundberg	v0.42 (or applicable)	Framework used to interpret model predictions using feature attribution values
XGBoost	DMLC (Distributed Machine Learning Community)	v2.0 (or applicable)	Gradient boosting algorithm widely used for structured data modeling