A subscription to JoVE is required to view this content. Sign in or start your free trial.
Research Article
This study aims to develop and externally validate a web-based system integrating machine learning models for early diagnosis and clinical phenotyping of pneumonia-associated ARDS to facilitate precision treatment.
Acute respiratory distress syndrome (ARDS) is a highly heterogeneous disease with clinical manifestations that may overlap with severe pneumonia, posing challenges for accurate differentiation. Therefore, early prediction and bedside rapid subtype clustering of ARDS patients are urgently needed. This study aims to develop a web-based system, which includes validated models of early bedside diagnosis and clinical subgroup classification, for predicting the development and phenotypes of pneumonia-associated ARDS. Diagnostic and subgroup models were developed and validated from the two large databases, Medical Information Mart for Intensive Care IV (MIMIC-IV) and Telehealth Intensive Care Unit (eICU) and were incorporated into a web-based prediction system. Data from patients with pneumonia hospitalized for more than 24 h between 2008 and 2019 were analyzed. The MIMIC-IV derivation cohort included 24,987 patients with pneumonia (14,121 with pneumonia-associated ARDS); the eICU verification cohort included 20,676 patients with pneumonia (9946 with pneumonia-associated ARDS). In diagnosis, the stacking method of machine learning performed best with an AUC of 0.919, an accuracy of 70.00%, a precision of 69.88% and a recall of 82.27% in the MIMIC-IV derivation cohort. The AUC, accuracy, precision, and recall of the eICU validation cohort were 0.915, 70.87%, 69.70% and 69.70% respectively. Pneumonia-associated ARDS was classified into three clinical phenotypes with different clinical characteristics and outcomes, all of which responded differently to treatment. Among patients in clusters 0 and 1, the in-hospital mortality rates were higher among those who received early corticosteroid treatment than among those who did not, whereas among patients in cluster 2, the in-hospital mortality rate was lower among those who received corticosteroids than among those who did not. We performed a web transformation of the diagnosis prediction and clinical subgroup classification of pneumonia-associated ARDS. Our web-based models of early bedside diagnosis and clinical subgroup classification of pneumonia-associated ARDS may assist clinicians in diagnosing and treating the disease and in promoting individualized precision treatment.
Acute respiratory failure, especially acute respiratory distress syndrome (ARDS) after lung infection, is a common, devastating problem encountered in critically ill patients. Studies have shown that the incidence of ARDS is as high as 10% among patients in intensive care unit (ICU)1, and the mortality rate is approximately 40%2,3. Severe pneumonia is widely considered to be the main cause of ARDS4. Because the clinical symptoms of severe pneumonia and ARDS are similar, it is often difficult to distinguish ARDS from severe pneumonia. Therefore, early prediction of the development of ARDS in cases of pneumonia may reduce the incidence of ARDS and rates of mortality5. In addition, because ARDS is a highly heterogeneous disease6, early and correct subgroup classification of ARDS can enable precision medicine. Such classification is also one of the main directions of research on respiratory critical illness throughout the world7, the goal is to improve the effectiveness of targeted subgroup intervention.
At present, the independent risk factors for ARDS have been studied extensively8, but few have focused on the prediction of ARDS in patients with pneumonia. Moreover, no studies have been conducted on specific clinical phenotypes of pneumonia-associated ARDS; most phenotype studies have focused on the entire population of patients with ARDS. Calfee et al.9 classified ARDS into hyperinflammatory and hypoinflammatory phenotypes. To define these biological phenotypes, plasma biomarkers must be used as categorically defining variables, but such an investigation is not readily available at the bedside. In one study, readily available clinical indicators were used to classify ARDS into three clinical phenotypes, whose responses to randomized interventions were also assessed, but the study was based on the entire population of patients with ARDS10. Defining the clinical phenotypes of ARDS according to different causes, such as pneumonia, may yield more refined and accurate results.
The aims of this study were to use machine learning to construct a predictive model of pneumonia-associated ARDS with early clinical data; to use early and easily available clinical data to classify pneumonia-associated ARDS into clinical phenotypes and to explore differences in their clinical characteristics, outcomes, and treatment responses; and to implement the prediction model and classification model as a Web-based application that would assist clinicians in the diagnosis and treatment of pneumonia-associated ARDS and promote further research.
This study accessed the Medical Information Mart for Intensive Care IV (MIMIC-IV) Database11(Version 1.0, PhysioNet: https://physionet.org/content/mimiciv/1.0/) and Telehealth Intensive Care Unit (eICU) Database12(Version 2.0, PhysioNet: https://physionet.org/content/eicu-crd/2.0/) after completing the Protecting Human Research Participants examination (Record ID: 44151052). This study was conducted in accordance with the principles of the Declaration of Helsinki (2013), and patients had provided consent for their data to be captured in the two databases. Ethical approval was waived for this study because the data in the eICU and MIMIC-IV databases were fully anonymized (no personal identifiers retained).
Materials and tools
Data Sources: MIMIC-IV Database: Version 1.0, single-center open-access registry containing 76,540 ICU admissions (2008-2019), accessed through PhysioNet. eICU Database: Version 2.0, multicenter database containing >200,000 electronic medical records from 335 units at 208 U.S. hospitals (2014-2015), accessed through PhysioNet. Software & Execution Environment: RapidMiner Studio: Version 9.10.001 (execution environment: Windows 10 Pro 64-bit), used for model building (classification/clustering) and feature selection; IBM SPSS Statistics: Version 23.0 (execution environment: Windows 10 Pro 64-bit), used for statistical analysis and missing value imputation; Java Development Kit (JDK): Version Java SE 8u381 (execution environment: Windows 10 Pro 64-bit), used for Web application development; supporting IDE: Eclipse IDE 2023-09; Apache Tomcat: Version 9.0.85, used for deploying the Web-based application.
Study design and settings
The concept of this study is illustrated in Supplementary Figure 1. This study analyzed data from two large sources, the Medical Information Mart for Intensive Care IV (MIMIC-IV) and the Telehealth Intensive Care Unit (eICU) databases, to construct a predictive model of pneumonia-associated ARDS and classify affected patients according to clinical phenotypes. This study followed the guidelines of the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD).
Study samples
This study's training and testing data were obtained from MIMIC-IV. The source of external verification was the multicenter eICU database. This study used the International Classification of Diseases (Ninth and Tenth Revision) to extract data about patients with pneumonia and pneumonia-associated ARDS. Patients who were admitted for less than 24 h were excluded.
Predictors
After evaluating data availability and the rate of missing clinical variables in the MIMIC-IV and eICU datasets, 52 variables for pneumonia-associated ARDS were selected as candidate training variables to be used in machine learning, including patients' demographic characteristics, laboratory findings, and scores of illness severity. This study used a weight-by-correlation algorithm (based on the correlation weight between variables and outcomes) to screen for key predictors (variables that are ultimately included in the models) of the 52 candidate variables and then selected subgroup clustering factors from the key predictors.
To do this, open RapidMiner Studio, create a new process, and add the Weight by Correlation operator by clicking on Process > Operators > Feature Selection > Weight by Correlation. Set the parameter: Correlation threshold = 0.04 (retain variables with weight > 0.04). To force the model to account for early stages in the disease and timeliness, the maximum and minimum values of laboratory indicators were obtained within 24 h after admission. Because the Acute Physiology Score III (APSIII) was not included in the eICU database, scores on the Acute Physiology and Chronic Health Evaluation IV (APACHE IV) were used instead. This study cleaned the data and interpolated the missing values by using the multiple imputation method and standardized the input variables when developing prediction and sub-phenotyping models.
Statistical analysis
Open SPSS 23.0, import the preprocessed dataset, and select Analyze > Descriptive Statistics > Explore. Check Normality Plots with Tests to perform the Kolmogorov-Smirnov test for continuous variables. Skewed distributed variables are reported as median (interquartile range, IQR); categorical variables are reported as count (percentage, %). To compare continuous variables between groups, the Mann-Whitney U test or the Kruskal-Wallis's test was used, as appropriate. To compare categorical variables between groups, Pearson's chi-squared test or Fisher's exact test was used, as appropriate.
Model development and web presentation
For model development, this study divided the MIMIC-IV derivation cohort into a training set (90% of the full sample randomly chosen for model development and hyperparameter tuning) and a test set (10% of the full sample randomly chosen for internal testing); this study performed external verification in eICU. This study implemented machine learning with five methods: decision tree, logistic regression, naive bayes, stacking (Base learners were the logistic regression, naive bayes, and random forest methods; the stacking learner was the decision tree method), and random forest. Decision tree: click on Process > Operators > Modeling > Classification > Decision Tree with Algorithm = C4.5, Minimum leaf size = 5. For logistic regression: click on Process > Operators > Modeling > Classification > Logistic Regression with Regularization strength = 0.1, Maximum iterations = 100. For Naive Bayes: click on Process > Operators > Modeling > Classification > Naive Bayes with Kernel type = Gaussian. For Random forest: click on Process > Operators > Modeling > Classification > Random Forest with Number of trees = 100, Maximum depth = 20. For Stacking: click on Process > Operators > Modeling > Ensemble > Stacking with Base learners = Logistic Regression + Naive Bayes + Random Forest; Stacking learner = Decision Tree. This study measured the prediction performance of the developed model by computing the area under the receiver operating characteristic curve (AUC), accuracy, precision, and recall.
In constructing the classification of pneumonia-associated ARDS clinical subgroups, this study used k-means clustering with key predictors. To determine the optimal number of cluster k, the Gap statistics' value in the Gap statistics plot was examined, which suggested the optimal number of clusters. This study analyzed the clinical characteristics and outcomes of different phenotypes, and we compared the differences in rates of in-hospital mortality among patients with different clusters who did and did not receive early low- to medium-dose corticosteroid treatment.
In RapidMiner Studio, select File > Export Model and save the diagnosis prediction and subgroup classification models as .model files. Use JDK Java SE 8u381 and Eclipse IDE 2023-09 to write Web pages in Java language; import the .model files into the project. This study used the Java language to develop a Web page in which the diagnostic model and clinical subgroup clustering model were embedded, and we uploaded the model online.
Participants
The MIMIC-IV database included data from 24,987 patients with pneumonia, of whom 14,121 had pneumonia-associated ARDS (Table 1). The eICU database included data from 20,676 patients with pneumonia, of whom 9946 had pneumonia-associated ARDS (Supplementary Table 1).
Establishment and verification of pneumonia-associated ARDS prediction model
We used the data of the MIMIC-IV cohort to construct a diagnostic model for pneumonia-associated ARDS. The model was validated externally with data from the eICU cohort. Patients in the MIMIC-IV cohort were randomly divided into a training cohort (n = 22,488 [90%]) and a test cohort (n = 2499 [10%]). We used the weight by correlation algorithm to select 18 key predictors among the input variables (Supplementary Figure 2) with high weight (>0.04): the APSIII; maximum levels of anion gap, bicarbonate, glucose, potassium, sodium, and hemoglobin; fraction of inspired oxygen (FiO2); minimum and maximum levels of blood urea nitrogen (BUN), partial pressure of oxygen (PO2), the PO2/FiO2 ratio, pondus hydrogenii (pH) and partial pressure of carbon dioxide (PCO2). For machine learning, the decision tree, logistic regression, naive bayes, stacking (Base learners were the logistic regression, naive bayes, and random forest methods; the stacking learner was the decision tree method), and random forest methods were applied to data from the MIMIC-IV test cohort to predict the development of pneumonia-associated ARDS. We found that the stacking model performed best: AUC of 0.919, 70.00% accuracy, 69.88% precision, and 82.27% recall. The performances of the methods are compared in Figure 1 and in Supplementary Table 2. External validation in the eICU cohort revealed an AUC of 0.915, 70.87% accuracy, 69.70% precision, and 69.70% recall (Supplementary Figure 3).
Classification and validation of clinical phenotypes of pneumonia-associated ARDS
We categorized the 14,121 patients with pneumonia-associated ARDS in the MIMIC-IV cohort into clinical subgroups and validated the data from the 9946 patients with pneumonia-associated ARDS in the eICU cohort. After evaluating data availability and the rate of missing clinical variables. Nine input factors were analyzed: APS III, FiO2, maximum anion gap, bicarbonate, glucose, potassium, sodium, hemoglobin, and BUN. To determine the optimal number of cluster k, we examined the Gap statistics. For the MIMIC-IV pneumonia-associated ARDS cohort (n = 14,121), 100 bootstrap samples were generated using 9 key predictors. Gap values were calculated for k = 1 to k = 5. When k=3, the difference (Diff value) between the observed Gap value and the expected Gap value (from bootstrap) was maximized, and Diff values for k=4/5 decreased significantly, confirming k=3 as optimal. Diff value suggested that the optimal number of clusters is 3. Patients with pneumonia-associated ARDS were clustered into three phenotypes, which were represented in a scatter plot and centroid chart (Figure 2A-D). The baseline characteristics and outcomes of the three phenotypes in the MIMIC-IV cohort are listed in Table 2. The three phenotypes had different clinical characteristics: Patients in cluster 0 had the lowest rates of mortality (11.73% of the MIMIC-IV cohort, 15.44% of the eICU cohort), the shortest ICU stays (2.60 days for the MIMIC-IV cohort, 3.5 days for the eICU cohort), the lowest disease scores, the lowest WBC counts, the lowest BUN levels, the lowest number of diabetics patients, and the mildest overall disease; patients in cluster 1 had the highest rates of mortality (29.80% of the MIMIC-IV cohort, 33.58% of the eICU cohort), the longest ICU stays (4.20 days for the MIMIC-IV cohort, 4.50 days for the eICU cohort), the highest disease scores, the highest BUN levels, more organ damage, the highest number of cancer patients, and the most severe overall disease; and patients in cluster 2 had moderate rates of mortality (20.09% of the MIMIC-IV cohort, 28.51% of the eICU cohort), moderately long ICU stays (2.80 days for the MIMIC-IV cohort, 4.00 days for the eICU cohort), the highest WBC counts, and the heaviest infection. We validated the three clinical phenotypes in the eICU cohort and obtained a similar classification (Supplementary Table 3).
Responses to different clinical phenotypes of pneumonia-associated ARDS to early low- to medium-dose corticosteroid treatment
We determined the interaction between phenotype and early low- to medium-dose corticosteroid treatment. Early treatment with low doses of corticosteroids was defined as methylprednisolone equivalents, ≤1 mg/kg/day, within 24 h of diagnosis, and early treatment with moderate doses of corticosteroids was defined as methylprednisolone equivalents, ≤2 mg/kg/day, within 24 h of diagnosis. In the MIMIC-IV cohort, the in-hospital mortality rates in clusters 0 and 1 were higher among patients who received early low- to medium-dose corticosteroids than among those who did not (p < 0.05), whereas in cluster 2, the rates of in-hospital mortality were lower among patients who received early low-medium dose corticosteroids than among those who did not (p < 0.05). Similar results were observed in the eICU validation cohort (Figure 3 and Supplementary Table 4).
Web-based application for the early diagnosis of pneumonia-associated ARDS and clinical phenotypes
To facilitate the transformation, we used the random forest method to identify the three clinical subgroups, and we used the three clinical subgroups classified by the k-means method as outcome labels and validated them with the eICU data. Patients in the MIMIC-IV cohort were randomly divided into a training set (70%) and a test set (30%). Nine cluster factors were included as predictors, with 98.37% accuracy and a kappa value of 0.967 (Supplementary Figure 4). In the eICU cohort, the accuracy was 98.46%, and the kappa value was 0.970 (Supplementary Figure 5). We then uploaded the models for diagnosis prediction and clinical subgroup classification of pneumonia-associated ARDS to a Web page (http://101.42.164.72:8080/sys2/index; Supplementary Figure 6), allowing researchers interested in using the models to access them for free.
Data availability
The raw data used in this study are derived from two publicly accessible intensive care databases: the Medical Information Mart for Intensive Care IV (MIMIC-IV, Version 1.0) and the Telehealth Intensive Care Unit (eICU, Version 2.0). Access to these databases requires completion of the Protecting Human Research Participants examination (as conducted in this study, Record ID: 44151052) and compliance with PhysioNet's data use agreement. No additional raw data generated in this study beyond the derived analysis results are required to replicate the findings, as all core data are publicly retrievable from the aforementioned repositories. Supporting information files contain all the required information.
Figure 1: Comparison of the area under the receiver operating characteristic curve (AUC) of five machine-learning methods for predicting pneumonia-associated ARDS in the MIMIC-IV cohort. For the decision tree method, the AUC was 0.747; for the logistic regression method, the AUC was 0.683; for the naive Bayes method, the AUC was 0.704; for the stacking method (Base learners were the logistic regression, naive Bayes, and random forest methods; the stacking learner was the decision tree method), the AUC was 0.919; and for the random forest method, the AUC was 0.763. ARDS, acute respiratory distress syndrome, and MIMIC-IV, Medical Information Mart for Intensive Care IV database. Please click here to view a larger version of this figure.
Figure 2: Scatter plot and centroid chart visualization of phenotype assignments by k-means clustering in pneumonia-associated ARDS in the MIMIC-IV derivation and eICU validation cohorts. We standardized the data before cluster analysis. (A) Scatter plot visualization of phenotype assignments by k-means clustering in the MIMIC-IV derivation cohort. Of the patients, 9035 were in cluster 0, 4409 were in cluster 1, and 677 were in cluster 2. (B) Centroid chart visualization of phenotype assignments by k-means clustering in the MIMIC-IV derivation cohort. Of the patients, 9035 were in cluster 0, 4409 were in cluster 1, and 677 were in cluster 2. (C) Scatter plot visualization of phenotype assignments by k-means clustering in the eICU validation cohort. Of the patients, 6288 were in cluster 0, 2746 were in cluster 1, and 912 were in cluster 2. (D) Centroid chart visualization of phenotype assignments by k-means clustering in the eICU validation cohort. Of the patients, 6288 were in cluster 0, 2746 were in cluster 1, and 912 were in cluster 2. Abbreviations: ARDS = acute respiratory distress syndrome; MIMIC-IV = Medical Information Mart for Intensive Care IV database; eICU = Telehealth Intensive Care Unit database; APSIII = Acute Physiology Score III; BUN = blood urea nitrogen; FiO2 = fraction of inspired oxygen; and APACHE IV = Acute Physiology and Chronic Health Evaluation IV. Please click here to view a larger version of this figure.
Figure 3: Comparison of in-hospital mortality rates across phenotypes between patients in the Medical Information Mart for Intensive Care IV (MIMIC-IV)-derived cohort and Telehealth Intensive Care Unit (eICU) database-validated cohort who received early low- to medium-dose corticosteroid treatment (receive) and those who did not (not receive). (A) Cluster 0 (MIMIC-IV). The mortality rate was higher among patients who received corticosteroids than among those who did not (p < .05). (B) Cluster 1 (MIMIC-IV). The mortality rate was higher among patients who received corticosteroids than among those who did not (p < .05). (C) Cluster 2 (MIMIC-IV). The mortality rate was lower among patients who received corticosteroids than among those who did not (p < .05). (D) Cluster 0 (eICU). The mortality rate was higher among patients who received corticosteroids than among those who did not (p < .05). (E) Cluster 1 (eICU). The mortality rate was non-significantly higher among patients who received corticosteroids than among those who did not (p > .05). (F) Cluster 2 (eICU). The mortality rate was lower among patients who received corticosteroids than among those who did not (p < .05). Please click here to view a larger version of this figure.
Table 1: Baseline characteristics of patients from the MIMIC-IV database. *p < .05. **p < .01. aUnless otherwise noted, data are expressed as medians (with interquartile ranges). Abbreviations: MIMIC-IV = Medical Information Mart for Intensive Care IV; BMI = body mass index; APSIII = Acute Physiology Score III; WBC = white blood cell count; ALT = alanine aminotransferase; ALT = aspartate aminotransferase; BUN = blood urea nitrogen; INR = international normalized ratio; PT = prothrombin time; pH = pondus hydrogenii; PO2 = partial pressure of oxygen; PCO2 = partial pressure of carbon dioxide; and FiO2 = fraction of inspired oxygen. Please click here to download this Table.
Table 2: Clinical characteristics and outcomes of three phenotypes in the MIMIC-IV derivation cohort. **p < .01. aUnless otherwise noted, data are expressed as medians (with interquartile ranges). Abbreviations: MIMIC-IV = Medical Information Mart for Intensive Care IV; APSIII = Acute Physiology Score III; BUN = blood urea nitrogen; FiO2 = fraction of inspired oxygen; and WBC = white blood cell count. Please click here to download this Table.
Supplementary Figure 1: Concept of this study. Please click here to download this File.
Supplementary Figure 2: Feature screening of the model for predicting pneumonia-associated ARDS. Abbreviations: ARDS = acute respiratory distress syndrome; APSIII = Acute Physiology Score III; PCO2 = partial pressure of carbon dioxide; max = maximum; BUN = blood urea nitrogen; min = minimum; pH = pondus hydrogenii; PO2 = partial pressure of oxygen; PaO2 = partial pressure of arterial oxygen; FiO2 = fraction of inspired oxygen; PT = prothrombin time; INR = international normalized ratio; BMI = body mass index; AST = aspartate aminotransferase; WBC = white blood cell; and ALT = alanine aminotransferase. Please click here to download this File.
Supplementary Figure 3: Validation of stacking machine learning methods of predicting pneumonia-associated ARDS in the eICU verification cohort. Receiver operating characteristic (ROC) curve of the stacking model. (Base learners were the logistic regression, naive Bayes, and random forest methods; the stacking learner was the decision tree method). Area under the ROC curve = 0.915. Abbreviations: ARDS = acute respiratory distress syndrome; and eICU = Telehealth Intensive Care Unit database. Please click here to download this File.
Supplementary Figure 4: Performance of the random forest model in predicting clinical subgroups of pneumonia-associated ARDS in the MIMIC-IV derivation cohort. Abbreviations: ARDS = acute respiratory distress syndrome; MIMIC-IV = Medical Information Mart for Intensive Care IV database; eICU = Telehealth Intensive Care Unit database; pred. cluster 0, prediction that a patient would be in cluster 0; pred. cluster 1, prediction that a patient would be in cluster 1; and pred. cluster 2, prediction that a patient would be in cluster 2. Please click here to download this File.
Supplementary Figure 5: Performance of the random forest model in predicting clinical subgroups of pneumonia-associated ARDS in the eICU validation cohort. Abbreviations: ARDS = acute respiratory distress syndrome; MIMIC-IV = Medical Information Mart for Intensive Care IV database; eICU = Telehealth Intensive Care Unit database; pred. cluster 0, prediction that a patient would be in cluster 0; pred. cluster 1, prediction that a patient would be in cluster 1; and pred. cluster 2, prediction that a patient would be in cluster 2. Please click here to download this File.
Supplementary Figure 6: Web-based system for the early diagnosis and subgroup classification of pneumonia-associated ARDS. Abbreviations: ARDS = acute respiratory distress syndrome; APSIII = Acute Physiology Score III; max = maximum; BUN = blood urea nitrogen; min = minimum; FiO2 = fraction of inspired oxygen; PaO2 = partial pressure of arterial oxygen; pH = pondus hydrogenii; PO2 = partial pressure of oxygen; and PCO2 = partial pressure of carbon dioxide. Please click here to download this File.
Supplementary Table 1: Baseline characteristics of patients in the eICU cohort. *p < .05. **p < .01. aUnless otherwise noted, data are expressed as medians (with interquartile ranges). Abbreviations: eICU = Telehealth Intensive Care Unit database; ARDS = acute respiratory distress syndrome; BMI = body mass index; APACHE IV = Acute Physiology and Chronic Health Evaluation IV; WBC = white blood cell; BUN = blood urea nitrogen; INR, international normalized ratio; PT = prothrombin time; pH = pondus hydrogenii; PO2 = partial pressure of oxygen; PCO2 = partial pressure of carbon dioxide; and FiO2 = fraction of inspired oxygen. Please click here to download this File.
Supplementary Table 2: Performance of the five machine learning methods for predicting pneumonia-associated ARDS. Abbreviation: ARDS = acute respiratory distress syndrome, and AUC = area under the receiver operating characteristic curve. Please click here to download this File.
Supplementary Table 3: Clinical characteristics and outcomes of three phenotypes in the eICU validation cohort. *p < .05. **p < .01. aUnless otherwise noted, data are expressed as medians (with interquartile ranges). Abbreviations: APACHE IV = Acute Physiology and Chronic Health Evaluation IV; BUN = blood urea nitrogen; FiO2 = fraction of inspired oxygen; and WBC = white blood cell. Please click here to download this File.
Supplementary Table 4: In-hospital mortality rates across phenotypes among patients in the MIMIC-IV derivation and eICU validated cohorts who did and did not receive early low- to medium-dose corticosteroid treatment. *p < .05 **p < .01. Abbreviations: MIMIC-IV = Medical Information Mart for Intensive Care IV database; eICU = Telehealth Intensive Care Unit database. Please click here to download this File.
As we know, this is the first diagnostic model and clinical subgroup classification model using machine learning to report ARDS in pneumonia patients, and the largest study to report the diagnosis and clinical subgroup classification of pneumonia-associated ARDS. In this study, we derived and validated two machine learning-based models and translated them into web-based applications for clinical practice and subsequent research. In the eICU validation cohort, the prediction of which patients with pneumonia would develop pneumonia-associated ARDS had an AUC of 0.915, 70.87% accuracy, 69.70% precision, and 69.70% recall. We also used simple and rapid clinical predictors to conduct clinical subgroup clustering for patients with pneumonia-associated ARDS. The three phenotypes had responded differently to early low- to medium-dose corticosteroid treatment.
The stacking method used in this study for machine learning was essentially a k-fold cross-validation model. The first layer of the model contained multiple basic classifiers, which provided the predicted results (meta-features) to the second layer (stacking model learner), and in the second-layer classifiers, the results of the first-layer classifiers were used for feature fitting and outputting the prediction results. In comparison with traditional analytical methods, the unique working principle of the stacking method is that it accounts for multifactorial diseases with complex relationships; moreover, it has good sensitivity and specificity13. In the diagnosis model, we identified 18 key predictors: APSIII/APACHE IV, FiO2, maximum anion gap, bicarbonate, glucose, potassium, sodium, and hemoglobin; minimum and maximum BUN, PO2/FiO2 ratio, pH, PCO2, and PO2. Most of these factors have been used in ARDS risk studies. Some researchers have proposed that disease score and glucose level after admission are closely related to the risk for ARDS14. Sodium and potassium electrolyte disorders, which may be related to the abnormal sodium and potassium channels in the lung epithelium of patients with ARDS, are more likely to be present in patients with ARDS than in those without ARDS15,16. The hallmark of ARDS is acute respiratory failure, and the blood gas profile is consistent with this condition; for example, decreased PO2 with or without increased PCO2, and abnormalities in pH, anion gap, and bicarbonate levels often indicate an acid-base imbalance, which is frequently accompanied by electrolyte imbalances. According to the Berlin definition of ARDS, the PO2/FiO2 ratio is a key factor in defining ARDS17, and a high FiO2 also indicates severe lung injury and increased oxygen demand. Abnormal hemoglobin levels may contribute to the development of acute lung injury and ARDS through several hypothesized mechanisms18. In addition to these indicators, we examined BUN level, which is the end product of nitrogen metabolism in humans and whose changes reflect renal function, nutritional status, and metabolic status. BUN levels were higher in patients with ARDS than in those without ARDS in these cohorts. In this study, these key predictors, in combination with machine learning, were used to accurately predict the development of ARDS in patients with pneumonia.
Another finding of this study is that patients with pneumonia-associated ARDS were divided into three clinical phenotypes. These phenotypes had different clinical characteristics; for example, patients in cluster 0 had the lowest rates of mortality, the lowest prevalence of sepsis, the lowest WBC count, and the lowest BUN levels; those in cluster 1 had the highest mortality rates, the highest prevalence of sepsis, the highest BUN levels, and more organ damage; and those in cluster 2 had moderate rates of mortality and the highest WBC counts. Of the patients with pneumonia-associated ARDS in both the MIMIC-IV and eICU cohorts, approximately 60% were in cluster 0, and approximately 40% were in clusters 1 and 2. These proportions were similar to those of patients with the biological phenotypes of hyperinflammatory and hypoinflammatory ARDS classified by Calfee et al.9. Approximately 40% of their patients were classified as the hyperinflammatory phenotype, and among those patients, the mortality rate was higher and illness was more severe. According to our classification, those patients can be further subdivided into clusters 1 and 2. In Calfee et al.'s study9, approximately 60% of the patients had hypoinflammatory ARDS, which was similar to the manifestations in cluster 0 patients, with relatively mild disease and a low mortality rate. Although numerous researchers have investigated the effects of corticosteroids in patients with ARDS, the results remain controversial19. The reasons for this controversy may include differences in treatment duration, initial corticosteroid dosage, and treatment regimen, but the main reason is the heterogeneity of ARDS. In this study, the rates of mortality were higher among patients in clusters 0 and 1 who received early treatment with low- to medium-dose corticosteroids than among those who did not receive corticosteroids, whereas the opposite result was found in cluster 2. Some studies have shown that patients with the hyperinflammatory phenotype of coronavirus disease 2019 (COVID-19) can also benefit from corticosteroid treatment20. Similarly, the patients in cluster 2 of this study were the most infected and were prone to cytokine storm and abnormal immune responses; early management of such immune disturbances may help prevent the progression of ARDS. Corticosteroid therapy for patients with severe ARDS, such as those in cluster 1, may suppress all working immune cells, including specific T and B cells that may control the activity of causative substances21; thus, corticosteroids may be less effective in such patients. In patients with milder ARDS, such as those in cluster 0, the response to early low- to medium-dose corticosteroids is also poor. Of interest is that Sinha et al.20 also found higher rates of mortality among patients with the hypoinflammatory phenotype of COVID-19 who received corticosteroids; their use may be associated with delayed clearance of infectious pathogens. At our institution, we therefore tend to try to predict the development of ARDS within 24 h after a patient with pneumonia is admitted, and we try to establish subgroup classification as early as possible if pneumonia-associated ARDS is indeed predicted. For patients in cluster 0 or 1, we tend not to administer low- to medium-dose corticosteroids in the early stages of hospitalization, whereas for patients in cluster 2, we do. Unlike previous ARDS phenotypic studies that mostly focused on the entire ARDS population the phenotypic classification in this study specifically targets pneumonia-associated ARDS -- this advantage allows it to better capture the unique clinical characteristics (e.g., infection severity-related WBC differences) and treatment response patterns (e.g., cluster 2's benefit from corticosteroids) of this subgroup, providing more direct guidance for precision management of pneumonia-induced ARDS.
At present, we have made a preliminary web transformation of the research results to facilitate bedside activities of health care workers and to promote an in-depth understanding of the heterogeneity of ARDS. In the future, the online system should be used for further clinical practice, continuous verification, and continuous optimization, and it should be promoted in different populations to facilitate precision medicine. Future work will conduct multicenter prospective studies to validate the model in real-time clinical settings and integrate the Web system with hospital electronic medical records (EMR) and clinical decision support systems (CDSS) for automatic data import and real-time prediction, enhancing clinical utility. The system will also undergo continuous optimization and promotion in diverse populations for precision medicine.
This study had several limitations. First, some of the predictors in this study, such as APACHE IV and WBC count, overlapped to a certain extent. However, the aim of this study was not to establish an independent association between risk factors and ARDS but rather to identify the combination of variables that best predicted the development of pneumonia-associated ARDS. Second, the study had a retrospective design, which has inherent limitations, such as selection bias, the presence of confounding factors, and the absence of certain data. Therefore, the results should be validated in randomized controlled trials. Third, machine learning models often yield results that are difficult to interpret, and the model itself does not provide information about the underlying mechanisms. While this study used clinical variables and k-means clustering for phenotyping and stacking for prediction, alternative strategies could further strengthen findings. For example, incorporating biological/genomic data (e.g., plasma IL-6, TNF-α, or transcriptomic profiles) could validate phenotypes by linking them to underlying biological mechanisms. Benchmarking the stacking model against traditional regression (e.g., multivariable logistic regression) would clarify machine learning's incremental value -- prior work notes machine learning's advantage in handling complex variable interactions, while regression offers better interpretability. Additionally, testing alternative clustering methods (e.g., hierarchical clustering, Gaussian mixture models) could help verify phenotype stability. Hierarchical clustering captures gradual subgroup differences, and Gaussian mixture models account for overlapping distributions.
Conclusions
We developed a Web-based system, available online, to predict the development of pneumonia-associated ARDS in patients with pneumonia. The early bedside diagnostic and clinical subgroup classification models for pneumonia-associated ARDS are included in this system. We identified three phenotypes of pneumonia-associated ARDS with different clinical characteristics and outcomes, all of which responded differently to treatment. These findings may assist clinicians in the diagnosis and treatment of the disease and may promote individualized precision medicine.
The authors declare that they have no competing interests.
None
| Apache Tomcat | Apache software foundation | Version 9.0.85 | |
| Eclipse IDE | Eclipse | 2023-09 | |
| Java Development Kit | Java | Version Java SE 8u381 | |
| RapidMiner Studio | Altair Engineering Inc. | Version 9.10.001 | |
| SPSS Statistics | IBM | Version 23.0 |
