$$\rightleftharpoonup{xx}$$
$$\longleftharp{xx}$$,
$$\longrightharp{xx}$$,
This study was approved by the Institutional Review Board of Shenzhen University General Hospital (IRB approval no. KYLL-2026-077-1). The requirement for written informed consent was waived due to the study's retrospective design.
1. Study cohort
Patients with acute ischemic stroke (AIS) who underwent multiparametric MRI were retrospectively identified from the institutional clinical database. The overall study workflow, including dataset partitioning, imaging model development, clinical model construction, and multimodal fusion, is illustrated in Figure 1.

Figure 1: Workflow of the multimodal outcome prediction framework. (A) Data acquisition. Patients with acute ischemic stroke (AIS) meeting predefined inclusion criteria were retrospectively identified. The dataset was divided into a training-validation cohort (n = 250), an internal independent test cohort (n = 50), and an external test cohort (n = 37). Stratified five-fold cross-validation was performed within the training-validation cohort. (B) Imaging model development. Multiparametric MRI sequences, including diffusion-weighted imaging (DWI), apparent diffusion coefficient (ADC), and T2 fluid-attenuated inversion recovery (T2-FLAIR), were processed using a hybrid 3-dimensional convolutional neural network-Vision Transformer (CNN-ViT) architecture for outcome prediction. (C) Clinical model development. Structured clinical variables were used to train machine learning models for prediction of 90-day functional outcome. (D) Multimodal fusion strategy. Predictions generated by the imaging and clinical models were integrated using a stacked logistic regression meta-learner to produce final outcome predictions. Please click here to view a larger version of this figure.
MRI examinations were performed using clinical MRI systems. A dedicated phased-array head coil was used for image acquisition. All sequences were acquired in the axial plane using consistent slice positioning across modalities. Diffusion-weighted imaging (DWI) was acquired using a single-shot spin-echo echo-planar imaging sequence with repetition time (TR) of 3,000–5,000 ms and echo time (TE) of 80–90 ms. Diffusion sensitization was applied using b-values of 0 and 800–1,000 s/mm2 in at least 3 orthogonal directions. The field of view ranged from 220–240 mm with a matrix size of 128 × 128. Slice thickness was 5–6 mm with an interslice gap of 1–1.5 mm. Two to four signal averages were acquired.
Apparent diffusion coefficient (ADC) maps were automatically generated from DWI data on the scanner workstation using mono-exponential fitting based on the acquired b-values. ADC values were calculated voxel-wise and exported for quantitative analysis. T2 fluid-attenuated inversion recovery (T2-FLAIR) images were acquired using an inversion recovery sequence with TR of 8,000–10,000 ms, TE of 80–140 ms, and inversion time of 2,200–2,600 ms. The field of view ranged from 220–240 mm with a matrix size of 192 × 192 to 256 × 256. Slice thickness was 4–5 mm with an interslice gap of 1–1.5 mm.
2. Data preprocessing
Clinical data were imported from structured spreadsheets containing case identifiers, mRS outcome scores, dataset split labels, and demographic and clinical variables. Case identifiers were standardized to ensure consistency with imaging filenames. Clinical variables included demographic characteristics, stroke severity measured using the National Institutes of Health Stroke Scale (NIHSS), vascular risk factors, and comorbid conditions recorded at admission.
The primary outcome was functional status at 90 days after stroke onset measured using the modified Rankin Scale (mRS). A favorable outcome was defined as mRS ≤ 2 and an unfavorable outcome as mRS > 2. The dataset was randomly divided into training-validation and internal independent test cohorts using stratified sampling based on mRS distribution to preserve outcome balance. An external test cohort was processed separately and was not used during model development. Within the training-validation cohort, stratified k-fold cross-validation was applied to maintain consistent outcome distributions across folds.
All DWI, ADC, and T2-FLAIR volumes were preprocessed before model training to ensure spatial and numerical consistency across modalities. The same preprocessing pipeline was applied to the training-validation, internal test, and external test cohorts without modification. Images were reoriented to canonical RAS orientation and converted to floating-point arrays. In-plane spatial resolution was resampled to 256 × 256 using linear interpolation. The through-plane dimension was standardized to 20 slices using a center-based strategy: volumes containing more than 20 slices were center-cropped, whereas volumes containing fewer than 20 slices were symmetrically zero-padded. The final volume size was 256 × 256 × 20.
Intensity normalization was performed independently for each volume using z-score normalization:

where x denotes voxel intensity, µ is the mean intensity of the volume, and σ is the standard deviation. If σ = 0, normalization was not applied to prevent numerical instability. Processed volumes were saved in NIfTI format with a standardized affine matrix for downstream deep learning analysis.
3. Clinical model development
Structured clinical variables were used to develop machine learning models for outcome prediction. Candidate predictors included demographic characteristics, vascular risk factors, stroke etiology, and baseline clinical severity measures. Continuous variables were imputed using median values and standardized. Categorical variables were imputed using the most frequent category and encoded using one-hot encoding.
Multiple machine learning algorithms were evaluated, including logistic regression, random forest, gradient boosting, support vector machine (SVM), extreme gradient boosting, and light gradient boosting machine models. Model development followed a stratified five-fold cross-validation framework within the training-validation cohort to estimate generalization performance. Final predictions for the internal and external test cohorts were generated by averaging predictions from models trained on each cross-validation fold. No external test samples were used during cross-validation or model selection.
4. Deep learning model architecture
A 3-dimensional CNN-ViT hybrid architecture was implemented for outcome prediction from multimodal MRI volumes. The network was designed to combine local spatial feature extraction with global contextual modeling within a unified framework. Input volumes consisted of multichannel 3-dimensional images processed end-to-end.
Feature extraction was initially performed using a hierarchical 3-dimensional convolutional backbone composed of 4 stages. Each stage included 2 convolutional layers with kernel size of 3 × 3 × 3 and padding of 1 voxel, followed by batch normalization and rectified linear unit activation. Spatial resolution was progressively reduced using 3-dimensional max-pooling layers applied after the first 3 stages, while feature-channel depth increased at each level to capture higher-level semantic representations. Dropout regularization (dropout rate = 0.1) was applied after the final convolutional stage to reduce overfitting. The convolutional backbone transformed the input volume of size C × D × H × W into a compact high-level feature representation with reduced spatial dimensions.
The resulting feature map was reshaped into a sequence of tokens by flattening spatial dimensions such that the number of tokens corresponded to:
N = D' x H' x W'
Global contextual relationships among tokens were modeled using Transformer encoder layers composed of multi-head self-attention and feedforward networks. Self-attention was computed as:

where Q, K, and V denote query, key, and value matrices, respectively, and d represents embedding dimension. Layer normalization and dropout were applied within each encoder layer to improve training stability. The Transformer module consisted of 3 encoder layers with 8 attention heads and an embedding dimension of 256.
After Transformer encoding, the representation corresponding to the classification token was extracted and normalized. A fully connected linear layer produced a single logit output for binary classification.
The proposed hybrid CNN-ViT architecture was intentionally designed as a lightweight and parameter-efficient model to balance representational capacity and overfitting risk. The model comprised 3.79 million trainable parameters (approximately 14.4 MB in fp32 precision), including 1.38 million in the convolutional backbone and 2.37 million in the Transformer encoder.
5. Imaging model training
Training of the imaging model was performed using a stratified five-fold cross-validation framework to preserve outcome distribution across folds while improving robustness of performance estimation. The dataset was divided into a training-validation cohort and an internal independent test cohort using stratified sampling based on outcome distribution. Within the training-validation cohort, stratified five-fold cross-validation was applied. For each fold, the imaging model was trained using the training subset and evaluated using the corresponding validation subset, whereas the held-out test cohort was reserved exclusively for final performance evaluation.
Model optimization was performed using a deep learning framework on a GPU-equipped workstation with the AdamW optimizer, learning rate of 3 × 10⁻5, and weight decay of 3 × 10⁻4. Training was conducted using a batch size of 8 for up to 200 epochs. Binary cross-entropy with logits was used as the loss function.
To address class imbalance, a positive-class weighting factor was computed for each fold based on the ratio of negative to positive samples and incorporated into the loss function. Gradient norm clipping with a maximum norm of 0.5 was applied to improve numerical stability during optimization. Automatic mixed-precision training was enabled to improve computational efficiency.
Early stopping was implemented when validation performance failed to improve by at least 1 × 10⁻4 over 30 consecutive epochs. The best-performing model checkpoint from each fold was retained. After completion of all folds, predictions for the internal and external test cohorts were generated using each fold-specific model, and final probabilities were obtained by averaging predictions across the 5 models to generate ensemble outputs.
6. Multimodal fusion model
A stacked fusion strategy was implemented to integrate imaging-derived predictions with structured clinical information. The deep learning imaging model and clinical prediction model served as base learners, and their predicted probabilities were used as input features for a logistic regression meta-learner. Additional interaction features, including the product and absolute difference of predicted probabilities, were incorporated to capture complementary information between imaging and clinical predictions.
To prevent information leakage, the meta-learner was trained using out-of-fold predicted probabilities generated from the training-validation cohort. Cross-validated out-of-fold probabilities from the imaging and clinical models were merged by patient identifier to construct the meta-learner training dataset. For the internal and external test cohorts, corresponding test-set probabilities from the imaging and clinical models were used as inputs to the trained meta-learner to generate fused probabilities. The trained meta-learner was applied to both test cohorts without refitting.
7. Ablation study
Ablation experiments were performed to assess the contributions of individual MRI sequences and the Vision Transformer component using identical training and evaluation settings as the primary model. Sequence ablations included single-sequence models, leave-one-sequence-out models, and the full multiparametric model. To evaluate the contribution of the Vision Transformer module, the proposed hybrid architecture was additionally compared with a CNN-only baseline in which the Transformer encoder was removed while preserving the same convolutional backbone.
8. Statistical analysis
Model performance was evaluated separately in the internal independent test cohort and external test cohort using area under the receiver operating characteristic curve (AUC) as the primary discrimination metric. Receiver operating characteristic (ROC) curves were constructed using predicted probabilities generated by each model. Sensitivity, specificity, and overall accuracy were additionally calculated to characterize classification performance.
Binary outcomes were generated using thresholds determined according to Youden’s index. Sensitivity, specificity, and accuracy were subsequently calculated at the optimal threshold. All statistical analyses, machine learning model development, and deep learning model training were implemented using standard scientific computing and machine learning software packages.