FedMediFormer-XAI: Federated Multimodal Transformers with Diffusion Augmentation and Graph-Based Drug Recommendation for Diabetes

Summary

This protocol proposes FedMediFormer-XAI, a multimodal diabetes intelligence framework that respects privacy through federated learning, multimodal transformers, diffusion-based augmentation, graph-based personalized drug recommendation, and explainable artificial intelligence, for accurate prediction, transparent decision-making, and personalized diabetes management.

Abstract

Diabetes management faces a number of obstacles, such as fragmented healthcare data, privacy concerns, poor explainability, and the lack of personalized therapeutic guidance. This research work introduces FedMediFormer-XAI, a unified and proper framework that incorporates federated learning, multimodal transformers, diffusion-based data augmentation, Graph Neural Networks (GNNs) for drug recommendation, and Explainable Artificial Intelligence (XAI) for diabetes intelligence. The system utilizes diverse healthcare data types, e.g., clinical records, population health indicators, continuous glucose monitoring data, retinal fundus images, wearable sensor measurements, and pharmacological information. To generate synthetic samples and address class imbalance, diffusion models are used, and multimodal transformer architectures are employed to learn relationships among very different data sources. Federated learning enables collaborative training of models in a privacy-preserving manner without sharing raw patient data. The GNN component captures patient-drug and drug-drug interactions for personalized drug recommendations. Explainability methods such as SHapley Additive exPlanations (SHAP), attention visualization, Integrated Gradients, and counterfactual reasoning give transparent interpretations of prediction and recommendation results. In the representative implementation, the proposed framework achieved an accuracy of 94.2%, precision of 93.1%, recall of 92.8%, F1-score of 92.9%, Matthews Correlation Coefficient (MCC) of 0.88, and an Area Under the Receiver Operating Characteristic Curve (AUC-ROC) of 0.96. The graph-based recommendation module achieved precision = 0.89, recall = 0.84, and Normalized Discounted Cumulative Gain (NDCG) = 0.91. The protocol provides a structured approach for integrating heterogeneous healthcare data using multimodal transformers, federated learning, diffusion-based augmentation, graph-based recommendation, and explainable artificial intelligence. The reported computational results demonstrate the framework's potential for diabetes prediction and personalized medication recommendation, while the federated design supports decentralized data handling. A prospective multicenter clinical evaluation is required to establish clinical utility, generalizability, and real-world applicability.

Introduction

Diabetes mellitus is among the top chronic metabolic diseases affecting humanity and poses a major public health issue as it keeps growing, with its long-term effects and the huge costs of treating patients^1,2. According to a report, there are currently over 537 million adults living with diabetes worldwide, and the figure is expected to rise significantly over the next few decades³. If high blood sugar levels are not controlled, the person will suffer from the worst complications, such as coronary heart disease⁴, kidney damage⁵, nerve damage⁶, eyesight loss⁷, and brain stroke⁸. Thus, recognizing the disease at the earliest stage, regularly monitoring the patient's condition, and providing him or her training through advanced medical approaches are necessary steps to help patients live healthier lives and, in doing so, reduce the expenses of the State's healthcare system^9,10.

Nowadays, the use of electronic health records (EHRs), wearable devices, continuous glucose monitoring (CGM) systems, retinal imaging technologies, and mobile health applications is spreading rapidly, generating large volumes of heterogeneous healthcare data that can be analyzed¹¹. These various data-collecting tools can provide diverse information about the human body, its ability to function at different stages of a disease, how the patient responds to different types of treatment, and how he or she behaves in real life¹². Artificial intelligence (AI) has proven to be the best solution for processing highly complex data, predicting diabetes, monitoring the disease, and assisting doctors with clinical decisions¹³. Furthermore, machine learning algorithms, including random forests, support vector machines, and gradient boosting methods, have given excellent results in assessing diabetes risk¹⁴. More recently, deep learning architectures have enabled automatic feature extraction and improved predictive performance across diverse healthcare applications¹⁵.

Using multimodal learning methods, diabetes intelligence has been greatly enhanced by merging structured clinical records, physiological measurements, retinal images, and behavioral indicators into a single predictive framework¹⁶. Using multiple sources of information, multimodal models can identify patterns that are not visible when each modality is examined separately¹⁷. Transformer-based models have recently learned long-range dependencies and contextual relationships from diverse types of healthcare data very effectively¹⁸. However, problems such as unbalanced classes, missing data, data heterogeneity, and limited generalizability still prevent the widespread use of multimodal AI systems in clinical settings¹⁹.

Even with these developments, many existing diabetes intelligence systems still rely on centralized learning frameworks that inherently require patient data from different institutions to be consolidated into a single repository²⁰. Methods like these are already leading to disputes over patient privacy, data custody, cybersecurity, and regulatory compliance²¹. In fact, due to legal and ethical considerations, healthcare providers are often constrained in their ability to share sensitive patient data²². In this context, federated learning is becoming a viable solution, as it enables training a collaborative model without sharing the actual patient data²³. Through decentralized optimization and secure parameter aggregation, federated learning enables privacy-protected health analysis by leveraging knowledge from geographically distributed datasets²⁴. Nevertheless, communication overhead, client heterogeneity, and convergence stability still represent major issues in federated healthcare scenarios²⁵.

One more drawback of the diabetes intelligence systems presently in use is the lack of transparency and support for personalized therapy. Many deep learning models behave like black boxes, which is a problem for clinicians who want to understand the reasoning behind predictions and recommendations²⁶. Explainable artificial intelligence (XAI) methods such as SHAP, Integrated Gradients, attention visualization, and counterfactual reasoning are proving to have great potential to enhance model transparency and clinician trust²⁷. On top of that, graph neural networks (GNNs) have become the most advanced tools for capturing patient-drug and drug-drug interactions, resulting in personalized drug recommendations and medication safety evaluation²⁸. New diffusion models have also introduced solutions for generating authentic synthetic healthcare data and addressing class imbalance²⁹.

Transformers, federated learning, diffusion-based synthetic data generation, graph neural networks, and explainable artificial intelligence have each demonstrated promising results in healthcare applications; however, their integration within a single reproducible diabetes intelligence framework remains limited. Existing approaches commonly address these components independently, focusing on privacy-preserving learning without personalized treatment recommendation, multimodal prediction without decentralized training, or explainability without graph-based clinical decision support. Although diffusion models can support class balancing and transformer architectures can learn relationships across heterogeneous healthcare modalities, standardized protocols for integrating these complementary technologies remain limited. Therefore, this work presents a unified methodological protocol that combines multimodal prediction, diffusion-based augmentation, federated optimization, graph-based personalized drug recommendation, and explainable artificial intelligence within a reproducible workflow, while providing a foundation for future clinical validation and translation.

Unlike studies that assume the availability of patient-level multimodal datasets, this protocol integrates heterogeneous public datasets without performing direct patient matching. Independent modality-specific models are trained for each data source, and latent feature embeddings are combined through a cross-modal attention mechanism. This design preserves methodological rigor while enabling multimodal knowledge integration across diverse healthcare repositories.

The proposed protocol is intended primarily for methodological development and distributed healthcare AI settings in which heterogeneous data sources are available across separate repositories or institutions and direct data centralization is restricted. In the current implementation, the modality-specific datasets are independent and are not patient-matched; therefore, multimodal integration is performed at the learned representation level rather than through direct patient-level correspondence. Practical application would require appropriately linked multimodal patient data, consistent preprocessing and outcome definitions across participating sites, suitable computational infrastructure, and compliance with applicable ethical, privacy, and regulatory requirements.

This paper introduces FedMediFormer-XAI, a federated multimodal transformer framework that enables privacy-preserving diabetes intelligence through prediction, recommendation, and explainability features under one roof. This framework integrates diffusion-based data augmentation to address class imbalance, multimodal transformer learning for diabetes prediction, federated learning for collaborative model development, graph neural network-based personalized drug recommendation, and explainable artificial intelligence methods to support clinical decisions in a transparent manner. The overall architecture and workflow of the proposed FedMediFormer-XAI framework are illustrated in Figure 1.

The major contributions of the proposed FedMediFormer-XAI framework are summarized as follows: (1) A unified protocol is presented that integrates federated learning, multimodal transformers, diffusion-based data augmentation, graph neural network-based personalized drug recommendation, and explainable artificial intelligence within a single reproducible workflow for diabetes intelligence. (2) A privacy-preserving federated learning strategy is developed to enable collaborative model training across simulated distributed healthcare clients without sharing raw patient data. (3) A multimodal transformer architecture is introduced to jointly learn complementary representations from structured clinical records, continuous glucose monitoring data, retinal fundus images, and population health indicators. (4) Diffusion-based synthetic data generation is incorporated to reduce class imbalance and improve model robustness while preserving the statistical characteristics of the original training data. (5) A GraphSAGE-based heterogeneous graph learning module is employed to model patient-drug and drug-drug relationships for personalized medication recommendation and interaction-aware treatment ranking. (6) Multiple explainable artificial intelligence techniques, including SHapley Additive exPlanations (SHAP), Integrated Gradients, attention visualization, and counterfactual explanations, are integrated to improve model transparency and provide interpretable prediction and recommendation output. (7) A comprehensive validation protocol is provided, including predictive evaluation, federated learning assessment, recommendation evaluation, explainability analysis, clinician-centered evaluation, external validation, and reproducibility assessment.

Protocol

All datasets used in this study were obtained from publicly available repositories and contained de-identified patient information. These included the Pima Indians Diabetes Dataset, CDC Diabetes Health Indicators Dataset, OhioT1DM Dataset, APTOS 2019 Blindness Detection Dataset, Drug Review Dataset, and Drug-Drug Interaction Dataset. No direct patient recruitment, intervention, specimen collection, or access to identifiable personal health information was involved. The study was limited to a secondary analysis of publicly available, anonymized datasets for methodological development and validation, and was conducted in accordance with institutional requirements for the use of such data. The human-centered evaluation described in this protocol is intended for future implementation and was not conducted in the present study. Researchers conducting this evaluation should obtain appropriate approval from the Institutional Ethics Committee, obtain written informed consent from all participants, and comply with applicable institutional requirements, data-use agreements, and national regulations.

1. Configure the computational environment

1. Install Python 3.11 as the primary programming environment. Install PyTorch 2.3 and TensorFlow 2.15 for deep learning model development and training. Install the Transformers library (v4.45) to implement transformer-based architectures.
2. Install PyTorch Geometric (v2.5) to support graph neural network construction and training. Install SHAP (v0.47) and Captum (v0.8) to perform explainability and feature attribution analyses.
3. Install NumPy, Pandas, and Scikit-learn for numerical computation, data preprocessing, and machine learning utilities. Install OpenCV and Matplotlib for image processing and visualization tasks. Install NetworkX to construct and analyze drug interaction graphs.
4. Install the CUDA Toolkit and compatible NVIDIA GPU drivers to enable hardware-accelerated computation. Set up an NVIDIA RTX 4090 GPU with at least 32 GB of system memory. Set the random seed to 42 for training, validation, and testing to ensure reproducible results. Check the installations and compatibility of software libraries in use. Record the software versions, hardware specifications, and model configuration parameters used for the analysis.

2. Prepare and preprocess datasets

1. Download and organize the required clinical, physiological, imaging, and pharmacological datasets into modality-specific repositories and verify their integrity. Confirm the dataset characteristics, prediction targets, and integration levels as specified in Tables 1 and 2. The workflow is illustrated in Figure 2.
2. Remove duplicate, invalid, or inconsistent records and records missing essential variables. Divide each applicable dataset into patient-independent training (70%), validation (15%), and testing (15%) subsets. Fit median imputation, Min-Max scaling, and categorical encoding using the training subset only, and apply the fitted transformations unchanged to the validation and testing subsets. Perform synthetic data generation only on the training subset. Verify partitioning and leakage prevention in accordance with Supplementary Table 1.
3. Import glucose, food intake, insulin dosage, physical activity, and sleep data. Standardize timestamps, align events to regular intervals, interpolate unrecorded glucose values linearly, divide continuous records into fixed-length windows, and apply z-score normalization to generate temporal transformer inputs.
4. Load the retinal fundus images and remove corrupted or unreadable files, duplicates, images with missing labels, and images with severe artifacts or insufficient retinal-field visibility.
5. Resize retained images to 224 × 224 pixels, normalize pixel intensities to [0, 1], and apply contrast-limited adaptive histogram equalization (CLAHE).
6. Augment training images using random rotation within ±15°, horizontal flipping with probability 0.5, random zooming within 0.9–1.1×, and brightness adjustment within ±20%. Do not apply stochastic augmentation to validation or testing images. Store the processed images for Vision Transformer training.
7. Integrate the modality-specific representations according to the dataset integration strategy summarized in Table 2. Do not perform direct patient-level matching across independent repositories when common patient identifiers are unavailable.
8. Store each public dataset as an independent modality-specific dataset because patient identifiers are not shared across repositories.
9. Train modality-specific feature extractors independently using the corresponding datasets: (a) Clinical records → TabTransformer encoder; (b) Population health indicators → TabTransformer encoder; (c) Continuous glucose monitoring → Temporal Transformer; (d) Retinal fundus images → Vision Transformer; (e) Pharmacological data → GraphSAGE module
10. Extract latent feature embeddings independently from each modality. Fuse only the learned latent embeddings via the proposed cross-modal attention fusion module, rather than merging raw patient records.
11. Ensure that no artificial patient matching is performed between independent public datasets. Maintain independent train, validation, and test partitions for each dataset throughout preprocessing, augmentation, and model development.
12. Store the fused multimodal feature representations for downstream prediction, federated learning, explainability, and personalized drug recommendation (Figure 2).

3. Perform diffusion-based data augmentation

1. Apply diffusion-based augmentation to the structured tabular clinical and population health datasets using TabDDPM. Generate synthetic samples exclusively from the preprocessed training partition to prevent information leakage. Configure the model according to Supplementary Table 2.
2. Train TabDDPM for a maximum of 500 epochs using the Adam optimizer (learning rate = 1 × 10⁻⁴; batch size = 256). Apply early stopping if validation loss does not improve by at least 0.001 for 20 consecutive epochs. Generate synthetic samples for underrepresented classes.
3. Evaluate synthetic data quality using Kolmogorov–Smirnov (KS) statistics, Jensen–Shannon divergence, Wasserstein distance, feature-wise correlation analysis, and downstream classification performance. Compare original and synthetic feature distributions using histogram overlap, kernel density estimation, and pairwise correlation coefficients.
4. Accept synthetic samples only when class-wise feature distributions remain consistent with the original training data (KS test, p > 0.05; Jensen–Shannon divergence < 0.10) and contain no unrealistic clinical values. Combine accepted samples with the original training dataset to generate a class-balanced dataset. The augmentation workflow is shown in Figure 3.
5. Validate the synthetic data by retraining the diabetes prediction model using the original and augmented training datasets. Compare Accuracy, F1-score, MCC, and AUC-ROC to assess changes in model generalization and distributional bias.

4. Develop the multimodal transformer model

Configure the multimodal transformer with modality-specific encoders for structured clinical data, CGM sequences, and retinal fundus images, as illustrated in Figure 4. Integrate the resulting representations using cross-modal attention for diabetes prediction and graph-based personalized drug recommendation.

1. Develop the Clinical Encoder
   1. Initialize the TabTransformer encoder architecture. Input normalized clinical variables and population health indicators. Generate contextual feature representations using transformer attention mechanisms.
   2. Configure the TabTransformer encoder with 32 structured clinical input variables, an embedding dimension of 128, four transformer layers, eight attention heads, a feed-forward dimension of 512, a dropout rate of 0.20, and Gaussian Error Linear Unit (GELU) activation.
   3. Learn latent clinical embeddings that capture relationships among patient attributes. Store the resulting clinical embeddings for multimodal integration.
2. Develop the Continuous Glucose Monitoring Encoder
   1. Initialize the temporal transformer encoder. Feed continuous glucose monitoring sequences and physiological measurements into it. Use self-attention mechanisms to encode temporal dependencies.
   2. Configure the temporal transformer with a sequence length of 96 time steps, a sampling interval of 15 min, an embedding dimension of 128, four transformer layers, eight attention heads, sinusoidal positional encoding, and a dropout rate of 0.20.
   3. Produce sequential embeddings that reflect the patient's glucose dynamics. Keep the temporal embeddings ready for the next fusion process.
3. Develop the Vision Transformer Encoder
   1. Initialize the Vision Transformer architecture. Input preprocessed retinal fundus images. Divide images into fixed-size patches.
   2. Configure the Vision Transformer with an input image resolution of 224 × 224 pixels, a patch size of 16 × 16 pixels, an embedding dimension of 768, 12 transformer blocks, 12 attention heads, and a dropout rate of 0.10.
   3. Generate patch-level feature representations. Acquire visual features that signify retinal pathologies and disease-related biomarkers. Keep image embeddings for multimodal fusion.
4. Perform Cross-Modal Fusion
   1. Extract the latent representations from the clinical, CGM, and Vision Transformer encoders and project each into a common 256-dimensional latent space using independent linear projections followed by Layer Normalization.
   2. Concatenate the projected modality tokens, add learnable modality-type embeddings, and apply multi-head cross-modal attention to enable information exchange among the clinical, CGM, and retinal representations. Configure the attention block with eight heads, a 256-dimensional model space, a 1,024-dimensional feed-forward layer, GELU activation, 0.10 dropout, residual connections, and Layer Normalization.
   3. Mean-pool each modality-specific token group and concatenate the resulting representations. Project the concatenated 768-dimensional vector to a 512-dimensional unified multimodal representation and retain the attention weights and fused representations for subsequent explainability and ablation analyses.
   4. Forward the unified representation to the diabetes prediction and GraphSAGE-based drug recommendation modules. Maintain the original modality-specific training, validation, and testing partitions throughout fusion without artificial patient-level matching across the independent datasets.
   5. Configure the multimodal transformer according to the model parameters summarized in Supplementary Table 3.

5. Configure federated learning

1. Configure the federated learning framework to enable collaborative model training across geographically distributed healthcare institutions without exchanging patient-level data. Perform local model optimization at each participating institution using locally available data. Share only protected model parameters with the central aggregation server. Preserve patient privacy while enabling collaborative development of multimodal models.
2. Separate the multimodal datasets and distribute them among multiple federated clients to mimic the scenario of different healthcare institutions collaborating without sharing their data.
3. Configure the federated learning network with 10 participating hospital clients, one central aggregation server, five local epochs per communication round, 100 communication rounds, an 80% client participation fraction, random client selection at each communication round, and a minimum of eight participating clients per round.
4. Give each client a dataset that belongs to a particular institution so that the data remains local and confidential. Partition the multimodal training data among 10 simulated hospital clients using a Dirichlet-based non-independent and identically distributed (non-IID) strategy to generate heterogeneous class distributions, demographic compositions, and client dataset sizes.
5. Use a fixed random seed to ensure reproducibility. Verify the resulting client-level distributions and retain these partitions throughout federated training. Keep the validation and testing datasets independent of the client partitions.
6. Initialize each local client with the global model parameters received from the central aggregation server. Train the local multimodal transformer for five epochs using the client's local training data. Optimize the local model using the AdamW optimizer with a learning rate of 1 × 10⁻⁴.
7. Compute the local model parameter updates after completion of local training. Protect the local model parameters before transmission to the central aggregation server. Transmit the protected local model parameters to the central aggregation server for global aggregation.
8. Receive the protected model parameters from all participating hospital clients. Verify the integrity of the received model updates before aggregation. Aggregate the local model weights using the Federated Averaging (FedAvg) algorithm. Update the global multimodal transformer using the aggregated model parameters.
9. Redistribute the updated global model parameters to the participating hospital clients. Repeat the local training and global aggregation procedures until 100 communication rounds are completed or the predefined convergence criterion is satisfied.
10. Protect transmitted model parameters using the specified secure aggregation mechanism and AES-256 encryption. Authenticate participating clients using digital certificates.
11. Establish encrypted communication channels using Transport Layer Security (TLS) 1.3. Retain only the aggregated global model parameters after completion of each communication round.
12. Evaluate privacy and security using raw-data exposure, unauthorized sharing, membership inference, and model inversion assessments. For membership inference, compare attack AUC-ROC with the random baseline (0.50); assess inversion using structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR) for retinal images and normalized reconstruction error for numerical features.
13. Verify secure aggregation by confirming that the server cannot access individual unaggregated client updates. Verify AES-256/TLS 1.3 protection and confirm that no unencrypted model-update transmissions occur.
14. Compare privacy metrics between federated and centralized configurations where applicable and report the metrics specified in Supplementary Table 4.
15. Estimate the computational complexity of local transformer training as O(N × L × d²), where N represents the number of samples, L represents the number of transformer layers, and d represents the embedding dimension.
16. Estimate the computational complexity of federated aggregation as O(K × P) per communication round, where K represents the number of participating clients and P represents the number of trainable model parameters.
17. Exchange only protected model parameters between the participating clients and the central aggregation server during each communication round. Calculate the communication overhead based on the model size, the number of participating clients, and the number of communication rounds.
18. Compare the parameter-exchange overhead with the estimated cost of transferring the corresponding multimodal healthcare datasets. Configure the federated learning framework according to the parameters summarized in Supplementary Table 4. The federated learning workflow and parameter aggregation process are illustrated in Figure 5.

6. Construct the personalized drug recommendation module

1. Import the pharmacological, medication, patient treatment, and drug-interaction data into the computational environment. Construct a heterogeneous healthcare graph containing patient, drug, disease, laboratory test, and clinical risk-factor nodes.
2. Define patient-disease, patient-drug, disease-drug, drug-drug interaction, and patient-laboratory relationships as graph edges. Verify the graph structure and remove duplicate, invalid, or disconnected relationships before model training.
3. Represent patient nodes using age, sex, body mass index (BMI), HbA1c, blood glucose, blood pressure, and diabetes duration. Represent drug nodes using drug class, mechanism of action, dosage, administration frequency, and known adverse effects.
4. Represent disease nodes using disease severity, disease stage, and associated complications. Encode categorical attributes and normalize continuous node features before GraphSAGE training.
5. Initialize the node embeddings using the preprocessed node features. Sample up to 25 neighboring nodes for each target node during neighborhood aggregation. Aggregate neighboring node representations using mean aggregation. Update the target-node embeddings using the aggregated neighborhood representations.
6. Apply the rectified linear unit (ReLU) activation function after each GraphSAGE layer. Normalize the resulting node embeddings. Repeat the neighborhood aggregation procedure across three GraphSAGE layers.
7. Configure the GraphSAGE model with a hidden dimension of 256, an embedding dimension of 128, three GraphSAGE layers, a neighbor-sampling size of 25, a dropout rate of 0.30, a learning rate of 1 × 10⁻⁴, a batch size of 512, and a maximum of 100 training epochs. Optimize the model using the Adam optimizer.
8. Compute the patient-specific embedding from the trained GraphSAGE model. Compute embeddings for candidate medications. Calculate suitability scores between the patient representation and candidate drug representations. Rank candidate medications according to their predicted suitability scores.
9. Identify and remove contraindicated or potentially unsafe medications using available drug-drug interaction information. Return the five highest-ranked eligible medications as the personalized Top-5 recommendations.
10. Train the recommendation model using Bayesian Personalized Ranking (BPR) loss. Optimize the ranking objective to assign higher scores to clinically appropriate medications than to less suitable candidate medications.
11. Identify influential neighboring nodes and clinically relevant graph relationships contributing to each medication recommendation. Calculate feature-attribution scores for patient, disease, laboratory, and medication-related characteristics contributing to the recommendation.
12. Generate visual graph-based explanations showing the relationships that contribute to each personalized medication recommendation. Configure the GraphSAGE recommendation model according to the parameters summarized in Supplementary Table 5. The graph-based personalized drug recommendation workflow is illustrated in Figure 6.

7. Perform explainability assessment

1. Perform SHAP analysis
   1. Load the trained multimodal transformer and the held-out test dataset into the computational environment.
   2. Initialize the SHAP explainer using the trained prediction model and a background sample of 100 training samples, selected exclusively from the training dataset using stratified random sampling according to the diabetes outcome class. Maintain the approximate class distribution of the complete training partition in the background sample, and use a fixed random seed of 42 to ensure reproducible sample selection. Compute SHAP values for the selected test samples.
   3. Calculate global feature importance from the absolute SHAP values, generate global and local SHAP visualizations, and retain the calculated values for subsequent quantitative and statistical analyses.
2. Perform integrated gradients analysis
   1. Select representative correctly classified and misclassified samples from the held-out test dataset. Define modality-specific baseline inputs before calculating Integrated Gradients. Use a zero baseline for normalized clinical and CGM numerical features, representing zero normalized feature contribution, and use a zero-valued baseline image for the normalized retinal-image input.
   2. Compute Integrated Gradients attribution scores using 100 interpolation steps between the baseline and the original input. Normalize the resulting attribution scores to enable comparison across features and modalities. Generate attribution visualizations to identify influential clinical, temporal, and imaging features contributing to individual model predictions.
3. Perform attention visualization
   1. Extract attention matrices from the trained transformer layers for representative test samples. Calculate the mean attention weights across the corresponding attention heads. Generate attention heatmaps to visualize cross-modal interactions among clinical, temporal, and retinal imaging representations. Identify the dominant clinical, temporal, and retinal features receiving high attention weights during prediction.
   2. Extract the attention weights from the trained multimodal transformer for a randomly selected held-out test sample and visualize the corresponding clinical-feature, CGM temporal, retinal spatial, and cross-modal attention distributions.
4. Generate counterfactual explanations
   1. Select representative patient records from the held-out test dataset. Define clinically permissible ranges and constraints for modifiable patient features before generating counterfactual samples. Restrict continuous features to their observed training-data range, and restrict categorical features to valid categories observed in the training data. Do not modify immutable demographic characteristics, including age and sex.
   2. Generate counterfactual samples by minimally modifying only the permitted modifiable features while preserving the predefined clinical and feature-domain constraints. Reject counterfactuals containing values outside the observed training-data range, invalid categorical states, or modifications to immutable features. Identify the smallest clinically plausible feature modifications required to alter the predicted outcome.
   3. Report the modified features, original values, counterfactual values, and corresponding changes in model predictions. The explainability assessment workflow is illustrated in Figure 7, while representative attention visualization and counterfactual explanation outputs are presented in Supplementary Figure 1.
5. Evaluate explanation quality
   1. Calculate fidelity to quantify the agreement between the generated explanations and the behavior of the prediction model. Calculate stability by repeatedly generating explanations under small input perturbations and comparing the resulting attribution patterns. Evaluate consistency by comparing explanations generated for clinically similar test samples.
   2. Calculate sparsity by determining the number or proportion of features contributing substantially to each explanation. Evaluate completeness to determine whether the attributed features adequately account for the corresponding model output. Evaluate sensitivity by measuring changes in attribution scores following controlled perturbations of the input features.
   3. Calculate infidelity to quantify disagreement between feature attributions and observed changes in model predictions. Record all quantitative explainability metrics for subsequent statistical analysis.
6. Perform prospective clinician validation
   1. For future prospective clinical validation, obtain approval from the appropriate Institutional Ethics Committee before recruiting clinical experts. After obtaining ethical approval, recruit 12 clinicians comprising six endocrinologists, three diabetes specialists, and three clinical pharmacologists, and obtain informed consent from all participants before initiating the evaluation.
   2. Randomly select representative model predictions and their corresponding explanations for clinician assessment. Present the selected explanations independently to each participating clinician using a standardized evaluation format.
   3. Ask each clinician to evaluate the explanation's usefulness, clinical relevance, interpretability, and trustworthiness using a five-point Likert scale. Record anonymized clinician ratings and calculate inter-rater agreement and the corresponding statistical measures as specified in Step 7.7.
7. Perform statistical analysis
   1. Assess the distribution of the quantitative evaluation scores before selecting the statistical comparison test. Apply a paired t-test to normally distributed paired measurements or the Wilcoxon signed-rank test to non-normally distributed paired measurements, as appropriate.
   2. Calculate Fleiss' kappa to quantify inter-rater agreement among the participating clinicians. Calculate 95% confidence intervals for the principal explainability and clinician-evaluation metrics. Set the statistical significance threshold at p < 0.05.
8. Report the calculated test statistics, confidence intervals, and p-values with the corresponding evaluation results. Evaluate and report the explainability metrics according to the criteria summarized in Supplementary Table 6.

8. Evaluate model performance

1. Compare FedMediFormer-XAI with Logistic Regression, Random Forest, XGBoost, TabTransformer, Vision Transformer, Temporal Transformer, and the centralized multimodal transformer using accuracy, precision, recall, F1-score, MCC, and AUC-ROC.
2. Perform each baseline evaluation using the same predefined training, validation, and testing strategy to ensure a consistent comparison with the proposed framework.
3. Repeat the model evaluation across independent experimental runs and record the performance metrics obtained in each run. Calculate the mean, standard deviation (SD), and 95% confidence interval (CI) for Accuracy, Precision, Recall, F1-score, MCC, and AUC-ROC. Report the performance results as mean ± SD together with the corresponding 95% CI.
4. Assess the normality of the paired performance differences between FedMediFormer-XAI and each baseline model. Apply a paired t-test to normally distributed paired measurements and the Wilcoxon signed-rank test to non-normally distributed paired measurements. Set the statistical significance threshold at p < 0.05.
5. Report the corresponding test statistics, p-values, and 95% confidence intervals to quantify the statistical significance and uncertainty of the observed performance differences.
6. Get a count of all communication rounds until the models finally converge. Keep an eye on how the global model performs during federated training. Study how convergence changes over time in communication rounds.
7. Check how well parameter aggregation works. Examine how decentralized learning influences predictive performance. Make a comparison between the results from federated learning and centralized learning.
8. Assess federated convergence across communication rounds and compare federated and centralized predictive performance. Evaluate drug recommendations using Precision@5, Recall@5, and NDCG@5.
9. Generate negative samples–drug samples from the eligible candidate-drug pool by selecting medications that are not observed as positive patient–drug interactions for the corresponding patient. Exclude all known positive interactions and all contraindicated medications from the negative-sampling pool. Maintain a fixed negative-to-positive sampling ratio of 1:1 for recommendation training and use a fixed random seed of 42 for reproducible negative-sample selection.
10. Establish the relevant item set for each patient exclusively from the held-out evaluation data. Define the relevant-item set Ru​ for patient u as the set of medications that satisfy the positive patient–drug criteria in the held-out ground-truth records. Do not use model predictions to construct Ru. Remove contraindicated medications from the candidate recommendation set before ranking.
11. Generate the five highest-ranked eligible medications for each evaluation patient. Calculate Precision@5 as the proportion of the five recommended medications that belong to the patient's relevant-item set Ru. Calculate Recall@5 as the proportion of the patient's relevant medications retrieved among the five recommendations. Calculate NDCG@5 using the graded relevance of the recommended medications, assigning higher relevance to clinically appropriate medications and zero relevance to non-relevant medications. Aggregate the metrics across all evaluation patients and report the mean Precision@5, Recall@5, and NDCG@5.
12. Construct the recommendation ground truth before model evaluation and keep the held-out patient- drug interactions separate from the recommendation-training data. Do not use test-set medication interactions, test-set labels, or future treatment information during GraphSAGE training, negative sampling, or candidate ranking.
13. Measure Fidelity to check how well the explanations match the model's behavior. Assess the explanation for stability through multiple examinations. Judge Comprehensibility based on doctor-oriented criteria.
14. Investigate how consistent explanations are across different types of data. Confirm the clinical value of the explanations produced. The evaluation metrics used throughout the study are summarized in Supplementary Table 7.

9. Perform prospective clinician validation

1. Conduct prospective clinician validation after obtaining appropriate ethics approval and informed consent, following the recruitment, evaluation, data-collection, and statistical-analysis procedures described in Steps 7.6–7.8. The prospective clinician-validation design, including participant composition, evaluation criteria, comparison conditions, and planned statistical analyses, is summarized in Supplementary Table 8.

10. Perform validation and reproducibility assessment

1. Conduct ablation studies by systematically removing individual components from the complete FedMediFormer-XAI framework while maintaining the same dataset partitions, preprocessing procedures, training settings, and evaluation criteria. Evaluate the full model and configurations without diffusion augmentation, federated learning, GraphSAGE-based drug recommendation, and multimodal fusion.
2. Compare the ablated configurations with the complete FedMediFormer-XAI framework using Accuracy, Precision, Recall, F1-score, MCC, and AUC-ROC. Quantify the change in each performance metric to determine the contribution of each framework component. Compare validation results with internal testing outcomes to evaluate generalizability. Perform statistical significance testing to determine the reliability of observed performance differences.
3. Assess the generalizability of the framework using an independent external dataset when such a dataset with compatible input variables and outcome definitions is available. Apply the same preprocessing and evaluation procedures used for the internal test dataset, and compare the resulting performance metrics with the internal test results.
4. Perform statistical significance analysis across repeated experimental runs. Assess the normality of paired performance differences before statistical testing. Apply a paired t-test to normally distributed paired measurements and the Wilcoxon signed-rank test to non-normally distributed paired measurements. Set the statistical significance threshold at p < 0.05.
5. Calculate the mean and 95% confidence interval for Accuracy, Precision, Recall, F1-score, MCC, and AUC-ROC. Report the corresponding test statistics, p-values, and confidence intervals for the principal model comparisons. Document all model parameters, preprocessing procedures, training configurations, and evaluation protocols required to reproduce the reported experiments.
6. Preserve the source code, configuration files, evaluation scripts, and trained-model specifications used for the reported experiments. Maintain detailed records of model training, experimental settings, random seeds, and evaluation results.
7. Verify the completeness and consistency of the available reproducibility materials. Ensure that the documented methods and materials provide sufficient information for independent reproduction of the experimental workflow.
8. Summarize the reproducibility resources and validation procedures used in the study in Supplementary Table 9. Document the ablation-study procedure and evaluation criteria for assessing the contribution of the major framework components.

Results

Predictive performance analysis
FedMediFormer-XAI demonstrated improved predictive performance compared with the evaluated unimodal and conventional baseline models. Diffusion-based augmentation improved minority-class representation, and the resulting predictive performance is summarized in Table 3. The repeated-run evaluation demonstrated stable predictive performance across the evaluated models. FedMediFormer-XAI achieved the highest overall performance, with an accuracy of 94.20 ± 0.34% (95% CI: 93.78–94.62), precision of 93.10 ± 0.30% (95% CI: 92.73–93.47), recall of 92.80 ± 0.28% (95% CI: 92.45–93.15), and F1-score of 92.90 ± 0.28% (95% CI: 92.55–93.25). The model achieved an MCC of 0.88 ± 0.02 (95% CI: 0.86–0.90) and AUC-ROC of 0.96 ± 0.01 (95% CI: 0.95–0.97). The centralized multimodal transformer provided the next-best overall performance, whereas the unimodal and conventional machine-learning models showed comparatively lower predictive performance. These results indicate that multimodal representation learning, combined with federated optimization, yields improved and more stable diabetes classification performance.

Ablation study
Component-wise ablation was used to assess the contribution of diffusion augmentation, federated learning, GraphSAGE-based drug recommendation, and multimodal fusion. The complete FedMediFormer-XAI framework achieved an accuracy of 94.20 ± 0.34%, precision of 93.10 ± 0.30%, recall of 92.80 ± 0.28%, F1-score of 92.90 ± 0.28%, MCC of 0.88 ± 0.02, and AUC-ROC of 0.96 ± 0.01 across five independent runs. Removing diffusion augmentation reduced accuracy to 91.80 ± 0.42%, corresponding to a 2.40 percentage-point decrease, while the F1-score and AUC-ROC decreased to 90.30 ± 0.38% and 0.94 ± 0.01, respectively. This reduction indicates the contribution of diffusion-based augmentation to minority-class representation and predictive robustness.

Removing federated learning resulted in an accuracy of 93.10 ± 0.37%, representing a 1.10 percentage-point decrease relative to the complete framework. Precision, recall, F1-score, MCC, and AUC-ROC were also reduced to 92.20 ± 0.34%, 91.80 ± 0.32%, 92.00 ± 0.33%, 0.86 ± 0.02, and 0.95 ± 0.01, respectively. This comparison demonstrates the contribution of the federated configuration to predictive performance.

Removing the GraphSAGE-based personalized drug-recommendation component resulted in a comparatively small change in diabetes prediction performance, with accuracy decreasing to 93.80 ± 0.35% and F1-score to 92.60 ± 0.30%. The corresponding MCC and AUC-ROC were 0.87 ± 0.02 and 0.96 ± 0.01, respectively. This result indicates that GraphSAGE primarily contributes to personalized medication recommendation rather than directly determining diabetes classification performance.

The largest reduction was observed when multimodal fusion was removed. Accuracy decreased to 87.60 ± 0.55%, representing a 6.60 percentage-point decrease, while precision, recall, F1-score, MCC, and AUC-ROC decreased to 86.80 ± 0.51%, 85.90 ± 0.54%, 86.30 ± 0.52%, 0.76 ± 0.03, and 0.91 ± 0.01, respectively. This finding demonstrates the importance of jointly modeling complementary information from clinical, CGM, and retinal modalities.

Federated learning analysis
The federated learning framework enabled collaborative model training across distributed clients while maintaining decentralized handling of raw patient data. During training, each client's local model was fine-tuned individually and combined via the Federated Averaging method. After 100 communication rounds, the global model converged, and its predictive performance remained consistent with centralized learning. The federated learning framework demonstrated stable convergence across communication rounds despite heterogeneous client data distributions. Protected parameter exchange preserved decentralized data handling, while the global model maintained competitive predictive performance relative to the centralized baseline. Table 4 compares the centralized and federated implementations. Federated learning required additional training time due to communication overhead, while maintaining predictive performance comparable to centralized learning.

Dataset-Level Performance Analysis
To evaluate generalizability across different healthcare modalities, we assessed performance on each dataset separately. The multimodal FedMediFormer-XAI model demonstrated strong and generally competitive performance across the evaluated datasets. It achieved an accuracy of 91.8%, 93.1%, 92.4%, and 94.2% on the Pima Indians Diabetes, CDC Diabetes Health Indicators, OhioT1DM, and APTOS 2019 datasets, respectively. Although the APTOS 2019 dataset achieved slightly higher accuracy (94.7%) and precision (93.9%) than the multimodal model, the proposed multimodal approach maintained competitive performance across all datasets and achieved an AUC-ROC of 0.96, comparable to the highest dataset-level AUC-ROC. The overall results at the dataset level are presented in Table 5. For single datasets, retinal image analysis achieved the best performance when used alone, whereas multimodal fusion yielded the most stable and well-balanced predictions.

Personalized drug recommendation analysis
The graph neural network-based recommendation component was evaluated using medication-effectiveness information and drug-interaction data, with candidate medications ranked according to learned patient-drug suitability scores and contraindicated combinations excluded during recommendation generation. The use of the heterogeneous graph format and the modeling of patient-drug and drug-drug interactions could be done thoroughly, thereby supporting personalized medication choices and safety assessment. The GraphSAGE recommendation model effectively captured complex relationships among patients, diseases, laboratory findings, and medications. Personalized recommendations demonstrated high ranking performance while maintaining clinically meaningful explanations through graph neighborhood analysis and feature attribution.

According to Table 6, the personalized drug recommendation module was evaluated using Precision@5, Recall@5, and NDCG@5. These metrics quantify the relevance and ranking quality of the top-five personalized medication recommendations generated by the GraphSAGE-based recommendation module. The recommendation system achieved strong ranking performance, with precision = 0.89, recall = 0.84, and NDCG = 0.91.

Explainability analysis
The framework incorporates SHAP, Integrated Gradients, attention visualization, and counterfactual explanations to support the interpretation of multimodal predictions. SHAP was used to quantify feature-level contributions, Integrated Gradients to attribute predictions to input features, attention visualization to examine model focus across modalities, and counterfactual explanations to identify feature changes associated with alternative predictions. Figure 8 presents a representative model-derived SHAP summary plot generated from the trained FedMediFormer-XAI model, whereas Figure 9 presents representative attention visualizations extracted from the trained transformer. These figures represent outputs obtained from model evaluation rather than schematic illustrations of the proposed architecture.

In Figure 8, the aggregate-level rankings of feature importance are presented. Features with higher absolute SHAP values had a greater influence on model predictions and also aligned well with existing clinical knowledge.

Figure 9 presents representative attention visualizations extracted directly from the trained FedMediFormer-XAI model for a randomly selected held-out test sample. The visualizations include clinical feature attention, CGM temporal attention, retinal spatial attention, and cross-modal attention between the three modalities. Attention visualization further demonstrated the model's learned allocation of attention across multiple modalities. The selected sample showed relatively closer attention to HbA1c, diabetes duration, and age among the clinical features, while the CGM attention map highlighted several periods with prominent glucose variability. The retinal attention map identified specific image regions contributing to the model representation, and the cross-modal attention matrix demonstrated both intra-modal and cross-modal attention patterns. As shown in Figure 9, the representative model-derived attention maps highlight selected temporal glucose patterns, influential clinical variables, and diagnostically relevant retinal image regions in a held-out test sample.

Human-centered evaluation results
A prospective human-centered evaluation protocol was designed to assess clinician trust, interpretability, usability, clinical utility, and explanation relevance under prediction-only and prediction-plus-explanation conditions. The proposed evaluation will involve endocrinologists, diabetes specialists, and clinical pharmacologists, with appropriate approval from the Institutional Ethics Committee and informed consent. Because this evaluation is intended for future prospective implementation, clinician-level outcome scores are not reported in the present protocol.

Table 7 summarizes the proposed prospective clinician-evaluation design and the predefined criteria for assessing the effects of explanations on clinician trust, interpretability, and perceived utility. The prospective evaluation will compare clinician assessments of model predictions with and without accompanying explanations using predefined Likert-scale criteria. The proposed prospective human-centered explainability evaluation framework is presented in Figure 10

Figure 1: Overall FedMediFormer-XAI framework architecture. Schematic overview of the FedMediFormer-XAI framework, showing multimodal data acquisition and preprocessing, diffusion-based augmentation, modality-specific transformer learning, federated model training, GraphSAGE-based personalized drug recommendation, and explainability analysis. The framework generates diabetes prediction, personalized medication recommendations, and interpretable model outputs. Please click here to view a larger version of this figure.

Figure 2: Multimodal dataset acquisition and preprocessing pipeline. Schematic workflow for acquiring and preprocessing the multimodal datasets used in FedMediFormer-XAI. Clinical and population health data, CGM records, retinal images, and pharmacological information undergo modality-specific quality control, preprocessing, normalization, encoding, and augmentation before being converted into model-ready representations for downstream analysis. Please click here to view a larger version of this figure.

Figure 3: Diffusion-based data augmentation workflow. Schematic workflow for diffusion-based augmentation of structured tabular training data using TabDDPM. Generated samples undergo quality and distribution-similarity assessments before being integrated with the original training data to improve class balance. Please click here to view a larger version of this figure.

Figure 4: Architecture of the proposed multimodal transformer framework. Schematic architecture comprising (A) a TabTransformer encoder for clinical data, (B) a temporal transformer encoder for CGM data, and (C) a Vision Transformer encoder for retinal images. (D) Modality-specific representations are integrated through cross-modal attention to generate a unified multimodal representation. (E) Task-specific prediction heads generate model outputs. (F) Regularization and optimization components support model training, and (G) classification, regression, and cross-modal consistency losses guide the optimization process. The resulting multimodal representation supports downstream prediction, recommendation, and explainability tasks. Please click here to view a larger version of this figure.

Figure 5: Federated learning communication framework. Schematic workflow for federated training across distributed hospital clients. Local multimodal models are trained using client-specific data, and protected model updates are transmitted to a central server for Federated Averaging. The aggregated global model is redistributed across clients for subsequent communication rounds. The framework also illustrates secure parameter exchange and evaluation of privacy, communication, and computational requirements. Please click here to view a larger version of this figure.

Figure 6: Graph-based personalized drug recommendation workflow. Schematic workflow for GraphSAGE-based personalized drug recommendation. Pharmacological and patient-representation data are organized into a heterogeneous graph that includes relevant healthcare entities and relationships. GraphSAGE learns patient and drug representations, which are used to calculate patient–drug suitability scores and rank candidate medications. Contraindicated or unsafe drug combinations are filtered before the generation of the final Top-K recommendations, with graph-based information supporting the interpretation of the recommendations. Please click here to view a larger version of this figure.

Figure 7: Explainability assessment framework. Schematic overview of the explainability workflow. (A) SHAP analysis evaluates global and local feature contributions; (B) Integrated Gradients provides attribution-based explanations; (C) attention visualization identifies influential features and modality interactions; and (D) counterfactual analysis identifies feature changes associated with altered predictions. The resulting explanation outputs support the interpretation of model predictions and personalized recommendations. Please click here to view a larger version of this figure.

Figure 8: Representative model-derived SHAP feature-importance analysis generated by the trained FedMediFormer-XAI model. The SHAP summary plot shows the distribution of SHAP values across the evaluated samples, with features ranked according to their mean absolute SHAP contribution. Positive SHAP values indicate contributions toward higher predicted diabetes risk, whereas negative values indicate contributions toward lower predicted risk. Color represents the corresponding feature value, with higher feature values shown in red and lower values shown in blue. The plot represents an actual model-derived explainability output rather than a schematic illustration. Please click here to view a larger version of this figure.

Figure 9: Representative attention outputs generated by the trained FedMediFormer-XAI model for one randomly selected held-out test sample. (A) Clinical feature attention obtained from an attention head of the multimodal transformer, showing the relative attention weights assigned to clinical features. (B) Temporal attention across the CGM sequence, illustrating time periods receiving higher model attention. (C) Spatial attention for the retinal fundus image, including the original image, attention heatmap, and attention overlay. (D) Cross-modal attention between clinical, CGM, and retinal modality tokens, showing intra-modal and cross-modal information weighting. The visualizations displayed are model-derived outputs generated by the trained model. Attention weights are shown for the selected test sample and should be interpreted as model attention patterns rather than as independent measures of clinical causality. Please click here to view a larger version of this figure.

Dataset	Data Type	Samples/Records	Features/Content	Purpose	Public Availability
Pima Indians Diabetes Dataset	Clinical Tabular	768 patients	8 clinical variables	Diabetes risk prediction	Public
CDC Diabetes Health Indicators	Population Health	253,680 records	Demographics, lifestyle, health indicators	Population risk analysis	Public
OhioT1DM Dataset	CGM Time-Series	12 subjects	Glucose, insulin, meals, exercise, sleep	Temporal diabetes modeling	Public
APTOS 2019 Blindness Detection	Retinal Images	3,662 images	Fundus photographs with severity labels	Diabetic retinopathy analysis	Public
Drug Review Dataset	Pharmacological	215,063 reviews	Drug effectiveness, ratings, reviews	Drug recommendation	Public
DrugBank Open Data	Drug Knowledge Graph	Thousands of drugs	Drug-drug interactions, targets, pathways	Graph construction	Public/Academic Access

Table 1: Dataset characteristics. Summary of publicly available datasets used in this study, including dataset type, sample size, data modality, feature content, intended purpose, and availability.

Dataset	Modality	Prediction Target	Fusion Level
Pima Indians Diabetes	Clinical	Diabetes classification	Feature embedding
CDC Diabetes Health Indicators	Population health	Diabetes risk prediction	Feature embedding
OhioT1DM	Continuous glucose monitoring	Glucose trend prediction	Feature embedding
APTOS 2019	Retinal fundus images	Diabetic retinopathy analysis	Feature embedding
Drug Review + DrugBank	Pharmacological	Drug recommendation	Graph embedding

Table 2: Multimodal dataset integration strategy. Summary of the datasets used for multimodal diabetes intelligence, including the data modality, prediction target, and level at which modality-specific representations are integrated. Clinical, population health, continuous glucose monitoring, and retinal image data are represented as feature embeddings, whereas pharmacological information is integrated via graph embeddings for personalized drug recommendation.

Model	Accuracy, Mean ± SD (95% CI)	Precision, Mean ± SD (95% CI)	Recall, Mean ± SD (95% CI)	F1-score, Mean ± SD (95% CI)	MCC, Mean ± SD (95% CI)	AUC-ROC, Mean ± SD (95% CI)
Random Forest	83.60 ± 0.74 (82.68–84.52)	82.70 ± 0.60 (81.96–83.44)	81.90 ± 0.56 (81.21–82.59)	82.30 ± 0.56 (81.61–82.99)	0.67 ± 0.03 (0.63–0.71)	0.88 ± 0.01 (0.87–0.89)
XGBoost	85.30 ± 0.71 (84.42–86.18)	84.70 ± 0.60 (83.96–85.44)	83.80 ± 0.56 (83.11–84.49)	84.20 ± 0.56 (83.51–84.89)	0.70 ± 0.03 (0.66–0.74)	0.90 ± 0.01 (0.89–0.91)
TabTransformer	87.50 ± 0.52 (86.85–88.15)	87.00 ± 0.60 (86.26–87.74)	86.20 ± 0.56 (85.51–86.89)	86.60 ± 0.56 (85.91–87.29)	0.75 ± 0.03 (0.71–0.79)	0.92 ± 0.01 (0.91–0.93)
Vision Transformer	88.80 ± 0.50 (88.18–89.42)	88.20 ± 0.60 (87.46–88.94)	87.60 ± 0.56 (86.91–88.29)	87.90 ± 0.56 (87.21–88.59)	0.78 ± 0.03 (0.74–0.82)	0.93 ± 0.01 (0.92–0.94)
Temporal Transformer	87.20 ± 0.51 (86.57–87.83)	86.60 ± 0.60 (85.86–87.34)	85.90 ± 0.56 (85.21–86.59)	86.20 ± 0.56 (85.51–86.89)	0.74 ± 0.03 (0.70–0.78)	0.91 ± 0.01 (0.90–0.92)
CNN-LSTM	86.90 ± 0.58 (86.18–87.62)	86.30 ± 0.60 (85.56–87.04)	85.60 ± 0.55 (84.92–86.28)	85.90 ± 0.56 (85.21–86.59)	0.73 ± 0.03 (0.69–0.77)	0.91 ± 0.01 (0.90–0.92)
Centralized Multimodal Transformer	91.30 ± 0.12 (91.15–91.45)	90.70 ± 0.60 (89.96–91.44)	90.10 ± 0.56 (89.41–90.79)	90.40 ± 0.56 (89.71–91.09)	0.83 ± 0.03 (0.79–0.87)	0.95 ± 0.01 (0.94–0.96)
FedMediFormer-XAI	94.20 ± 0.34 (93.78–94.62)	93.10 ± 0.30 (92.73–93.47)	92.80 ± 0.28 (92.45–93.15)	92.90 ± 0.28 (92.55–93.25)	0.88 ± 0.02 (0.86–0.90)	0.96 ± 0.01 (0.95–0.97)

Table 3: Diabetes prediction performance. Comparative predictive performance of FedMediFormer-XAI and baseline machine learning and deep learning models using accuracy, precision, recall, F1-score, MCC, and AUC-ROC metrics.

Metric	Centralized Learning	Federated Learning
Accuracy (%)	94.7	94.2
AUC-ROC	0.97	0.96
Privacy Preservation	No	Yes
Raw Data Sharing Required	Yes	No
Communication Rounds	N/A	100
Training Time (hours)	4.8	5.6
Regulatory Compliance	Moderate	High

Table 4: Federated versus centralized learning performance. Comparison of centralized and federated learning implementations with respect to predictive performance, raw-data sharing requirements, communication rounds, and training time.

Dataset	Accuracy (%)	Precision (%)	Recall (%)	AUC-ROC
Pima Indians Diabetes Dataset	91.8	90.5	89.8	0.93
CDC Diabetes Health Indicators	93.1	92.2	91.6	0.95
OhioT1DM Dataset	92.4	91.8	91.1	0.94
APTOS 2019 Blindness Detection	94.7	93.9	93.5	0.96
Multimodal Fusion (FedMediFormer-XAI)	94.2	93.1	92.8	0.96

Table 5: Dataset-level results. Performance analysis across individual datasets and multimodal fusion settings. Results illustrate the contribution of different data modalities to overall predictive performance.

Metric	Value
Precision@5	0.89
Recall@5	0.84
NDCG@5	0.91
Drug Interaction Detection Accuracy	0.95
Recommendation Coverage	0.88
Mean Reciprocal Rank (MRR)	0.86
Safety Compliance Score	0.94

Table 6: Personalized drug recommendation performance. Evaluation of graph-based personalized drug recommendation using ranking metrics, interaction detection accuracy, recommendation coverage, and medication safety assessment.

Evaluation criterion	Proposed evaluation design
Clinician trust	Five-point Likert-scale assessment
Interpretability	Five-point Likert-scale assessment
Clinical relevance	Five-point Likert-scale assessment
Explanation usefulness	Five-point Likert-scale assessment
Perceived utility	Five-point Likert-scale assessment
Inter-rater agreement	Fleiss' kappa, to be calculated after prospective evaluation
Statistical comparison	Paired statistical test, as appropriate after data collection
Participants	12 clinicians, subject to ethics approval and informed consent
Evaluation status	Prospective/future evaluation

Table 7: Proposed human-centered evaluation criteria. Proposed prospective clinician-centered evaluation framework for assessing the usefulness, clinical relevance, interpretability, trustworthiness, and usability of FedMediFormer-XAI explanations. The evaluation involves a standardized five-point Likert-scale assessment, inter-rater agreement using Fleiss' kappa, statistical comparison of prediction-only and prediction-plus-explanation conditions, and 95% confidence intervals. The clinician's evaluation should be conducted only after obtaining approval from the appropriate Institutional Ethics Committee and informed consent.

Supplementary Table 1: Dataset validation strategy. Dataset partitioning, validation procedures, class-balancing strategies, and quality-control measures were implemented to ensure reproducibility and reliable model evaluation. Please click here to download this file.

Supplementary Table 2: Diffusion model parameters. Configuration settings for the TabDDPM diffusion model, including training parameters, optimization settings, latent dimensions, noise scheduling strategy, and synthetic sample generation specifications. Please click here to download this file.

Supplementary Table 3: Multimodal transformer configuration. Configuration of the multimodal transformer architecture, including the clinical, temporal, and image encoders, embedding and fusion dimensions, attention heads, optimizer, learning rate, batch size, training epochs, early-stopping criterion, and combined training loss. Please click here to download this file.

Supplementary Table 4: Federated learning configuration. Parameters used for privacy-preserving federated training, including the number of participating hospitals, communication rounds, local training epochs, client participation rate, aggregation algorithm, optimizer, learning rate, encryption method, communication protocol, and raw-data sharing policy. Please click here to download this file.

Supplementary Table 5: GraphSAGE configuration. Configuration of the heterogeneous GraphSAGE-based personalized drug recommendation module, including graph type, hidden and embedding dimensions, number of graph layers, neighborhood sampling size, optimizer, learning rate, batch size, training epochs, Bayesian Personalized Ranking loss, and number of top medication recommendations. Please click here to download this file.

Supplementary Table 6: Explainability evaluation metrics. Quantitative and human-centered metrics used to evaluate the explainability of the FedMediFormer-XAI framework. The metrics assess explanation fidelity, stability, consistency, sparsity, completeness, sensitivity, infidelity, inter-rater agreement, and clinician-perceived explanation quality. Please click here to download this file.

Supplementary Table 7: Evaluation metrics. Predictive, federated learning, recommendation, and explainability metrics are used to evaluate framework performance, robustness, ranking quality, and interpretability. Please click here to download this file.

Supplementary Table 8: Human-centered evaluation design. Design of the clinician-centered evaluation study, including participant groups, evaluation conditions, assessment criteria, and statistical analysis procedures. Please click here to download this file.

Supplementary Table 9: Reproducibility assets. Summary of the datasets, implementation specifications, configuration files, evaluation procedures, documentation, and experimental records required to support reproducibility of the proposed FedMediFormer-XAI framework. Please click here to download this file.

Discussion

Interpretation of key findings
The representative results show that it is possible to combine multimodal transformer learning, federated learning, diffusion-based augmentation, graph neural network-based drug recommendation, and explainable artificial intelligence into a unified diabetes intelligence framework. By integrating clinical records, population health indicators, continuous glucose monitoring data, retinal fundus images, and pharmacological information, the proposed FedMediFormer-XAI framework demonstrated strong predictive performance while supporting interpretable and decentralized model development. The multimodal learning method extracted informative features from different healthcare modalities, leading to better diabetes prediction and personalized treatment recommendations.

Federated learning benefits
One major contribution of the proposed framework is the integration of federated learning to enable decentralized collaborative model building. Federated learning differs from traditional centralized learning methods by allowing multiple institutions to participate in model training without directly sharing raw patient data. The key findings showed that federated learning maintained predictive accuracy comparable to centralized learning while supporting decentralized data handling. This feature is particularly relevant in healthcare scenarios where data sharing is limited by institutional policies and privacy requirements.

Explainability analysis
The explainability assessment found that SHAP analysis, Integrated Gradients, attention visualization, and counterfactual explanations can all produce clinically meaningful insights into the model's behavior. Global and local explanations largely agreed on the top predictors from a clinical perspective (i.e., glucose measurements, body mass index, age, patterns of insulin administration, and retinal abnormalities). Attention visualizations also showed that the transformer architecture focused on clinically relevant temporal and imaging features. These approaches improve transparency by providing complementary interpretations of model predictions and feature contributions.

Personalized drug recommendation analysis
We propose a GNN-based recommendation model that, along with modeling patient-drug and drug-drug relationships, provides personalized drug recommendations. In fact, the heterogeneous graph topology facilitated both the mastery of treatment efficacy patterns and the consideration of medication safety. The excellent drug recommendation results obtained in this study illustrate the capabilities of graph-based learning methods for developing diabetes intelligence systems beyond disease prediction, towards a personalized therapeutic support scaffold.

Human-centered evaluation
The framework incorporates a prospective human-centered evaluation protocol to assess the effects of explainability on clinician trust, interpretability, usability, and perceived clinical utility. The proposed evaluation will compare prediction-only and prediction-plus-explanation conditions using standardized assessment criteria. Future implementation of this evaluation, following appropriate approval from the Institutional Ethics Committee and informed consent, will provide empirical evidence on clinician acceptance and the practical usefulness of the generated explanations.

Critical steps for successful and reproducible implementation
Several stages of the proposed protocol require particular attention to ensure reproducibility. Dataset preprocessing and partitioning should maintain patient-level separation and prevent data leakage, while the specified non-IID client distributions should be preserved during federated learning. Cross-modal fusion should maintain the defined latent dimensions, attention configuration, modality-type embeddings, normalization, and final projection. Diffusion-based augmentation should use consistent preprocessing and training parameters, while GraphSAGE-based recommendation should maintain consistent patient-drug representations, interaction screening, definitions of relevant items, and Top-K ranking criteria. Explainability analysis should use fixed SHAP background samples, Integrated Gradients baselines, attention-extraction procedures, and clinically permissible counterfactual constraints. Software versions, random seeds, model configurations, dataset partitions, and evaluation parameters should be documented to minimize implementation-dependent variation.

Limitations
Several limitations should be considered when interpreting the findings of this study. First, the framework was developed and evaluated using publicly available datasets, which may not fully represent the diversity of real-world healthcare populations and clinical environments. Second, although federated learning was evaluated using simulated hospital clients, large-scale validation involving independent healthcare institutions was not performed. Third, federated learning introduces additional communication and computational overhead that may affect scalability in resource-constrained environments. Finally, the explainability techniques employed in this study are primarily post hoc interpretation methods and may not fully capture the complex model's behavior.

The proposed human-centered validation protocol considers an initial panel of 12 clinical experts comprising endocrinologists, diabetes specialists, and clinical pharmacologists. This sample size is intended for a preliminary assessment of usability and interpretability rather than for definitive clinical validation. The relatively small expert panel may limit statistical power and generalizability. Future prospective studies should therefore involve larger, multicenter clinician cohorts representing diverse clinical settings and specialties.

A key limitation of the proposed framework is that the clinical, CGM, retinal imaging, and drug-related modalities are derived from independent public datasets rather than patient-matched multimodal records. Consequently, the cross-modal fusion component should be interpreted as a methodological framework for integrating complementary modality representations rather than as evidence derived from simultaneous multimodal measurements on the same patients. Differences in population characteristics, acquisition protocols, feature distributions, and disease definitions across the source datasets may introduce distributional discrepancies and may limit the extent to which the learned cross-modal relationships represent true patient-level associations. Although the protocol explicitly avoids artificial patient-level matching across independent datasets, this design limits direct assessment of patient-specific multimodal interactions and may affect clinical generalizability. Therefore, the reported multimodal performance should not be interpreted as equivalent to validation on a prospectively collected, patient-matched multimodal cohort. Future studies should validate the framework using larger, independently collected datasets containing synchronized clinical, CGM, retinal, and treatment information from the same patients.

Troubleshooting and protocol modifications
Several practical considerations may affect the implementation of the proposed protocol. Class imbalance may reduce predictive performance when minority classes are underrepresented; this challenge can be mitigated through diffusion-based augmentation and resampling strategies. Missing values and inconsistent data formatting may affect model stability and should be addressed through rigorous preprocessing procedures. Federated learning convergence may become unstable when client datasets exhibit substantial heterogeneity; adjusting local training epochs, learning rates, and aggregation frequencies may improve stability. Hardware limitations may also affect training efficiency, particularly when processing large multimodal datasets and transformer architectures. In such cases, reducing batch size or model complexity may improve computational feasibility while maintaining reproducibility.

Future directions
Future research may focus on integrating digital twin technologies for personalized disease simulation and treatment planning. Foundation models trained on large-scale multimodal healthcare datasets may further improve representation learning and model generalization. Causal explainability techniques could provide deeper insight into disease mechanisms and treatment effects beyond correlation-based explanations. Additionally, reinforcement learning approaches may support adaptive treatment optimization and dynamic medication recommendation strategies. These advances could further enhance the clinical utility and personalization capabilities of diabetes intelligence systems. Furthermore, AI-generated predictions and medication recommendations should be considered decision-support tools rather than replacements for professional clinical judgment. Appropriate human oversight remains essential for the responsible and ethical deployment of healthcare artificial intelligence systems.

Conclusion
FedMediFormer-XAI provides a reproducible framework integrating multimodal learning, federated training, personalized drug recommendation, and explainability for diabetes intelligence. Representative results support the feasibility of the proposed workflow, while further validation using patient-matched multimodal datasets, larger clinical settings, and prospective clinician evaluation is required before clinical deployment.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
AES Encryption	NIST	AES-256	Data-at-rest / protected model-parameter encryption in federated learning
APTOS 2019 Blindness Detection Dataset	Kaggle/APTOS	2019 Release	Retinal fundus image dataset; publicly available and de-identified; https://www.kaggle.com/competitions/aptos2019-blindness-detection/data
Captum	Meta AI	Version 0.8	Model interpretability and attribution analysis
CUDA Toolkit	NVIDIA	Version 12.2	GPU-accelerated computation
Diabetes Health Indicators Dataset (CDC BRFSS 2015)	CDC/BRFSS; Kaggle	2015 Release	Population health indicators; publicly available and de-identified; https://www.kaggle.com/datasets/alexteboul/diabetes-health-indicators-dataset
Diabetic Retinopathy Detection Dataset	Kaggle	Public Release	Retinal image dataset; publicly available and de-identified; https://www.kaggle.com/c/diabetic-retinopathy-detection/data
Digital certificates	-	-	Client authentication in federated learning
Drug Interaction Dataset	Kaggle	Public Release	Drug-drug interaction graph data; publicly available and de-identified; https://www.kaggle.com/datasets/rohanharode07/drug-drug-interaction
Drug Review Dataset	UCI Machine Learning Repository	Dataset 462	Drug effectiveness and review dataset; publicly available and de-identified; https://archive.ics.uci.edu/dataset/462/drug+review+dataset+drugs+com
Matplotlib	Matplotlib Developers	Version 3.9	Visualization
MTS Diabetes Dataset	Kaggle	Public Release	Wearable sensor / diabetes prediction dataset; publicly available and de-identified; https://www.kaggle.com/datasets/marshalpatel3558/diabetespredictiondataset
NetworkX	NetworkX Developers	Version 3.3	Drug interaction graph construction and analysis
NumPy	NumPy Developers	Version 1.26	Numerical computation
NVIDIA RTX 4090 GPU	NVIDIA	RTX 4090	Model training and inference; at least 32 GB system memory
OhioT1DM Dataset	Ohio University	Public Release	Continuous glucose monitoring dataset; publicly available and de-identified; https://webpages.charlotte.edu/rbunescu/data/ohiot1dm/OhioT1DM-dataset.html
OpenCV	OpenCV Foundation	Version 4.10	Image processing
Pandas	PyData	Version 2.2	Data processing and preprocessing
Pima Indians Diabetes Dataset	UCI Machine Learning Repository	Dataset 34	Clinical diabetes prediction dataset; publicly available and de-identified; https://archive.ics.uci.edu/dataset/34/diabetes
Python	Python Software Foundation	Version 3.11	Primary programming environment
PyTorch	Meta AI	Version 2.3	Deep learning model development and training
PyTorch Geometric	PyG Team	Version 2.5	Graph neural network construction and training
Scikit-learn	Scikit-learn Developers	Version 1.5	Machine learning utilities and evaluation
Secure aggregation mechanism	-	-	Protected aggregation of local model parameters in federated learning
SHAP	SHAP Developers	Version 0.47	Explainable AI and feature attribution analysis
TensorFlow	Google	Version 2.15	Deep learning model development and training
Transformers	Hugging Face	Version 4.45	Transformer-based architecture implementation
Transport Layer Security	IETF	TLS 1.3	Encrypted communication channel for federated parameter exchange
Workstation	Generic	-	Computational environment; at least 32 GB system memory