FedMediFormer-XAI: Federated Multimodal Transformers with Diffusion Augmentation and Graph-Based Drug Recommendation for Diabetes

Summary

This protocol proposes FedMediFormer-XAI, a multimodal diabetes intelligence framework that respects privacy through federated learning, multimodal transformers, diffusion-based augmentation, graph-based personalized drug recommendation, and explainable artificial intelligence, for accurate prediction, transparent decision-making, and personalized diabetes management.

Abstract

Diabetes management faces a number of obstacles, such as fragmented healthcare data, privacy concerns, poor explainability, and the lack of personalized therapeutic guidance. This research work introduces FedMediFormer-XAI, a unified and proper framework that incorporates federated learning, multimodal transformers, diffusion-based data augmentation, Graph Neural Networks (GNNs) for drug recommendation, and Explainable Artificial Intelligence (XAI) for diabetes intelligence. The system utilizes diverse healthcare data types, e.g., clinical records, population health indicators, continuous glucose monitoring data, retinal fundus images, wearable sensor measurements, and pharmacological information. To generate synthetic samples and address class imbalance, diffusion models are used, and multimodal transformer architectures are employed to learn relationships among very different data sources. Federated learning enables collaborative training of models in a privacy-preserving manner without sharing raw patient data. The GNN component captures patient-drug and drug-drug interactions for personalized drug recommendations. Explainability methods such as SHapley Additive exPlanations (SHAP), attention visualization, Integrated Gradients, and counterfactual reasoning give transparent interpretations of prediction and recommendation results. In the representative implementation, the proposed framework achieved an accuracy of 94.2%, precision of 93.1%, recall of 92.8%, F1-score of 92.9%, Matthews Correlation Coefficient (MCC) of 0.88, and an Area Under the Receiver Operating Characteristic Curve (AUC-ROC) of 0.96. The graph-based recommendation module achieved precision = 0.89, recall = 0.84, and Normalized Discounted Cumulative Gain (NDCG) = 0.91. The protocol provides a structured approach for integrating heterogeneous healthcare data using multimodal transformers, federated learning, diffusion-based augmentation, graph-based recommendation, and explainable artificial intelligence. The reported computational results demonstrate the framework's potential for diabetes prediction and personalized medication recommendation, while the federated design supports decentralized data handling. A prospective multicenter clinical evaluation is required to establish clinical utility, generalizability, and real-world applicability.

Introduction

Diabetes mellitus is among the top chronic metabolic diseases affecting humanity and poses a major public health issue as it keeps growing, with its long-term effects and the huge costs of treating patients^1,2. According to a report, there are currently over 537 million adults living with diabetes worldwide, and the figure is expected to rise significantly over the next few decades³. If high blood sugar levels are not controlled, the person will suffer from the worst complications, such as coronary heart disease⁴, kidney damage⁵, nerve damage⁶, eyesight loss⁷, and brain stroke⁸. Thus, recognizing the disease at the earliest stage, regularly monitoring the patient's condition, and providing him or her training through advanced medical approaches are necessary steps to help patients live healthier lives and, in doing so, reduce the expenses of the State's healthcare system^9,10.

Nowadays, the use of electronic health records (EHRs), wearable devices, continuous glucose monitoring (CGM) systems, retinal imaging technologies, and mobile health applications is spreading rapidly, generating large volumes of heterogeneous healthcare data that can be analyzed¹¹. These various data-collecting tools can provide diverse information about the human body, its ability to function at different stages of a disease, how the patient responds to different types of treatment, and how he or she behaves in real life¹². Artificial intelligence (AI) has proven to be the best solution for processing highly complex data, predicting diabetes, monitoring the disease, and assisting doctors with clinical decisions¹³. Furthermore, machine learning algorithms, including random forests, support vector machines, and gradient boosting methods, have given excellent results in assessing diabetes risk¹⁴. More recently, deep learning architectures have enabled automatic feature extraction and improved predictive performance across diverse healthcare applications¹⁵.

Using multimodal learning methods, diabetes intelligence has been greatly enhanced by merging structured clinical records, physiological measurements, retinal images, and behavioral indicators into a single predictive framework¹⁶. Using multiple sources of information, multimodal models can identify patterns that are not visible when each modality is examined separately¹⁷. Transformer-based models have recently learned long-range dependencies and contextual relationships from diverse types of healthcare data very effectively¹⁸. However, problems such as unbalanced classes, missing data, data heterogeneity, and limited generalizability still prevent the widespread use of multimodal AI systems in clinical settings¹⁹.

Even with these developments, many existing diabetes intelligence systems still rely on centralized learning frameworks that inherently require patient data from different institutions to be consolidated into a single repository²⁰. Methods like these are already leading to disputes over patient privacy, data custody, cybersecurity, and regulatory compliance²¹. In fact, due to legal and ethical considerations, healthcare providers are often constrained in their ability to share sensitive patient data²². In this context, federated learning is becoming a viable solution, as it enables training a collaborative model without sharing the actual patient data²³. Through decentralized optimization and secure parameter aggregation, federated learning enables privacy-protected health analysis by leveraging knowledge from geographically distributed datasets²⁴. Nevertheless, communication overhead, client heterogeneity, and convergence stability still represent major issues in federated healthcare scenarios²⁵.

One more drawback of the diabetes intelligence systems presently in use is the lack of transparency and support for personalized therapy. Many deep learning models behave like black boxes, which is a problem for clinicians who want to understand the reasoning behind predictions and recommendations²⁶. Explainable artificial intelligence (XAI) methods such as SHAP, Integrated Gradients, attention visualization, and counterfactual reasoning are proving to have great potential to enhance model transparency and clinician trust²⁷. On top of that, graph neural networks (GNNs) have become the most advanced tools for capturing patient-drug and drug-drug interactions, resulting in personalized drug recommendations and medication safety evaluation²⁸. New diffusion models have also introduced solutions for generating authentic synthetic healthcare data and addressing class imbalance²⁹.

Transformers, federated learning, diffusion-based synthetic data generation, graph neural networks, and explainable artificial intelligence have each demonstrated promising results in healthcare applications; however, their integration within a single reproducible diabetes intelligence framework remains limited. Existing approaches commonly address these components independently, focusing on privacy-preserving learning without personalized treatment recommendation, multimodal prediction without decentralized training, or explainability without graph-based clinical decision support. Although diffusion models can support class balancing and transformer architectures can learn relationships across heterogeneous healthcare modalities, standardized protocols for integrating these complementary technologies remain limited. Therefore, this work presents a unified methodological protocol that combines multimodal prediction, diffusion-based augmentation, federated optimization, graph-based personalized drug recommendation, and explainable artificial intelligence within a reproducible workflow, while providing a foundation for future clinical validation and translation.

Unlike studies that assume the availability of patient-level multimodal datasets, this protocol integrates heterogeneous public datasets without performing direct patient matching. Independent modality-specific models are trained for each data source, and latent feature embeddings are combined through a cross-modal attention mechanism. This design preserves methodological rigor while enabling multimodal knowledge integration across diverse healthcare repositories.

The proposed protocol is intended primarily for methodological development and distributed healthcare AI settings in which heterogeneous data sources are available across separate repositories or institutions and direct data centralization is restricted. In the current implementation, the modality-specific datasets are independent and are not patient-matched; therefore, multimodal integration is performed at the learned representation level rather than through direct patient-level correspondence. Practical application would require appropriately linked multimodal patient data, consistent preprocessing and outcome definitions across participating sites, suitable computational infrastructure, and compliance with applicable ethical, privacy, and regulatory requirements.

This paper introduces FedMediFormer-XAI, a federated multimodal transformer framework that enables privacy-preserving diabetes intelligence through prediction, recommendation, and explainability features under one roof. This framework integrates diffusion-based data augmentation to address class imbalance, multimodal transformer learning for diabetes prediction, federated learning for collaborative model development, graph neural network-based personalized drug recommendation, and explainable artificial intelligence methods to support clinical decisions in a transparent manner. The overall architecture and workflow of the proposed FedMediFormer-XAI framework are illustrated in Figure 1.

The major contributions of the proposed FedMediFormer-XAI framework are summarized as follows: (1) A unified protocol is presented that integrates federated learning, multimodal transformers, diffusion-based data augmentation, graph neural network-based personalized drug recommendation, and explainable artificial intelligence within a single reproducible workflow for diabetes intelligence. (2) A privacy-preserving federated learning strategy is developed to enable collaborative model training across simulated distributed healthcare clients without sharing raw patient data. (3) A multimodal transformer architecture is introduced to jointly learn complementary representations from structured clinical records, continuous glucose monitoring data, retinal fundus images, and population health indicators. (4) Diffusion-based synthetic data generation is incorporated to reduce class imbalance and improve model robustness while preserving the statistical characteristics of the original training data. (5) A GraphSAGE-based heterogeneous graph learning module is employed to model patient-drug and drug-drug relationships for personalized medication recommendation and interaction-aware treatment ranking. (6) Multiple explainable artificial intelligence techniques, including SHapley Additive exPlanations (SHAP), Integrated Gradients, attention visualization, and counterfactual explanations, are integrated to improve model transparency and provide interpretable prediction and recommendation output. (7) A comprehensive validation protocol is provided, including predictive evaluation, federated learning assessment, recommendation evaluation, explainability analysis, clinician-centered evaluation, external validation, and reproducibility assessment.

Protocol

All datasets used in this study were obtained from publicly available repositories and contained de-identified patient information. These included the Pima Indians Diabetes Dataset, CDC Diabetes Health Indicators Dataset, OhioT1DM Dataset, APTOS 2019 Blindness Detection Dataset, Drug Review Dataset, and Drug-Drug Interaction Dataset. No direct patient recruitment, intervention, specimen collection, or access to identifiable personal health information was involved. The study was limited to a secondary analysis of publicly available, anonymized datasets for methodological development and validation, and was conducted in accordance with institutional requirements for the use of such data. The human-centered evaluation described in this protocol is intended for future implementation and was not conducted in the present study. Researchers conducting this evaluation should obtain appropriate approval from the Institutional Ethics Committee, obtain written informed consent from all participants, and comply with applicable institutional requirements, data-use agreements, and national regulations.

1. Configure the computational environment

1. Install Python 3.11 as the primary programming environment. Install PyTorch 2.3 and TensorFlow 2.15 for deep learning model development and training. Install the Transformers library (v4.45) to implement transformer-based architectures.
2. Install PyTorch Geometric (v2.5) to support graph neural network construction and training. Install SHAP (v0.47) and Captum (v0.8) to perform explainability and feature attribution analyses.
3. Install NumPy, Pandas, and Scikit-learn for numerical computation, data preprocessing, and machine learning utilities. Install OpenCV and Matplotlib for image processing and visualization tasks. Install NetworkX to construct and analyze drug interaction graphs.
4. Install the CUDA Toolkit and compatible NVIDIA GPU drivers to enable hardware-accelerated computation. Set up an NVIDIA RTX 4090 GPU with at least 32 GB of system memory. Set the random seed to 42 for training, validation, and testing to ensure reproducible results. Check the installations and compatibility of software libraries in use. Record the software versions, hardware specifications, and model configuration parameters used for the analysis.

2. Prepare and preprocess datasets

1. Download and organize the required clinical, physiological, imaging, and pharmacological datasets into modality-specific repositories and verify their integrity. Confirm the dataset characteristics, prediction targets, and integration levels as specified in Tables 1 and 2. The workflow is illustrated in Figure 2.
2. Remove duplicate, invalid, or inconsistent records and records missing essential variables. Divide each applicable dataset into patient-independent training (70%), validation (15%), and testing (15%) subsets. Fit median imputation, Min-Max scaling, and categorical encoding using the training subset only, and apply the fitted transformations unchanged to the validation and testing subsets. Perform synthetic data generation only on the training subset. Verify partitioning and leakage prevention in accordance with Supplementary Table 1.
3. Import glucose, food intake, insulin dosage, physical activity, and sleep data. Standardize timestamps, align events to regular intervals, interpolate unrecorded glucose values linearly, divide continuous records into fixed-length windows, and apply z-score normalization to generate temporal transformer inputs.
4. Load the retinal fundus images and remove corrupted or unreadable files, duplicates, images with missing labels, and images with severe artifacts or insufficient retinal-field visibility.
5. Resize retained images to 224 × 224 pixels, normalize pixel intensities to [0, 1], and apply contrast-limited adaptive histogram equalization (CLAHE).
6. Augment training images using random rotation within ±15°, horizontal flipping with probability 0.5, random zooming within 0.9–1.1×, and brightness adjustment within ±20%. Do not apply stochastic augmentation to validation or testing images. Store the processed images for Vision Transformer training.
7. Integrate the modality-specific representations according to the dataset integration strategy summarized in Table 2. Do not perform direct patient-level matching across independent repositories when common patient identifiers are unavailable.
8. Store each public dataset as an independent modality-specific dataset because patient identifiers are not shared across repositories.
9. Train modality-specific feature extractors independently using the corresponding datasets: (a) Clinical records → TabTransformer encoder; (b) Population health indicators → TabTransformer encoder; (c) Continuous glucose monitoring → Temporal Transformer; (d) Retinal fundus images → Vision Transformer; (e) Pharmacological data → GraphSAGE module
10. Extract latent feature embeddings independently from each modality. Fuse only the learned latent embeddings via the proposed cross-modal attention fusion module, rather than merging raw patient records.
11. Ensure that no artificial patient matching is performed between independent public datasets. Maintain independent train, validation, and test partitions for each dataset throughout preprocessing, augmentation, and model development.
12. Store the fused multimodal feature representations for downstream prediction, federated learning, explainability, and personalized drug recommendation (Figure 2).

3. Perform diffusion-based data augmentation

1. Apply diffusion-based augmentation to the structured tabular clinical and population health datasets using TabDDPM. Generate synthetic samples exclusively from the preprocessed training partition to prevent information leakage. Configure the model according to Supplementary Table 2.
2. Train TabDDPM for a maximum of 500 epochs using the Adam optimizer (learning rate = 1 × 10⁻⁴; batch size = 256). Apply early stopping if validation loss does not improve by at least 0.001 for 20 consecutive epochs. Generate synthetic samples for underrepresented classes.
3. Evaluate synthetic data quality using Kolmogorov–Smirnov (KS) statistics, Jensen–Shannon divergence, Wasserstein distance, feature-wise correlation analysis, and downstream classification performance. Compare original and synthetic feature distributions using histogram overlap, kernel density estimation, and pairwise correlation coefficients.
4. Accept synthetic samples only when class-wise feature distributions remain consistent with the original training data (KS test, p > 0.05; Jensen–Shannon divergence < 0.10) and contain no unrealistic clinical values. Combine accepted samples with the original training dataset to generate a class-balanced dataset. The augmentation workflow is shown in Figure 3.
5. Validate the synthetic data by retraining the diabetes prediction model using the original and augmented training datasets. Compare Accuracy, F1-score, MCC, and AUC-ROC to assess changes in model generalization and distributional bias.

4. Develop the multimodal transformer model

Configure the multimodal transformer with modality-specific encoders for structured clinical data, CGM sequences, and retinal fundus images, as illustrated in Figure 4. Integrate the resulting representations using cross-modal attention for diabetes prediction and graph-based personalized drug recommendation.

1. Develop the Clinical Encoder
   1. Initialize the TabTransformer encoder architecture. Input normalized clinical variables and population health indicators. Generate contextual feature representations using transformer attention mechanisms.
   2. Configure the TabTransformer encoder with 32 structured clinical input variables, an embedding dimension of 128, four transformer layers, eight attention heads, a feed-forward dimension of 512, a dropout rate of 0.20, and Gaussian Error Linear Unit (GELU) activation.
   3. Learn latent clinical embeddings that capture relationships among patient attributes. Store the resulting clinical embeddings for multimodal integration.
2. Develop the Continuous Glucose Monitoring Encoder
   1. Initialize the temporal transformer encoder. Feed continuous glucose monitoring sequences and physiological measurements into it. Use self-attention mechanisms to encode temporal dependencies.
   2. Configure the temporal transformer with a sequence length of 96 time steps, a sampling interval of 15 min, an embedding dimension of 128, four transformer layers, eight attention heads, sinusoidal positional encoding, and a dropout rate of 0.20.
   3. Produce sequential embeddings that reflect the patient's glucose dynamics. Keep the temporal embeddings ready for the next fusion process.
3. Develop the Vision Transformer Encoder
   1. Initialize the Vision Transformer architecture. Input preprocessed retinal fundus images. Divide images into fixed-size patches.
   2. Configure the Vision Transformer with an input image resolution of 224 × 224 pixels, a patch size of 16 × 16 pixels, an embedding dimension of 768, 12 transformer blocks, 12 attention heads, and a dropout rate of 0.10.
   3. Generate patch-level feature representations. Acquire visual features that signify retinal pathologies and disease-related biomarkers. Keep image embeddings for multimodal fusion.
4. Perform Cross-Modal Fusion
   1. Extract the latent representations from the clinical, CGM, and Vision Transformer encoders and project each into a common 256-dimensional latent space using independent linear projections followed by Layer Normalization.
   2. Concatenate the projected modality tokens, add learnable modality-type embeddings, and apply multi-head cross-modal attention to enable information exchange among the clinical, CGM, and retinal representations. Configure the attention block with eight heads, a 256-dimensional model space, a 1,024-dimensional feed-forward layer, GELU activation, 0.10 dropout, residual connections, and Layer Normalization.
   3. Mean-pool each modality-specific token group and concatenate the resulting representations. Project the concatenated 768-dimensional vector to a 512-dimensional unified multimodal representation and retain the attention weights and fused representations for subsequent explainability and ablation analyses.
   4. Forward the unified representation to the diabetes prediction and GraphSAGE-based drug recommendation modules. Maintain the original modality-specific training, validation, and testing partitions throughout fusion without artificial patient-level matching across the independent datasets.
   5. Configure the multimodal transformer according to the model parameters summarized in Supplementary Table 3.

5. Configure federated learning

1. Configure the federated learning framework to enable collaborative model training across geographically distributed healthcare institutions without exchanging patient-level data. Perform local model optimization at each participating institution using locally available data. Share only protected model parameters with the central aggregation server. Preserve patient privacy while enabling collaborative development of multimodal models.
2. Separate the multimodal datasets and distribute them among multiple federated clients to mimic the scenario of different healthcare institutions collaborating without sharing their data.
3. Configure the federated learning network with 10 participating hospital clients, one central aggregation server, five local epochs per communication round, 100 communication rounds, an 80% client participation fraction, random client selection at each communication round, and a minimum of eight participating clients per round.
4. Give each client a dataset that belongs to a particular institution so that the data remains local and confidential. Partition the multimodal training data among 10 simulated hospital clients using a Dirichlet-based non-independent and identically distributed (non-IID) strategy to generate heterogeneous class distributions, demographic compositions, and client dataset sizes.
5. Use a fixed random seed to ensure reproducibility. Verify the resulting client-level distributions and retain these partitions throughout federated training. Keep the validation and testing datasets independent of the client partitions.
6. Initialize each local client with the global model parameters received from the central aggregation server. Train the local multimodal transformer for five epochs using the client's local training data. Optimize the local model using the AdamW optimizer with a learning rate of 1 × 10⁻⁴.
7. Compute the local model parameter updates after completion of local training. Protect the local model parameters before transmission to the central aggregation server. Transmit the protected local model parameters to the central aggregation server for global aggregation.
8. Receive the protected model parameters from all participating hospital clients. Verify the integrity of the received model updates before aggregation. Aggregate the local model weights using the Federated Averaging (FedAvg) algorithm. Update the global multimodal transformer using the aggregated model parameters.
9. Redistribute the updated global model parameters to the participating hospital clients. Repeat the local training and global aggregation procedures until 100 communication rounds are completed or the predefined convergence criterion is satisfied.
10. Protect transmitted model parameters using the specified secure aggregation mechanism and AES-256 encryption. Authenticate participating clients using digital certificates.
11. Establish encrypted communication channels using Transport Layer Security (TLS) 1.3. Retain only the aggregated global model parameters after completion of each communication round.
12. Evaluate privacy and security using raw-data exposure, unauthorized sharing, membership inference, and model inversion assessments. For membership inference, compare attack AUC-ROC with the random baseline (0.50); assess inversion using structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR) for retinal images and normalized reconstruction error for numerical features.
13. Verify secure aggregation by confirming that the server cannot access individual unaggregated client updates. Verify AES-256/TLS 1.3 protection and confirm that no unencrypted model-update transmissions occur.
14. Compare privacy metrics between federated and centralized configurations where applicable and report the metrics specified in Supplementary Table 4.
15. Estimate the computational complexity of local transformer training as O(N × L × d²), where N represents the number of samples, L represents the number of transformer layers, and d represents the embedding dimension.
16. Estimate the computational complexity of federated aggregation as O(K × P) per communication round, where K represents the number of participating clients and P represents the number of trainable model parameters.
17. Exchange only protected model parameters between the participating clients and the central aggregation server during each communication round. Calculate the communication overhead based on the model size, the number of participating clients, and the number of communication rounds.
18. Compare the parameter-exchange overhead with the estimated cost of transferring the corresponding multimodal healthcare datasets. Configure the federated learning framework according to the parameters summarized in Supplementary Table 4. The federated learning workflow and parameter aggregation process are illustrated in Figure 5.

6. Construct the personalized drug recommendation module

1. Import the pharmacological, medication, patient treatment, and drug-interaction data into the computational environment. Construct a heterogeneous healthcare graph containing patient, drug, disease, laboratory test, and clinical risk-factor nodes.
2. Define patient-disease, patient-drug, disease-drug, drug-drug interaction, and patient-laboratory relationships as graph edges. Verify the graph structure and remove duplicate, invalid, or disconnected relationships before model training.
3. Represent patient nodes using age, sex, body mass index (BMI), HbA1c, blood glucose, blood pressure, and diabetes duration. Represent drug nodes using drug class, mechanism of action, dosage, administration frequency, and known adverse effects.
4. Represent disease nodes using disease severity, disease stage, and associated complications. Encode categorical attributes and normalize continuous node features before GraphSAGE training.
5. Initialize the node embeddings using the preprocessed node features. Sample up to 25 neighboring nodes for each target node during neighborhood aggregation. Aggregate neighboring node representations using mean aggregation. Update the target-node embeddings using the aggregated neighborhood representations.
6. Apply the rectified linear unit (ReLU) activation function after each GraphSAGE layer. Normalize the resulting node embeddings. Repeat the neighborhood aggregation procedure across three GraphSAGE layers.
7. Configure the GraphSAGE model with a hidden dimension of 256, an embedding dimension of 128, three GraphSAGE layers, a neighbor-sampling size of 25, a dropout rate of 0.30, a learning rate of 1 × 10⁻⁴, a batch size of 512, and a maximum of 100 training epochs. Optimize the model using the Adam optimizer.
8. Compute the patient-specific embedding from the trained GraphSAGE model. Compute embeddings for candidate medications. Calculate suitability scores between the patient representation and candidate drug representations. Rank candidate medications according to their predicted suitability scores.
9. Identify and remove contraindicated or potentially unsafe medications using available drug-drug interaction information. Return the five highest-ranked eligible medications as the personalized Top-5 recommendations.
10. Train the recommendation model using Bayesian Personalized Ranking (BPR) loss. Optimize the ranking objective to assign higher scores to clinically appropriate medications than to less suitable candidate medications.
11. Identify influential neighboring nodes and clinically relevant graph relationships contributing to each medication recommendation. Calculate feature-attribution scores for patient, disease, laboratory, and medication-related characteristics contributing to the recommendation.
12. Generate visual graph-based explanations showing the relationships that contribute to each personalized medication recommendation. Configure the GraphSAGE recommendation model according to the parameters summarized in Supplementary Table 5. The graph-based personalized drug recommendation workflow is illustrated in Figure 6.

7. Perform explainability assessment

1. Perform SHAP analysis
   1. Load the trained multimodal transformer and the held-out test dataset into the computational environment.
   2. Initialize the SHAP explainer using the trained prediction model and a background sample of 100 training samples, selected exclusively from the training dataset using stratified random sampling according to the diabetes outcome class. Maintain the approximate class distribution of the complete training partition in the background sample, and use a fixed random seed of 42 to ensure reproducible sample selection. Compute SHAP values for the selected test samples.
   3. Calculate global feature importance from the absolute SHAP values, generate global and local SHAP visualizations, and retain the calculated values for subsequent quantitative and statistical analyses.
2. Perform integrated gradients analysis
   1. Select representative correctly classified and misclassified samples from the held-out test dataset. Define modality-specific baseline inputs before calculating Integrated Gradients. Use a zero baseline for normalized clinical and CGM numerical features, representing zero normalized feature contribution, and use a zero-valued baseline image for the normalized retinal-image input.
   2. Compute Integrated Gradients attribution scores using 100 interpolation steps between the baseline and the original input. Normalize the resulting attribution scores to enable comparison across features and modalities. Generate attribution visualizations to identify influential clinical, temporal, and imaging features contributing to individual model predictions.
3. Perform attention visualization
   1. Extract attention matrices from the trained transformer layers for representative test samples. Calculate the mean attention weights across the corresponding attention heads. Generate attention heatmaps to visualize cross-modal interactions among clinical, temporal, and retinal imaging representations. Identify the dominant clinical, temporal, and retinal features receiving high attention weights during prediction.
   2. Extract the attention weights from the trained multimodal transformer for a randomly selected held-out test sample and visualize the corresponding clinical-feature, CGM temporal, retinal spatial, and cross-modal attention distributions.
4. Generate counterfactual explanations
   1. Select representative patient records from the held-out test dataset. Define clinically permissible ranges and constraints for modifiable patient features before generating counterfactual samples. Restrict continuous features to their observed training-data range, and restrict categorical features to valid categories observed in the training data. Do not modify immutable demographic characteristics, including age and sex.
   2. Generate counterfactual samples by minimally modifying only the permitted modifiable features while preserving the predefined clinical and feature-domain constraints. Reject counterfactuals containing values outside the observed training-data range, invalid categorical states, or modifications to immutable features. Identify the smallest clinically plausible feature modifications required to alter the predicted outcome.
   3. Report the modified features, original values, counterfactual values, and corresponding changes in model predictions. The explainability assessment workflow is illustrated in Figure 7, while representative attention visualization and counterfactual explanation outputs are presented in Supplementary Figure 1.
5. Evaluate explanation quality
   1. Calculate fidelity to quantify the agreement between the generated explanations and the behavior of the prediction model. Calculate stability by repeatedly generating explanations under small input perturbations and comparing the resulting attribution patterns. Evaluate consistency by comparing explanations generated for clinically similar test samples.
   2. Calculate sparsity by determining the number or proportion of features contributing substantially to each explanation. Evaluate completeness to determine whether the attributed features adequately account for the corresponding model output. Evaluate sensitivity by measuring changes in attribution scores following controlled perturbations of the input features.
   3. Calculate infidelity to quantify disagreement between feature attributions and observed changes in model predictions. Record all quantitative explainability metrics for subsequent statistical analysis.
6. Perform prospective clinician validation
   1. For future prospective clinical validation, obtain approval from the appropriate Institutional Ethics Committee before recruiting clinical experts. After obtaining ethical approval, recruit 12 clinicians comprising six endocrinologists, three diabetes specialists, and three clinical pharmacologists, and obtain informed consent from all participants before initiating the evaluation.
   2. Randomly select representative model predictions and their corresponding explanations for clinician assessment. Present the selected explanations independently to each participating clinician using a standardized evaluation format.
   3. Ask each clinician to evaluate the explanation's usefulness, clinical relevance, interpretability, and trustworthiness using a five-point Likert scale. Record anonymized clinician ratings and calculate inter-rater agreement and the corresponding statistical measures as specified in Step 7.7.
7. Perform statistical analysis
   1. Assess the distribution of the quantitative evaluation scores before selecting the statistical comparison test. Apply a paired t-test to normally distributed paired measurements or the Wilcoxon signed-rank test to non-normally distributed paired measurements, as appropriate.
   2. Calculate Fleiss' kappa to quantify inter-rater agreement among the participating clinicians. Calculate 95% confidence intervals for the principal explainability and clinician-evaluation metrics. Set the statistical significance threshold at p < 0.05.
8. Report the calculated test statistics, confidence intervals, and p-values with the corresponding evaluation results. Evaluate and report the explainability metrics according to the criteria summarized in Supplementary Table 6.

8. Evaluate model performance

1. Compare FedMediFormer-XAI with Logistic Regression, Random Forest, XGBoost, TabTransformer, Vision Transformer, Temporal Transformer, and the centralized multimodal transformer using accuracy, precision, recall, F1-score, MCC, and AUC-ROC.
2. Perform each baseline evaluation using the same predefined training, validation, and testing strategy to ensure a consistent comparison with the proposed framework.
3. Repeat the model evaluation across independent experimental runs and record the performance metrics obtained in each run. Calculate the mean, standard deviation (SD), and 95% confidence interval (CI) for Accuracy, Precision, Recall, F1-score, MCC, and AUC-ROC. Report the performance results as mean ± SD together with the corresponding 95% CI.
4. Assess the normality of the paired performance differences between FedMediFormer-XAI and each baseline model. Apply a paired t-test to normally distributed paired measurements and the Wilcoxon signed-rank test to non-normally distributed paired measurements. Set the statistical significance threshold at p < 0.05.
5. Report the corresponding test statistics, p-values, and 95% confidence intervals to quantify the statistical significance and uncertainty of the observed performance differences.
6. Get a count of all communication rounds until the models finally converge. Keep an eye on how the global model performs during federated training. Study how convergence changes over time in communication rounds.
7. Check how well parameter aggregation works. Examine how decentralized learning influences predictive performance. Make a comparison between the results from federated learning and centralized learning.
8. Assess federated convergence across communication rounds and compare federated and centralized predictive performance. Evaluate drug recommendations using Precision@5, Recall@5, and NDCG@5.
9. Generate negative samples–drug samples from the eligible candidate-drug pool by selecting medications that are not observed as positive patient–drug interactions for the corresponding patient. Exclude all known positive interactions and all contraindicated medications from the negative-sampling pool. Maintain a fixed negative-to-positive sampling ratio of 1:1 for recommendation training and use a fixed random seed of 42 for reproducible negative-sample selection.
10. Establish the relevant item set for each patient exclusively from the held-out evaluation data. Define the relevant-item set Ru​ for patient u as the set of medications that satisfy the positive patient–drug criteria in the held-out ground-truth records. Do not use model predictions to construct Ru. Remove contraindicated medications from the candidate recommendation set before ranking.
11. Generate the five highest-ranked eligible medications for each evaluation patient. Calculate Precision@5 as the proportion of the five recommended medications that belong to the patient's relevant-item set Ru. Calculate Recall@5 as the proportion of the patient's relevant medications retrieved among the five recommendations. Calculate NDCG@5 using the graded relevance of the recommended medications, assigning higher relevance to clinically appropriate medications and zero relevance to non-relevant medications. Aggregate the metrics across all evaluation patients and report the mean Precision@5, Recall@5, and NDCG@5.
12. Construct the recommendation ground truth before model evaluation and keep the held-out patient- drug interactions separate from the recommendation-training data. Do not use test-set medication interactions, test-set labels, or future treatment information during GraphSAGE training, negative sampling, or candidate ranking.
13. Measure Fidelity to check how well the explanations match the model's behavior. Assess the explanation for stability through multiple examinations. Judge Comprehensibility based on doctor-oriented criteria.
14. Investigate how consistent explanations are across different types of data. Confirm the clinical value of the explanations produced. The evaluation metrics used throughout the study are summarized in Supplementary Table 7.

9. Perform prospective clinician validation

1. Conduct prospective clinician validation after obtaining appropriate ethics approval and informed consent, following the recruitment, evaluation, data-collection, and statistical-analysis procedures described in Steps 7.6–7.8. The prospective clinician-validation design, including participant composition, evaluation criteria, comparison conditions, and planned statistical analyses, is summarized in Supplementary Table 8.

10. Perform validation and reproducibility assessment

1. Conduct ablation studies by systematically removing individual components from the complete FedMediFormer-XAI framework while maintaining the same dataset partitions, preprocessing procedures, training settings, and evaluation criteria. Evaluate the full model and configurations without diffusion augmentation, federated learning, GraphSAGE-based drug recommendation, and multimodal fusion.
2. Compare the ablated configurations with the complete FedMediFormer-XAI framework using Accuracy, Precision, Recall, F1-score, MCC, and AUC-ROC. Quantify the change in each performance metric to determine the contribution of each framework component. Compare validation results with internal testing outcomes to evaluate generalizability. Perform statistical significance testing to determine the reliability of observed performance differences.
3. Assess the generalizability of the framework using an independent external dataset when such a dataset with compatible input variables and outcome definitions is available. Apply the same preprocessing and evaluation procedures used for the internal test dataset, and compare the resulting performance metrics with the internal test results.
4. Perform statistical significance analysis across repeated experimental runs. Assess the normality of paired performance differences before statistical testing. Apply a paired t-test to normally distributed paired measurements and the Wilcoxon signed-rank test to non-normally distributed paired measurements. Set the statistical significance threshold at p < 0.05.
5. Calculate the mean and 95% confidence interval for Accuracy, Precision, Recall, F1-score, MCC, and AUC-ROC. Report the corresponding test statistics, p-values, and confidence intervals for the principal model comparisons. Document all model parameters, preprocessing procedures, training configurations, and evaluation protocols required to reproduce the reported experiments.
6. Preserve the source code, configuration files, evaluation scripts, and trained-model specifications used for the reported experiments. Maintain detailed records of model training, experimental settings, random seeds, and evaluation results.
7. Verify the completeness and consistency of the available reproducibility materials. Ensure that the documented methods and materials provide sufficient information for independent reproduction of the experimental workflow.
8. Summarize the reproducibility resources and validation procedures used in the study in Supplementary Table 9. Document the ablation-study procedure and evaluation criteria for assessing the contribution of the major framework components.

Results

Predictive performance analysis
FedMediFormer-XAI demonstrated improved predictive performance compared with the evaluated unimodal and conventional baseline models. Diffusion-based augmentation improved minority-class representation, and the resulting predictive performance is summarized in Table 3. The repeated-run evaluation demonstrated stable predictive performance across the evaluated models. FedMediFormer-XAI achieved the highest overall performance, with an accuracy of 94.20 ...

Discussion

Interpretation of key findings
The representative results show that it is possible to combine multimodal transformer learning, federated learning, diffusion-based augmentation, graph neural network-based drug recommendation, and explainable artificial intelligence into a unified diabetes intelligence framework. By integrating clinical records, population health indicators, continuous glucose monitoring data, retinal fundus images, and pharmacological information, the proposed FedMediFormer-XAI fram...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
AES Encryption	NIST	AES-256	Data-at-rest / protected model-parameter encryption in federated learning
APTOS 2019 Blindness Detection Dataset	Kaggle/APTOS	2019 Release	Retinal fundus image dataset; publicly available and de-identified; https://www.kaggle.com/competitions/aptos2019-blindness-detection/data
Captum	Meta AI	Version 0.8	Model interpretability and attribution analysis
CUDA Toolkit	NVIDIA	Version 12.2	GPU-accelerated computation
Diabetes Health Indicators Dataset (CDC BRFSS 2015)	CDC/BRFSS; Kaggle	2015 Release	Population health indicators; publicly available and de-identified; https://www.kaggle.com/datasets/alexteboul/diabetes-health-indicators-dataset
Diabetic Retinopathy Detection Dataset	Kaggle	Public Release	Retinal image dataset; publicly available and de-identified; https://www.kaggle.com/c/diabetic-retinopathy-detection/data
Digital certificates	-	-	Client authentication in federated learning
Drug Interaction Dataset	Kaggle	Public Release	Drug-drug interaction graph data; publicly available and de-identified; https://www.kaggle.com/datasets/rohanharode07/drug-drug-interaction
Drug Review Dataset	UCI Machine Learning Repository	Dataset 462	Drug effectiveness and review dataset; publicly available and de-identified; https://archive.ics.uci.edu/dataset/462/drug+review+dataset+drugs+com
Matplotlib	Matplotlib Developers	Version 3.9	Visualization
MTS Diabetes Dataset	Kaggle	Public Release	Wearable sensor / diabetes prediction dataset; publicly available and de-identified; https://www.kaggle.com/datasets/marshalpatel3558/diabetespredictiondataset
NetworkX	NetworkX Developers	Version 3.3	Drug interaction graph construction and analysis
NumPy	NumPy Developers	Version 1.26	Numerical computation
NVIDIA RTX 4090 GPU	NVIDIA	RTX 4090	Model training and inference; at least 32 GB system memory
OhioT1DM Dataset	Ohio University	Public Release	Continuous glucose monitoring dataset; publicly available and de-identified; https://webpages.charlotte.edu/rbunescu/data/ohiot1dm/OhioT1DM-dataset.html
OpenCV	OpenCV Foundation	Version 4.10	Image processing
Pandas	PyData	Version 2.2	Data processing and preprocessing
Pima Indians Diabetes Dataset	UCI Machine Learning Repository	Dataset 34	Clinical diabetes prediction dataset; publicly available and de-identified; https://archive.ics.uci.edu/dataset/34/diabetes
Python	Python Software Foundation	Version 3.11	Primary programming environment
PyTorch	Meta AI	Version 2.3	Deep learning model development and training
PyTorch Geometric	PyG Team	Version 2.5	Graph neural network construction and training
Scikit-learn	Scikit-learn Developers	Version 1.5	Machine learning utilities and evaluation
Secure aggregation mechanism	-	-	Protected aggregation of local model parameters in federated learning
SHAP	SHAP Developers	Version 0.47	Explainable AI and feature attribution analysis
TensorFlow	Google	Version 2.15	Deep learning model development and training
Transformers	Hugging Face	Version 4.45	Transformer-based architecture implementation
Transport Layer Security	IETF	TLS 1.3	Encrypted communication channel for federated parameter exchange
Workstation	Generic	-	Computational environment; at least 32 GB system memory