Hybrid Deep Reinforcement Learning for Multimodal Biomedical Signal Fusion

Summary

This research proposes a hybrid deep reinforcement learning (HDRL) framework that integrates deep neural networks (DNNs) for feature extraction with reinforcement learning (RL) for dynamic multimodal signal fusion. Evaluated on biomedical datasets, it excels in accuracy, robustness, and rare-event detection, providing major applications in personalized medicine and disease classification.

Abstract

Improving decision-making and diagnostic accuracy in health care requires combining multimodal biomedical signals such as electrocardiograms (ECG), electroencephalograms (EEG), and medical imaging. Traditional signal fusion techniques face significant challenges due to signal variability and inherent complexity. We propose a Hybrid Deep Reinforcement Learning (HDRL) model that employs Deep Neural Networks (DNNs) for feature extraction and Reinforcement Learning (RL) for dynamic fusion optimization. This hybrid model utilizes real-time feedback to optimize fusion policy learning and adapt to varying data characteristics. To evaluate its performance, the proposed model was compared with conventional fusion techniques, such as Deep Neural Network-based methods and Principal Component Analysis (PCA). The hybrid deep reinforcement learning model was tested on several real-world biomedical datasets. The proposed model is applicable to personalised medicine and disease classification, as experimental evidence demonstrates improved performance in terms of robustness under noisy conditions, classification accuracy, and rare-event detection. The proposed methodology, in addition to addressing concerns with traditional signal fusion techniques, offers promising solutions that advance multimodal biomedical signal processing. The model uses a DNN for feature mining and RL agents (DQN, PPO) for policy optimization.

Introduction

Modern healthcare increasingly relies on integrating diverse biomedical data sources to improve diagnostic accuracy and clinical decision-making. Physiological signals such as electrocardiograms (ECG), electroencephalograms (EEG), and medical imaging modalities including magnetic resonance imaging (MRI) and computed tomography (CT) provide complementary physiological information that supports disease diagnosis and personalized treatment. However, biomedical signals are inherently complex and often contain noise, missing values, and high-dimensional structures, making their analysis and interpretation challenging. Multimodal biomedical signal fusion, which integrates information from multiple physiological data streams, has therefore become an important approach for improving diagnostic performance, enhancing robustness to variations in data quality, and enabling comprehensive patient monitoring in clinical environments such as intensive care units (ICUs)¹.

Traditional multimodal signal fusion methods have largely relied on statistical and linear techniques. Principal Component Analysis (PCA) reduces the dimensionality of data by transforming signals into orthogonal components, while Canonical Correlation Analysis (CCA) identifies linear relationships between modalities to align multimodal datasets². These techniques have been successfully applied in biomedical multimodal fusion settings³. However, their reliance on linear assumptions limits their ability to capture complex nonlinear relationships that often exist among physiological signals. In practice, biomedical data are rarely purely linear. Conventional multimodal fusion methods often fail to capture meaningful interactions among modalities because of these nonlinear dependencies^4,5. As a result, conventional fusion methods may fail to capture meaningful interactions among modalities, limiting their effectiveness in tasks such as disease diagnosis and real-time clinical decision-making.

Recent advances in deep learning have significantly transformed biomedical signal processing by enabling neural networks to automatically extract hierarchical representations from raw data. Convolutional Neural Networks (CNNs) have demonstrated strong performance in analyzing spatial patterns in medical images^5,6. Multimodal CNN architectures have been used to integrate imaging modalities such as MRI and CT to improve diagnostic accuracy³. Similarly, Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks have been applied to sequential biomedical signals, including ECG and EEG, for disease classification tasks⁷. Despite these advances, many deep learning-based fusion approaches rely on fixed fusion strategies, such as feature concatenation or weighted averaging. These static fusion mechanisms do not adapt effectively to variations in signal quality. In real-world clinical environments, signals may degrade due to motion artifacts, electrode displacement, distortion, or environmental noise. Such variations can reduce the reliability of individual modalities and negatively impact the overall fusion process. Addressing these challenges requires adaptive fusion strategies capable of responding to dynamic data conditions in real-time clinical environments⁸.

Reinforcement Learning (RL) provides a powerful framework for sequential decision-making in uncertain environments. In RL, an agent learns to select actions that maximize cumulative reward through interactions with its environment. This decision-making process is commonly formulated using the Bellman equation, which defines the value of a state as the sum of the immediate reward and the discounted value of future rewards⁹. Methods such as Deep Q-Networks (DQN) extend reinforcement learning to high-dimensional problems by approximating the action-value function with deep neural networks¹⁰. Similarly, policy-based deep reinforcement learning methods enable stable and efficient policy learning through iterative updates of the agent’s policy¹¹. Reinforcement learning techniques have already been explored in several healthcare applications, including treatment optimization and anomaly detection¹², but their application to adaptive multimodal signal fusion remains relatively limited.

Hybrid Deep Reinforcement Learning (HDRL) integrates deep learning with reinforcement learning to combine feature extraction and adaptive decision-making. In this framework, deep neural networks transform multimodal biomedical signals into high-level feature representations, while reinforcement learning agents dynamically select fusion strategies based on task performance⁴. This adaptive mechanism enables the system to respond to variations in signal quality and modality availability, allowing more robust operation in complex clinical environments such as ICU monitoring and wearable diagnostic systems⁸.

In addition to classical reinforcement learning approaches, several hybrid learning techniques combining fuzzy logic, genetic algorithms, and reinforcement learning have been explored for biomedical classification tasks. For example, Fuzzy Q-learning combined with Genetic Algorithms has been applied to epileptic seizure classification using EEG signals, demonstrating improved adaptability to uncertain temporal patterns¹³. Similarly, fuzzy lattice-based evolutionary learning approaches have been proposed for pulmonary disease classification using medical images¹⁴. Other studies have investigated reinforcement learning-based approaches for seizure recognition¹⁵ and modified fuzzy Q-learning classifiers for detecting pneumonia and tuberculosis from chest X-ray images¹⁶. While these studies demonstrate the potential of hybrid learning methods in biomedical analysis, most existing approaches focus on single-modality classification rather than multimodal signal fusion.

Recent developments in multimodal machine learning have also explored transformer-based architectures for cross-modal medical applications. These models have shown promising performance in tasks such as healthcare image synthesis and analysis of heterogeneous biomedical datasets in which one or more modalities may be incomplete or missing^17. However, even with these advancements, the need for adaptive fusion mechanisms that dynamically respond to variations in multimodal data remains an important research challenge.

In this work, a Hybrid Deep Reinforcement Learning (HDRL) framework for multimodal biomedical signal fusion is proposed. Modality-specific CNN models are used to extract features from ECG, EEG, and MRI data, which are then combined to form a unified multimodal representation. A Deep Q-Network (DQN) agent dynamically selects fusion strategies, such as feature concatenation or weighted combination, according to the current feature state. A downstream classifier provides a reward signal based on classification performance, enabling the reinforcement learning agent to optimize the fusion policy through continuous feedback. This adaptive framework allows the model to adjust to variations in signal quality and modality availability. The proposed approach is evaluated using the PhysioNet 2016, CHB-MIT EEG, and OASIS-3 datasets and is compared with conventional PCA-based and deep learning-based fusion techniques.

Figure 1: Block diagram illustrating the architecture of the proposed HDRL-based multimodal signal-fusion system. Schematic representation of the multimodal fusion framework showing ECG, EEG, and medical image inputs processed through deep neural network feature extractors, followed by reinforcement learning–based adaptive fusion and reward feedback. Please click here to view a larger version of this figure.

The architecture of the proposed multimodal signal fusion framework is illustrated in Figure 1. In this architecture, modality-specific CNN models independently extract features from ECG, EEG, and MRI signals. These features are then combined into a unified representation, which the reinforcement learning agent uses to determine the optimal fusion strategy.

Biomedical signal processing plays a crucial role in modern healthcare by enabling accurate diagnosis, monitoring, and treatment planning through the analysis of physiological signals obtained from multiple modalities. Integrating signals such as ECG, EEG, and medical imaging data requires sophisticated fusion techniques that capture complementary information from different sources. Table 1 summarizes commonly used multimodal signal fusion approaches, including their key characteristics, advantages, and limitations, and highlights the motivation for the proposed HDRL-based framework.

Method	Key Features	Advantages	Limitations
PCA	Linear dimensionality reduction	Fast, simple	Limited to linear relationships
DNN	Non-linear feature learning	Captures complex patterns	Fixed fusion strategy
DRL	Dynamic policy learning	Adapts to data	High computation
HDRL	DNN + RL hybrid	Adaptive, robust, efficient	Requires careful tuning

Table 1: Comparison of multimodal fusion methods. Summary of commonly used multimodal signal fusion approaches, outlining their key characteristics, advantages, and limitations.

Protocol

This protocol describes the step-by-step implementation of the Hybrid Deep Reinforcement Learning (HDRL) model for multimodal biomedical signal fusion. The framework integrates deep learning and reinforcement learning techniques to optimize real-time fusion of ECG, EEG, and medical imaging data. The section also outlines the feature-extraction, training, and data-preparation procedures used in the proposed model.

Materials
All experiments were conducted on a workstation with a multi-core CPU, ≥32 GB of RAM, and a dedicated GPU (≥8 GB of VRAM). The implementation environment consisted of Python (v3.9 or later), TensorFlow (v2.12), and Keras (v2.12). Reinforcement Learning libraries were installed using pip install stable-baselines3. The Stable-Baselines3 package¹⁸ provides efficient implementations of Reinforcement Learning algorithms, such as Deep Q-Network (DQN) and Proximal Policy Optimisation (PPO).

Datasets
The proposed Hybrid Deep Reinforcement Learning (HDRL) framework was evaluated using three publicly available biomedical datasets representing multimodal signal types. (1) PhysioNet Challenge 2016: A collection of 12-lead ECG recordings from patients with heart disease, including atrial fibrillation, ventricular arrhythmia, and ischemic heart disease. The dataset includes annotated labels for heart conditions¹⁹. (2) CHB-MIT Scalp EEG Database: A dataset of scalp EEG recordings is used for epileptic seizure detection, with annotations marking seizure and non-seizure events²⁰. (3) OASIS-3 MRI Database: A collection of MRI brain images and clinical ratings is used for the detection and monitoring of Alzheimer’s disease²¹. Representative samples from each modality demonstrate the inherent variability and noise characteristics present in multimodal biomedical data (Figure 2).

Figure 2: Sample biomedical signals from ECG, EEG, and medical imaging. Representative examples of ECG waveform, EEG signal, and a medical imaging scan illustrating the different modalities used in the study. Please click here to view a larger version of this figure.

Data preprocessing
Each modality underwent modality-specific preprocessing procedures to ensure standardized input representation. EEG and ECG signals were filtered using a fourth-order Butterworth band-pass filter. The frequency range was set to 0.5–40 Hz for EEG and 0.5–45 Hz for ECG. Samples were segmented into 5-second windows with 50% overlap. Each signal was normalized by segmenting it to achieve zero mean and unit variance. MRI images were processed using skull stripping using the FSL Brain Extraction Tool (BET). Images were resized to 224 × 224 pixels. Pixel intensities were normalized to the [0,1] range to standardize input distributions.

Feature extraction
Feature extraction was performed independently for each modality using DNNs, ensuring that each modality’s unique characteristics were captured effectively.

ECG and EEG feature extractors:
A 1-D Convolutional Neural Network (1D-CNN) architecture was employed for time-series signal processing. The architecture consisted of (1) an input layer (window_length, 1), (2) Conv1D layers with 64 filters and a kernel size of 3, (3) ReLU activation, (4) MaxPooling1D with pool size=2, (5) Flatten and Dense layers (128 units, ReLU). The penultimate dense layer representations were extracted as modality-specific feature vectors.

MRI feature extractor
A pre-trained two-dimensional CNN model (e.g., VGG16 or ResNet50) was fine-tuned for extracting spatial features from MRI images. The CNN was trained on the OASIS-3 dataset, with the last few layers of the pre-trained model modified to adapt to the specific classification task²¹. The final classification layers were replaced with task-specific layers, and high-level spatial features were extracted from the penultimate layer^5,6.

Model architecture and training
The proposed Hybrid Deep Reinforcement Learning (HDRL) framework was developed and trained in two sequential stages: modality-specific feature extraction followed by reinforcement learning–based fusion optimization. In the first stage, separate Convolutional Neural Networks (CNNs) were trained on labeled data for ECG, EEG, and MRI modalities. The Adaptive Moment Estimation (Adam) optimizer with a learning rate of 0.001 and categorical cross-entropy loss was employed during supervised training. After convergence, high-level representations were extracted from the penultimate layer of each CNN, yielding modality-specific feature vectors F_ECG = CNN_ECG(X_ECG), F_ECG = CNN_ECG(X_ECG) and F_MRI = CNN_MRI(X_MRI). These vectors were concatenated to form a unified multimodal representation F_concat = [F_ECG, F_EEG, F_MRI], which serves as input to the Reinforcement Learning stage.

The fusion process was formulated as a Markov Decision Process (MDP), where the state s corresponded to the concatenated feature vector F_concat, the action represented the selection of a fusion strategy (either weighted summation or direct concatenation) and the reward r was defined as the classification accuracy obtained from a downstream classifier trained on the fused features. The action-value function Q(s,a) was approximated using a Deep Q-Network (DQN), which estimated the expected cumulative reward . Model parameters were iteratively updated according to the rule.

where α = 0.0001 is the learning rate, γ = 0.99 is the discount factor and s' denotes the next state.

During Reinforcement Learning optimization, the model was trained for 1,000 episodes with a batch size of 32. An epsilon-greedy strategy was adopted to balance exploration and exploitation, with initialized at 1.0 and decayed by 0.995 per episode. With probability , a random fusion strategy was selected, while with probability 1 - , the strategy with the highest estimated Q-value was chosen. Reward convergence was monitored throughout training, with stable performance (>0.85 cumulative reward) typically observed by approximately 500 episodes.

The HDRL workflow involves preprocessing multimodal signals, extracting modality-specific deep features, forming a unified state representation, selecting fusion strategies, and improving the policy using classification feedback from the DQN agent. The interaction between CNN-based feature extractors and the reinforcement learning agent forms a closed-loop adaptive fusion system (Figure 3), enabling policy updates based on reward feedback.

Figure 3: Flow diagram of the proposed Hybrid Deep Reinforcement Learning (HDRL) workflow. Workflow diagram showing modality-specific CNN feature extraction, feature concatenation, reinforcement learning–based selection of a fusion strategy (DQN/PPO), downstream classification, and reward-based policy update. Please click here to view a larger version of this figure.

Results

The experimental results indicate that the proposed Hybrid Deep Reinforcement Learning (HDRL) model achieves improved performance for multimodal biomedical signal fusion. The HDRL model was assessed by validating it against several real-world biomedical datasets. The results, in terms of robustness, adaptability, and classification accuracy, depict its superiority over other traditional fusion methods, such as PCA and DNN-based fusion.

Experimental Setup
Three publicly ava...

Discussion

Fusion efficiency:
This section presents a comparative analysis of fusion efficiency among HDRL, PCA, and DNN. The multimodal dataset, obtained by fusing EEG, ECG, and MRI data from the PhysioNet Challenge 2016¹⁹, CHB-MIT Scalp EEG Database²⁰, and OASIS-3 MRI Database²¹, was used to compute fusion efficiency. Processing time is a critical parameter in real-time clinical applications such as Intensive Care Unit (ICU) monitoring an...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Butterworth Filter (4th order, SciPy implementation)	SciPy Community	N/A	EEG/ECG signal preprocessing
CHB-MIT Scalp EEG Database	PhysioNet	DOI: 10.13026/C2K01R	EEG dataset
FSL Brain Extraction Tool (BET)	FMRIB Software Library (FSL)	N/A	MRI skull stripping
Keras (v2.12)	Google LLC	N/A	High-level neural network API
NVIDIA GPU (≥8 GB VRAM)	NVIDIA Corporation	Model-dependent	Required for accelerated deep learning training
OASIS-3 MRI Database	Washington University	DOI: 10.1002/alz.047259	MRI dataset
PhysioNet Challenge 2016 Dataset	PhysioNet	DOI: 10.22489/CinC.2016.179-154	ECG dataset
Python (v3.9 or later)	Python Software Foundation	N/A	Programming environment
Stable-Baselines3	DLR-RM (Open-source)	N/A	Reinforcement learning implementation (DQN, PPO)
TensorFlow (v2.12)	Google LLC	N/A	Deep learning framework
Workstation (Multi-core CPU, ≥32 GB RAM, ≥8 GB GPU)	Custom-built / Local Vendor	N/A	Used for model training and experimentation