Convolutional Neural Network-Transformer Model to Predict and Classify Early Arrhythmia Using Electrocardiogram Signal

Summary

The developed model aims to classify early arrhythmias into N, L,R, V, and A classes. Here, all of the datasets are combined to create a principal dataset, which the model uses as input to produce different arrhythmia classes as output.

Abstract

As a major cause of death worldwide, cardiovascular diseases-especially arrhythmias-require the creation of precise and automated technologies for early diagnosis and detection. To identify arrhythmias from electrocardiogram (ECG) signals, this paper introduces a deep learning-based classification model that focuses on five main heartbeat types: Normal (N), Left Bundle Branch Block (L), Right Bundle Branch Block (R), Atrial Premature Beat (A), and Premature Ventricular Contraction (V). We leverage Lead I signals from several sources, such as the INCART 12-lead, Sudden Cardiac Death Holter, Supraventricular, and MIT-BIH Arrhythmia databases, yielding more than 3.9 million training and 112,575 testing segments.

Examples of data preparation include 180 sample, fixed-window segmentation, Min-Max normalization, and class balancing with the Synthetic Minority Over-sampling Technique (SMOTE). The hybrid architecture uses Transformer layers to model temporal dependencies and 1D Convolutional Neural Networks (CNNs) to extract spatial features. The Adam optimizer with dropout and batch normalization for regularization trains the model.

The proposed system outperforms the TN4 model and other cutting-edge benchmarks, achieving 99.99% accuracy, precision, and F1-score across all classes. Feature robustness is further improved by applying deep hybrid architectures and convolutional neural networks, which were motivated by earlier studies. The suggested paradigm advances artificial intelligence-driven, individualized digital healthcare and has great promise for scalable, real-time arrhythmia identification.

Introduction

Cardiovascular diseases (CVDs) remain one of the leading health concerns globally, responsible for nearly 31% of deaths worldwide each year, according to the World Health Organization (WHO)¹. A significant subset of these cases involves arrhythmias-irregularities in the heart's rhythm that can range from benign to life-threatening. Arrhythmias are often marked by irregular times. These disruptions substantially contribute to patient morbidity and mortality, heightening the risk of severe health issues such as stroke, heart failure, and sudden cardiac arrest. Early identification and accurate classification of arrhythmias are therefore crucial for improving patient outcomes, managing healthcare costs, and advancing the quality of cardiac care².

Electrocardiography (ECG) remains a key non-invasive diagnostic tool for detecting arrhythmias. By capturing the heart's electrical activity as visual waveforms, ECG enables clinicians to identify subtle variations that may point to specific arrhythmic patterns³. However, manually interpreting ECG signals is time-consuming and prone to variability due to individual differences among practitioners, which introduces a margin of human error. These challenges are magnified when handling extensive datasets or recognizing nuanced arrhythmias. With the global shift towards digital healthcare, the need for reliable and automated diagnostic systems capable of delivering consistent, real-time ECG analysis is increasingly vital evident⁴.

Traditional machine learning models have been applied to ECG analysis with moderate success; however, their reliance on handcrafted features and domain-specific expertise poses notable limitations. To address this limitation, the model employs the Synthetic Minority Over-sampling Technique (SMOTE), which generates synthetic data points for underrepresented classes, thereby balancing the dataset and improving the model's capacity to generalize across all heartbeat types. The integration of SMOTE with our deep learning model enhances classification performance, especially for rare arrhythmias, and supports a more equitable diagnostic tool⁵. The primary objectives of this research are threefold. First, the aim is to develop an accurate, scalable model capable of real-time arrhythmia detection, contributing to the global shift toward digital and personalized healthcare. Second, to demonstrate the effectiveness of a hybrid CNN-Transformer architecture in ECG analysis, highlighting its potential to outperform traditional methods in both accuracy and robustness⁶.

Finally, the aim is to develop a model with meaningful applications in clinical diagnostics, enhancing early detection of cardiac abnormalities. This study seeks to bridge the gap between deep learning advancements and practical medical applications, positioning our approach as a valuable tool in pursuing improved cardiac care and patient outcomes⁶.

Protocol

1. Acquiring datasets

1. Acquire publicly available electrocardiogram (ECG) datasets to develop and validate the deep learning model for arrhythmia classification⁷.
2. Combine Lead-I datasets from MIT-BIH Arrhythmia Database, MIT-BIH Supraventricular Arrhythmia Database, the St. Petersburg INCART 12-lead Arrhythmia Database, and the Sudden Cardiac Death Holter Database.
   NOTE: Datasets are chosen for their diversity in patient demographics and arrhythmia types, ensuring the model can generalize across varied cases. Each dataset provides high-quality annotated ECG recordings, covering a range of heartbeat classes. Only Lead I data was extracted and used across all datasets to standardize the input, maintaining consistency and focusing on the most diagnostically relevant signal. As shown in Figure 1, Lead I measures the potential difference between the right arm (negative electrode) and the left arm (positive electrode), and Lead II measures the potential difference between the right arm (negative electrode) and the left leg (positive electrode). Table 1 illustrates that the principal dataset is created by combining four datasets and eliminating null values, and the file is named Alldata.CSV.

2. Data preprocessing

1. Divide the data from Alldata.CSV into two parts-train and test. In preprocessing 2, filter the N, L, R, A, V classes from Alldata.CSV and then divide it into two parts-Train and test parts (Figure 2).
2. Divide each patient's ECG signal into fixed 180-ms windows, each annotated according to the corresponding arrhythmia classes such as N, L, R, A, and V classes (Figure 3).
   NOTE: This segmentation allowed for consistent input length across samples and captured the temporal characteristics within each heartbeat segment.
3. Read raw ECG files of format (.dat, .hea, and .atr) using Python's Library.

3. Normalization

1. To ensure uniformity in signal amplitude, normalize all segmented ECG data using Min-Max scaling, transforming each sample to a range between 0 and 1.
   NOTE: The normalization technique used here is MinmaxScaler⁸.

4. Class balancing with SMOTE

NOTE: A common challenge in arrhythmia classification is the significant class imbalance, where normal heartbeats vastly outnumber abnormal ones.

1. To address the above challenge, apply the Synthetic Minority Over-sampling Technique (SMOTE) to the training set, which generates synthetic samples for minority classes by interpolating between existing instances⁹.
   NOTE: The model's capacity to generalize across all arrhythmia types is enhanced by new synthetic samples, which guarantee that each class is fairly represented.

5. Train-test split

1. After preprocessing, split the data into training and testing sets using a 70-30 split, with stratification to maintain the class distribution across sets.
   NOTE: This split ensured that each arrhythmia class is adequately represented in both the training and testing phases, allowing for a reliable evaluation of the model's performance.

6. Dataset splitting and class distribution

1. Split the final Dataset into training and testing sets, with 70% of the data used for training and 30% for testing.
2. Apply SMOTE to balance the class distribution, ensuring that each arrhythmia type is adequately represented in both the training and testing sets, as shown in Table 2.
   NOTE: In total, 3,966,620 ECG segments are used for training, and 112,575 ECG segments are used for testing the model. This large volume of data, along with the diversity of the arrhythmia types, contributed to the model's robustness in detecting different types of arrhythmias in real-world ECG signals.

7. Methodology

1. Model architecture
   1. To achieve accurate arrhythmia classification, implement a hybrid deep learning model comprising 1D Convolutional Neural Networks (CNNs) and Transformer layers.
      NOTE: Hybrid model architecture is specifically designed to leverage the strengths of both in spatial and temporal features from ECG signals. It combines feature extraction and sequence modelling capabilities.
2. 1D-Convolutional Neural Network (CNN) Blocks
   1. Set up the CNN block to consist of two 1D convolutional layers, each followed by a ReLU activation function and a max-pooling layer. These layers extract spatial dependencies from the input ECG signal. To enhance feature extraction, add more ReLU activation functions after each transformation step in the CNN layers¹⁰.
      NOTE: This configuration enables the model to learn spatial dependencies within each segment effectively.
   2. First Convolutional Layer: Use this layer to apply 32 filters of size 3 to the input signal. The operation is expressed as:
       (1)
      Where y_i represents the output, w_j are the weights, x_(i+j) is the input ECG segment, b is the bias term, and σ is the activation function (ReLU, in this case)¹¹.
   3. Pooling Layer: After each convolution, set up a max-pooling operation to reduce the spatial dimension by a factor of 2, defined as:
       (2)
      NOTE: This operation retains the most salient features while reducing the computational complexity.
   4. Second Convolutional Layer: Use this layer to apply 64 filters of size 3 to the feature maps from the previous layer, extracting higher-level features¹². Apply a ReLU activation function after convolution to introduce non-linearity:
       (3)
      1. Additional Activation Layers: apply ReLU activation at every intermediate stage after convolution to enhance the expressive power of the network, ensuring that the model focuses on positive activations¹³.
   5. Dropout: Specify a Dropout rate of 0.5 in the fully connected layer section¹⁴.
3. Transformer block:
   1. To follow this protocol, set up the Transformer block to include two multi-head self-attention layers, enabling the model to capture temporal dependencies across different parts of the ECG signal¹⁵.
   2. Multi-Head Self-Attention: This mechanism computes relationships between every pair of sequence elements. For a sequence with query Q, key K, and value V, calculate the attention as:
      (4)
      Where Q, K, and V are the query, key, and value matrices, respectively, and d_k represents the dimensionality of the key vectors.
      NOTE: By allowing the model to weigh different segments according to their relevance, this mechanism enhances the model's ability to distinguish between arrhythmias with subtle temporal characteristics.
   3. Activation Functions: Use ReLU explicitly across layers.
4. Continuous Wavelet Transform (CWT):
   1. Before feeding the data into the CNN layers, apply continuous wavelet transforms to convert the ECG signals into time-frequency representations¹⁶.
      ​NOTE: This transformation provides a more comprehensive feature set by capturing the frequency variations over time, which are crucial for distinguishing between different arrhythmia types.
5. Training process:
   1. To train the model, use the Adam optimizer, which dynamically adjusts the learning rate based on the first and second moments of the gradient. Define the update rule for Adam as:
      (5)
      Where m_t and v_t are the first and second moment estimates, α is the learning rate, and ε is a small constant to prevent division by zero¹⁷.
   2. Conduct training over 100 epochs, with a batch size of 64, utilizing early stopping to mitigate overfitting.
      ​NOTE: The training data were fed into the model via the PyTorch DataLoader utility, and both dropout and batch normalization were applied to regularize the network and improve convergence.
6. Hyperparameter tuning
   1. Perform manual tuning using early stopping, Adam optimizer, dropout, batch normalization, and a batch size of 1024.
      ​NOTE: While grid or Bayesian search is not explicitly mentioned in this paper, these choices reflect practical tuning strategies.
7. Validation techniques
   1. Use a 70-30 stratified train-test split after applying SMOTE¹⁸.
      NOTE: While k-fold is not used, stratification preserves class distribution, which is often sufficient with large datasets.

Results

Performance metrics of the proposed model
The accuracy, sensitivity, specificity, and F1-score of the proposed model are calculated for each arrhythmia class. The model's performance is evaluated on the MIT-BIH and other pertinent ECG databases. Key results are summarized as follows:

Accuracy: The hybrid CNN-Transformer model achieved an accuracy of 99.32% on the MITDB dataset and 97.15% on combined databases, demonstrating the model's robustness across different ECG sourc...

Discussion

This study's results indicate that the hybrid CNN-Transformer model proficiently classifies ECG arrhythmias with elevated accuracy, sensitivity, specificity, and F1-score, markedly surpassing conventional CNN-only and CNN-LSTM models. Incorporating Transformer layers has enhanced the model's ability to capture temporal dependencies, a crucial element of ECG analysis. Moreover, continuous wavelet transformations (CWT) provide extensive time-frequency characteristics, enabling the CNN layers to differentiate between subtle...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Computer system			(For training) Processor: AMD Ryzen 7 7840HS, CPU RAM:16 GB, GPU RAM:6GBNVIDIA GeForce RTX 3050
imbalanced-learn			python package used for resampling
pytorch			PyTorch is a Python package that provides two high-level features: - Tensor computation (like NumPy) with strong GPU acceleration - Deep neural networks built on a tape-based autograd system
seaborn			Seaborn is a Python visualization library based on matplotlib.
wfdb			used for reading ,writing, processing, and plotting physiological signal and annotation data