Combined Weighted Feature Correlation Approach for Enhanced EEG-Based Emotion Recognition Across Diverse Datasets

Summary

The paper is centered on EEG-based emotion recognition, which is an emerging domain in neuroscience and emotional computing. The traditional linear methods are insufficient as it is impossible to represent complicated emotional structure in video. To address this, the authors propose a new model, Combined Weighted Feature Correlation (CWFC), that capture the emotional relationships. After band pass filtering, Independent Component Analysis (ICA), Discrete Wavelet Transform (DWT) and Fast Fourier Transform (FFT) are employed by the model to extract non-linear features from EEG. In this case best feature is selected by LSTM and these features are evaluated by Random Forest Classifier. Introduction Generative Adversarial Network (GAN) based data augmentation gives 88% (for valence) and 86% (for Evaluation of Test Results The proposed GAN based arousal) accuracy on the DEAP dataset which is very high in data augmentation has been able to improve the performance when performance. The model validates that CWFC is beneficial across datasets, as it delivers an overall classification accuracy of 89% for emotional states on the SEED dataset.

Abstract

Emotion recognition using EEG Signals is an active area of research in affective computing and neuroscience, aiding scientists into their quest to understand the brain processes involved in emotions. In contrast to the existing works which focused on linear EEG features, the present work concentrates on non-linear features in order to better characterize the finer changes of emotional responding. Inspired by this, in current paper, proposed method is called Combined Weighted Feature Correlation (CWFC) to investigate the effect of non-linear components on the performance of emotion detection. As a first step, the EEG data is collected and pre-processed using bandpass filters to separate the frequency bands such as beta, alpha, gamma, delta, and theta. To detect intricate emotional patterns, features extraction is performed by Independent Component Analysis (ICA), DWT (Discrete Wavelet Transform) and FFT (Fast Fourier Transform). These features can be built as combined features into the CWFCC model to improve its power as well as its overall predictive accuracy. Then apply a Random Forest classifier to see how significant are these combined features. Proposed work integrates GAN data augmentation after optimal feature selection (with an emphasis on LSTM). On the DEAP dataset, this augmentation results in astonishing 88% valence accuracy and 86% arousal accuracy, which improves the recognition accuracy of emotion remarkably. Proposed model achieved valence and arousal accuracy of 62% and 65%, respectively, on the early DEAP without GAN data augmentation. Moreover, this model can distinguish between SEED dataset during Neutral emotional states, Negative, and Positive with an average accuracy of 89%. The proposed paper proclaims the superior performance of CWFC model with GAN based data augmentation on EEG for emotion recognition over different datasets through in-depth analysis and comparative study.

Introduction

Emotion recognition is important in human interaction and human communication. In order to establish successful interpersonal connections, one must be able to perceive and understand emotions, thus having far-reaching implications in domains as diverse as psychology, neurology, and human-computer interaction¹. In past few years, the electroencephalogram (EEG) based BCI (Brain Computer Interface) has been a merit tool that researchers have focused on to detect emotions. Although EEG-based BCI provide a direct window into the electrical activity of the human brain and an appropriate research tool for exploring emotional states. The automatic and accurate emotional states classification has fundamentally transformed the area of emotion recognition through machine learning techniques and EEG-based BCI².

In the previous work, emotion recognition is introduced and its importance in various fields, including healthcare, human–computer interaction, and neuroscience³. Despite substantial progress, a clear research gap remains in current EEG-based emotion recognition studies. First, approaches introduced in the literature use single-domain feature extraction (e.g., spectral or time frequency features) or singular non-linear descriptors available and isolatable, which do not offer enough for modelling by working only with a partial representation of inherently complex and non-stationary EEG signals. Second, while a variety of non-linear features have reported to have superior discriminative power, they are usually used separately and isolate the relationships among these features across domains. This constraint makes them less capable of modeling the multiscale and source-level dynamics involved with emotional brain activation. Third, most of the deep learning methods are designed for optimizing classification performance but not considering feature interpretability, redundancy reduction, and robustness in the presence of class imbalance. Consequently, cross-dataset generalization is still an open problem between investigations with different electrode setups, sampling rate (128–256 Hz), and recording duration (30–60 s). The main dimensions of emotional intelligence are presented in Figure 1.

To overcome such drawbacks, in this paper, we introduce a Combined Weighted Feature Concatenation (CWFC) framework that aggregates multi-domain non-linear features in one architecture. Instead of treating ICA, DWT and FFT as independent extraction methods like conventional approaches, we model source-level decomposition (ICA), multi-resolution temporal dynamics (DWT) and spectral power representations (FFT) through a weighted correlation. This structure can help the model make use of complementary neural information and reduce redundancy to increase discriminative ability. Also, both feature importance ranking and GAN-based augmentation are employed to achieve the robustness on fewest imbalanced EEG samples. Integrating multi-modal feature relationships and temporal dependencies through LSTM classification, the combined model improves generalization across diverse EEG acquisition conditions. Thus, the major contribution of this work is to overcome the lack of integration between non-linear EEG representations and insufficient generalizability in existing methods and introduced a systematic, reproducible, and stable emotion recognition framework within real-life BCI applications.

Figure 1: Phases of emotional intelligence. Phases of emotional intelligence represents how Emotion recognition from EEG signals can be extracted, and then it’s uses in the diagnosis and monitoring of emotional problems in mental health assessments. Please click here to view a larger version of this figure.

Non-linear EEG features for emotion recognition research offer a more profound understanding of the neural processes involved in affective processing. Being a non-invasive, high temporal resolution technique which is sensitive to fast neural dynamics, EEG has been broadly used for the study of brain activity related to emotional states. Non-linear methods are especially appropriate for the study of EEG signals, due to their non-stationary and complex nature that constitutes a better model both of its evoked and induced activity in emotional processes. In any number of computational models, emotion is represented with reference to extant theoretical models such as the Basic Emotion Model⁴ (BEM) and the Circumplex Model^5,6. Two structurally and functionally distinct conceptions of emotion are the Basic Emotion model, which assumes that there exists a set of biologically-based fundamental core emotions^7,8 (especially love), and the Circumplex Model that organizes emotion according to two underlying dimensions: valence⁹ (positive–negative) and arousal^10,11 (low–high). The valence–arousal framework shown in Figure 2 is prevalent in EEG-based emotion recognition studies, mainly because of its alignment with the standard datasets and applicability for modeling continuous emotional states. This dimension-based representation offers a systematic and quantifiable means to map neural activity into emotional categories and thus can facilitate machine learning-based classification.

Figure 2: Valence-Arousal Emotion Representation. Two-axis diagram showing emotional states, with valence indicating positive or negative feelings and arousal reflecting low to high emotional intensity. Please click here to view a larger version of this figure.

Arousal is emotional intensity, valence positive or negative. Emotion analysis has been recently focused on EEG for probing neural correlates of emotions. Feature extraction is very important which maps the EEG into quantities in time, frequency and time–frequency domain. Some of the most common nonlinear characteristics are Wavelet Entropy¹², Correlation Dimension¹³, and this proposed study use a combined weighted approach which is a fusion of DWT and ICA and FFT¹³. Standard DEAP¹⁴ and SEED¹⁵ datasets are available for reproducible evaluation. Traditional linear features are often no longer adequate, calling for the design of nonlinear approaches¹⁶. Frequency-band based emotional patterns are shown in Figure 2, and the weights of the contributions of EEG bands are listed in Table 1.

Different Frequency Band	Frequency Band range in Hz	Functions of frequency band in Emotion recognition
Gamma	Greater than 30 Hz	The integration of emotional data and the synchronization of several brain regions during emotional processing could be reflected in gamma activity.
Beta	In between 13 to 30 Hz	Variations in beta power could be a sign of emotional arousal and the motor reactions connected to emotional displays.
Alpha	In between 8 to 13 Hz	Emotional states and variations in alpha power, particularly alpha asymmetry, are related. When emotionally aroused and paying more attention to emotional inputs, decreased alpha power is frequently seen.
Theta	In between 4 to 8 Hz	Emotional experiences can be associated with increased theta power, which may aid in the encoding and retrieval of emotional memories.
Delta	0.5-4 Hz	An increase in delta activity in the context of emotion detection could point to a relaxed or resting emotional state.

Table 1: Functions of EEG frequency band in Emotion recognition along with different range. This table outlines the commonly recognized EEG frequency bands, specifies their respective frequency ranges, and describes their functional significance in emotion recognition research. It highlights how each band is associated with different cognitive and affective processes, providing a foundation for interpreting EEG signals in emotional state classification.

The recent development of EEG-based emotion recognition is stimulated by the breakthroughs in deep learning and nonlinear signal processing. CNN-based models achieved high performance by using electrode selection and hyperparameter tuning¹⁷, while subject-independent CNN model using dynamic differential entropy enhanced wearability and decreased complexity¹⁸. But models for this issue usually cannot be generalized across subjects. In order to model sequence dependencies, attention-based LSTM architectures with domain adaptation mechanisms were proposed^19,20 which require large computation and lack interpretability. Beyond the realm of deep networks, non-linear and topological methods have also received considerable interest. By using persistent homology-based phase space analysis, enhanced discrimination was obtained for subtle emotional states^21,22. Ensemble methods by combining entropy and fractal outputs with various feature selections were known to be improved predictors. Hybrid CNN–LSTM models²³ outperformed direct valence–arousal classification on DEAP²⁴ and multicolumn CNNs exploited the spatial distribution of electrodes effectively²⁵. Topographic Feature Maps^26,27 (TOPO-FM) and Holographic Feature Maps (HOLO-FM) based feature representations were evaluated on a number of publicly available datasets^28,29,30. Despite such results showing substantial advances in classification accuracy, challenges regarding cross-subject generalization and interpretability of the model and robustness over heterogeneity of EEG acquisition conditions still remain. Author³¹ offer an extensive survey on the main challenges, approaches and further directions in this area, focusing on signal quality, feature extraction and scalability in Applied Soft Computing. In a related work³², advanced functional connectivity measure corrected amplitude envelope correlation across different EEG frequency bands in identifying adverse driving conditions highlighting the strength of multi-band connectivity for on-the-fly behavioural monitoring.

In a broader neuroimaging relation^33,36, a strong overview of multimodal learning methods combining different signals such as EEG and facial expressions or physiological data for more reliable emotion detection with reference to demonstrating the potential of modalities integration that considerably raises performance in AI systems. Besides DEAP and SEED, the DREAMER dataset is also a recent addition to representative benchmarks in EEG-based emotion recognition studies. DREAMER was proposed by Katsigiannis and Ramzan³⁷ that contains EEG and ECG signals acquired from subjects who watch the audiovisual emotional stimuli with inexpensive wireless device. The content of the dataset is applicable for dimensional modeling of emotions with valence, arousal and dominance hierarchies in line with the circumplex model of affect. Owing to its well-considered experimental paradigm, DREAMER datasets have been used extensively for the evaluation of deep learning and subject-independent emotion recognition models. Its applications can afford to compare with other databases of dimensional emotion representation, and also shown that it is a valuable alternative in supplementary to the cirrus dance, DEAP and SEED dataset. These datasets are extensively used in affective computing studies and encompass EEG recordings acquired under controlled experimental conditions with standardized emotional stimuli. This dataset consists of 32-channel EEG signals from 32 subjects who were received emotional video clips. The data is annotated concerning valence and arousal. SEED: multi-session EEG recordings of subjects during film-based emotion elicitation experiments with categorical emotion labels. DREAMER is an EEG and ECG dataset recorded during emotion induction via audio-visual stimulus presentation. All datasets were originally collected using standardized protocols, hardware configurations and validation of labelling procedures as reported in their original publications. Here, EEG signals from only these datasets were analysed in this study.

Protocol

The research did not include any direct engagement with human subjects. All datasets utilized in this study were sourced from publicly accessible platforms and were used in accordance with their stated usage policies. As such, formal ethical approval was not necessary.

1. Computational Environment Preparation

1. Prepare a Python environment (version 3.8 or higher) on a standard workstation.
2. Install TensorFlow (v2.x), Keras, Scikit-learn, NumPy, SciPy, MNE, and PyWavelets using pip.
3. Fix the random seed (seed = 42) in NumPy and TensorFlow to ensure consistent experimental results.

2. Dataset Acquisition and Structuring (DEAP, SEED, and DREAMER)

1. Obtain the EEG datasets from the public repositories of DEAP, SEED, and DREAMER. The raw EEG datasets used in this study are available through public records. DEAP dataset (http://www.eecs.qmul.ac.uk/mmv/datasets/deap/), SEED dataset (BCMI Laboratory repository: http://bcmi.sjtu.edu.cn/~seed/seed.html) and DREAMER dataset can be accessed by IEEE Data Port (https://ieee-dataport.org/documents/dreamer-dataset).
2. Extract raw EEG recordings (.dat) for all subjects and trials.
3. Convert the recordings into NumPy array format and arrange them subject-wise and trial-wise for processing.
4. Read in the DEAP dataset and extract EEG recordings for all subjects. Retain a total of 32 EEG channels (from Fp1 to O2) and remove non-EEG physiological signals.
5. Download the SEED dataset and extract all EEG recordings from each experiment. Down-sample to 200 Hz if needed. Apply 0–75Hz bandpass filter and segment the signals into epoch-wise (eeg_1 to eeg_15) arrays.
6. Label each frame with emotion (−1: negative, 0: neutral, +1: positive) and rearrange data in channel × time.
7. Get the DREAMER dataset and retrieve all subjects / trials EEG recordings. Choose the EEG channels available as well as the emotional labels.
8. Categorize every experiment into small windows of specified duration, and reshape the data as subject-wise structured matrices with encoded categorical variables for valence, arousal and dominance scores.
   NOTE: The detailed specifications of the pre-processed DEAP dataset used for analysis are summarized in Table 2, while Figure 3 illustrates the distribution of valence and arousal scores.

Signal recorded on different steps	EEG (Electroencephalogram) device by DEAP dataset
Data taken	24 subjects (s02, s04, s05, s06, s07, s08, s09, s10, s11, s12, s13, s14, s15, s16, s17, s18, s19, s20, s21, s22, s23, s24, s25, s26)
Channel indices	"Fp1", "AF3", "F3", "F7", "FC5", "FC1", "C3", "T7", "CP5", "CP1", "P3", "P7", "PO3", "O1", "Oz", "Pz", "Fp2", "AF4", "Fz", "F4", "F8", "FC6", "FC2", "Cz", "C4", "T8", "CP6", "CP2", "P4", "P8", "PO4", "O2"
Window size	256
No of Video/trial	40
Step size	16 (each 0.125 sec updated once)
Sample rate	128 Hz
Subjects List	32
Emotions	Arousal, Valence

Table 2: DATASET PRE-PROCESSING EXPLANATION. Summary of EEG dataset preprocessing steps, including filtering, artifact removal, segmentation, and normalization, performed prior to feature extraction and model training.

Figure 3: DEAP dataset Valence- Arousal representation of first 40 data. Valence–Arousal mapping of the initial 40 EEG trials from the DEAP dataset, illustrating the emotional state distribution and variability across the two dimensions. This representation provides insight into the spread of emotional responses and supports the feature extraction process for emotion classification. Please click here to view a larger version of this figure.

NOTE: EEG electrode locations across different brain regions are shown in Figure 4. The channel configuration follows the standard scalp EEG electrode placements used for data collection in the DEAP, SEED, and DREAMER datasets. The electrodes are distributed across frontal, temporal, parietal, and occipital regions, allowing coverage of both local and distributed neural activity related to emotional processing.

Figure 4: EEG Signal Electrode Representation. Electrode placement in the EEG 10–20 international system for scalp signal acquisition. Please click here to view a larger version of this figure.

3. Signal Resampling and Window Segmentation

1. Down-sample SEED EEG recordings from 512 Hz to 200 Hz using scipy.signal.resample() to reduce computational complexity. Then segment each trial into fixed-length windows of 256 samples with a step size of 16 samples.
2. Retain the original sampling rates for DEAP and DREAMER (128 Hz datasets). Step size 16 samples (0.125 s overlap at 128 Hz).
3. Segment each EEG trial into windows of 256 samples with a step size of 16 samples to create overlapping segments.

4. Bandpass Filtering and Frequency Decomposition

1. Define the sampling frequency (fs) and set filter order to 2. Compute the Nyquist frequency (0.5 × fs) and normalize the cutoff frequencies.
2. Design a Butterworth bandpass filter using scipy.signal.butter().
3. Apply zero-phase filtering using scipy.signal.filtfilt() to avoid phase distortion.
4. Extract five canonical EEG frequency bands: delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), beta (13–30 Hz), and gamma (30–45 Hz).

5. Artifact Removal Using Independent Component Analysis

1. Apply FastICA from Scikit-learn with the number of components equal to the number of EEG channels.
2. Identify artifactual components using statistical indicators such as variance and kurtosis.
3. Remove the identified components and reconstruct the cleaned EEG signals for further analysis.

6. CWFC-Based Differential Entropy Feature Extraction

1. Compute the variance of each filtered EEG segment along the channel axis.
2. Calculate differential entropy using:
3. Extract entropy values for all frequency bands and EEG channels.
4. Store the resulting entropy features in a structured feature matrix.

7. Discrete Wavelet Transform (DWT) Feature Extraction

1. Perform multi-level wavelet decomposition using the “db4” mother wavelet.
2. Decompose each EEG segment up to level 4.
3. Extract statistical descriptors (mean, variance, energy, entropy) from each decomposition level.
4. Append DWT-derived features to the existing feature matrix.

8. Spectral Feature Extraction Using FFT

1. Apply Fast Fourier Transform using numpy.fft.fft() on each EEG segment.
2. Compute spectral power density across the defined frequency bands.
3. Extract band power values for all channels.
4. Concatenate FFT-based features with previously extracted features.

9. Feature Selection via Random Forest

1. Train a Random Forest classifier with 300 trees using Gini impurity criterion.
2. Set maximum tree depth to 20 and enable bootstrap sampling with out-of-bag estimation.
3. Rank features based on importance scores.
4. Remove features below a predefined importance threshold to reduce redundancy.

10. Data Partitioning

1. Stratified sampling to shuffle the dataset in such a way that class balance across emotional categories is preserved.
2. Split the dataset into 10 equal folds. With every iteration, use 9 folds for training and 1 fold for testing, always ensuring each fold is test set once.
3. Use this approach for each fold and calculate average performance metrics (accuracy, precision, recall, F1-score) plus standard deviation to check model stability.
   NOTE: Perform all splits on a sample-by-sample basis while ensuring that no samples are shared between the training and testing sets within each fold.

11. Data Augmentation Using Generative Adversarial Network

1. Construct a feature-level GAN using the Keras Sequential API. Define fully connected layers for both generator and discriminator networks.
2. Train the GAN for 200 epochs using minority-class feature vectors. Generate synthetic feature samples from the trained generator.
3. Retain only high-confidence synthetic samples as evaluated by the discriminator. Merge the selected synthetic samples with the original training set.

12. LSTM Model Architecture

1. Construct a Sequential LSTM model using Keras. Add the first LSTM layer with 128 hidden units and set return_sequences=true.
2. Add a second LSTM layer with 128 hidden units. Apply dropout (rate = 0.3) after each LSTM layer to prevent overfitting.
3. Add a Dense output layer with SoftMax activation for multi-class classification.

13. Model Compilation and Training

1. Compile the model using the Adam optimizer with a learning rate of 0.001.
2. Use categorical cross-entropy loss. Train the model with batch size = 64 for 80 epochs.
3. Apply early stopping with patience = 10 based on validation accuracy.

14. Performance Evaluation

1. Evaluate the trained model using the independent test dataset. Compute performance metrics including accuracy, precision, recall, and F1-score.
2. Save the trained model weights and evaluation results for reproducibility. Such well-organized pipeline guarantees that the proposed framework could be easily reproduced.

Results

The collection of two different EEG datasets; DEAP and SEED formed the first stage of the analysis. Figure 5 shows the graph of Valence-Arousal of LAHV, HALV, LALV, and HAHV are short form of low arousal high valence, high arousal low valence, low arousal low valence and high arousal high valence respectively. Four categories of psychological response have been reported: HAHV, LAHV, HALV, and LALV. Two fundamental emotional features, valence and arousal, are associated with each state and are measured by their mean and s...

Discussion

In this paper, research conducted a thorough investigation of the EEG-based emotion recognition on two benchmark datasets (i.e., DEAP and SEED) and showed that the proposed CWFC model can effectively capture emotion-related discriminative information. Unlike previous works based essentially on linear descriptions or feature types, the proposed methodology unifies various nonlinear feature extractions that are interconnected and leverages them via a weighted correlation model to infer functional relationships. This multi-...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
DEAP Dataset	Queen Mary University of London	v1.0	EEG emotion dataset
DREAMER Dataset	IEEE DataPort	Public Release	EEG emotion dataset
Keras	Google LLC	2.x (TensorFlow backend)	Neural network modeling
Workstation	Lenovo	Intel Core i7, 16 GB RAM, 8 GB NVIDIA GPU	Model training and computation
MNE-Python	MNE Developers	1.x	EEG preprocessing
NumPy	NumPy Developers	1.x	Numerical computation
Operating System	Microsoft Corporation	Windows 11 (64-bit)	Computational platform
Python	Python Software Foundation	3.8+	Programming environment
PyWavelets	PyWavelets Developers	1.x	Wavelet decomposition
Scikit-learn	scikit-learn Developers	1.x	Machine learning utilities
SciPy	SciPy Community	1.x	Signal processing and resampling
SEED Dataset	BCMI Laboratory, Shanghai Jiao Tong University	Public Release	EEG emotion dataset
TensorFlow	Google LLC	2.x	Deep learning framework