FusionNetX: A Deep Feature Fusion Model Leveraging MRI and Deep Learning for Enhanced Brain Tumor Detection

Summary

This research work presented an approach, FusionNetX, by enriching the feature fusion capability of DenseNet121 and EfficientNetB7 for the classification of brain tumors on five publicly available datasets with data augmentation, preprocessing, and hybrid model customization (intermediate Fusion), achieving results ranging from 98.77% to 99.89%, outperforming across multiple evaluation parameters.

Abstract

Brain tumors are considered one of the deadliest diseases in the world, and misdiagnosing them puts patients’ lives in danger and lowers the survival rate. Magnetic resonance imaging (MRI) with deep learning-based techniques, especially convolutional neural networks, is crucial for overcoming this obstacle, as it enables a more detailed examination of the brain's internal structure and offers outstanding learning and predictive capabilities. The need for accurate diagnosis of the tumor in the brain is still significant. Therefore, an enhanced feature-fusion-based DL model (FusionNetX) is proposed that leverages the strengths of two pretrained models, DenseNet121 and EfficientNetB7, using customized fine-tuning hyperparameters, including trainable and non-trainable layers. The proposed FusionNetX model was applied to five publicly available brain tumor datasets: Br35H, Figshare, Sartaj, Masoud, and Balanced Brain Tumor. Some preprocessing steps were applied to improve image quality and address overfitting through resizing and data augmentation. Using a variety of performance evaluation metrics, such as accuracy, loss, positive predicted value, negative predictive value, true positive rate, true negative rate, F1-score, false negative rate, and false positive rate, the proposed model FusionNetX was found to have an accuracy of 99.33% for Br35H, 99.13% for Figshare, 99.89% for Masoud, 99.19% for BBT-Dataset, and 98.77% for Sartaj. Additionally, the five brain tumor datasets were evaluated using late fusion and attention-based fusion to demonstrate the significance of the proposed FusionNetX model, and the results were substantially better, with gains of 0.3% to 1.9% across all datasets. Furthermore, the receiver operating characteristic curve is computed, and results from various performance evaluation metrics indicate that the proposed FusionNetX model can accurately detect and predict brain tumors and help health practitioners make timely decisions.

Introduction

Cancer is the deadliest disease in the human body nowadays because of its severity and growing rate, while multiple cancerous diseases, like brain tumors, breast cancer, lung nodules, kidney cancer, and others, are found in the human body, causing an increase in the death rate. Cancer develops when cells or tissues in a person's body multiply abnormally.  When cancer metastasizes, it can damage other human organs and become more dangerous¹. Because the brain controls every bodily function, any damage to it can have far-reaching consequences for the entire body.  That is why brain tumors or cancer are so lethal to humans

Protocol

The most threatening disease in the world is a brain tumor. An accurate diagnosis can be beneficial to patients’ survival. Still, there is a need for an accurate diagnostic system that can detect brain tumors at an early stage. To address this problem, a more reliable technique for accurately detecting brain tumors in their early stages has been proposed by using a hybrid dual deep learning mechanism with optimized hyperparameters and fine-tuned layers based on the DenseNet121 and EfficientNetB7 models. The proposed approach takes 2D MRI images as input and outputs a prediction of either tumor exit or no tumor exit for the binary classification dataset Br35H, and tumo....

Results

This part details the results of testing the proposed dual feature fusion model on five open-source datasets on brain cancers, including Br35H, Figshare, Sartaj, Masoud, and Balanced Brain tumor datasets, and their performance is evaluated regarding different statistical performance evaluation parameters, including accuracy^42,43,44, misclassification rate (MCR)⁴⁵, precision, also referred as PPV

Discussion

As concluded, the proposed model worked as follows: First, the image size was chosen as 224 × 224, which is the moderate size for any deep learning model to extract and learn suitable features from the data samples while keeping all important information, and further performed data augmentation, which is more beneficial to make data samples in more diverse directions, which plays an important role for models to extract features from different planes or angles, and the rotational range was selected as 7% with a width.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Br35H Brain Tumor Dataset	Kaggle (dataset contributor)	https://www.kaggle.com/datasets/ahmedhamada0/brain-tumor-detection	Public dataset; binary classification (tumor / no tumor)300 MRI images
Figshare Brain Tumor Dataset	Kaggle / Figshare	https://doi.org/10.6084/m9.figshare.1512427	Public dataset; multiclass (glioma, meningioma, pituitary tumor) 3064 MRI images
Sartaj Brain Tumor Classification MRI Dataset	Kaggle (dataset by Sartaj Bhuvaji)	https://doi.org/10.34740/kaggle/dsv/12745533	Public dataset; multiclass (glioma, meningioma, no tumor, pituitary tumor) 3264 MRI images
Masoud Brain Tumor MRI Dataset	Kaggle (dataset by Masoud Nickparvar)	https://doi.org/10.34740/kaggle/dsv/14832123	Public dataset; multiclass (glioma, meningioma, no tumor, pituitary tumor) 7023 MRI images
BBT-Dataset (Brain Tumor Dataset)	Kaggle (dataset contributor)	https://doi.org/10.34740/kaggle/dsv/6758053	Public dataset; balanced multiclass brain tumor 5248 MRI images
Python	Python Software Foundation	https://www.python.org	Programming language, version 3.10.13
TensorFlow / Keras	Google / TensorFlow developers	https://www.tensorflow.org	Deep learning framework; model construction, training, and evaluation, version 2.15.0
EfficientNetB7 (pretrained)	Google / Keras Applications	https://keras.io/api/applications/efficientnet/	ImageNet-pretrained backbone; feature extraction
DenseNet121 (pretrained)	Keras Applications	https://keras.io/api/applications/densenet/	ImageNet-pretrained backbone; feature extraction
scikit-learn	scikit-learn developers (NumFOCUS)	https://scikit-learn.org	Label encoding, train-test split, evaluation metrics (classification report, confusion matrix, ROC/AUC)
OpenCV (cv2)	OpenCV team	https://opencv.org	Image loading and resizing
NumPy	NumPy developers (NumFOCUS)	https://numpy.org	Numerical computing and array operations
pandas	pandas development team (NumFOCUS)	https://pandas.pydata.org	Data organization and metrics tabulation