Innovative Hybrid CNN-Transformer Deep Learning Models For The Automated Diagnosis Of Monkeypox From Medical Images

Summary

The use of a curated collection of 2,280 skin-lesion images produced a standardized binary deep-learning workflow to classify the presence of mpox in each image. A custom and InceptionV3, ResNetV2, ResNet50, DenseNet121, and a hybrid CNN-transformer model were compared in the context of a common preprocessing and training pipeline.

Abstract

The visual appearance of monkey pox skin lesions remains poorly established due to their similarity to other vesicular and pustular skin diseases. This paper tested the hypothesis that a hybrid deep-learning model, integrated with convolutional and transformer-based feature encoders, can automatically classify mpox images within a unified experimental pipeline. To ensure consistency across models and reports, the main analysis was conducted as a binary task: mpox vs other conditions with the same or similar presentation. These benchmark data consisted of 2,280 images: 1,020 mpox and 1,260 non-mpox lesion images. The images were downsampled to a standard input size of 150 × 150 pixels, and the values were in the range [0, 1]. During training, the images were augmented with rotation, translation, shear, zoom, and horizontal flipping. A custom Sequential CNN, InceptionV3, ResNetV2, ResNet50, DenseNet121, and a proposed hybrid CNN-transformer architecture were compared. Every baseline model was trained for 15 epochs using the Adam optimizer and binary cross-entropy loss. Training and validation accuracy, loss, precision, recall, F1-score, and area under the receiver operating characteristic curve were measured to assess performance, where possible. The best reported internal validation accuracy was observed with the Sequential CNN, and the hybrid model achieved a precision of 98.50, a recall of 98.50, and an F1-score of 98.58, yielding a balanced discriminative profile. InceptionV3 shows signs of overfitting, and ResNetV2 did not provide sufficient validation data to support a robust generalization test. On the whole, the hybrid model can be viewed as a viable and balanced strategy and not necessarily better than all baselines.

Introduction

Automated and fast detection of mpox in medical cases is clinically significant since the disease can be similar to a variety of other vesicular or pustular eruptions at the initial visual evaluation. Clinicians in practice do not pay much attention to the morphology of lesions alone, but they combine pattern, distribution, history, exposure risk, epidemiological context, and laboratory confirmation¹. Nonetheless, computational tools use images and can also perform a helpful triage procedure in environments where skilled dermatological review is constrained or delayed. It is especially applicable when it comes to outbreak tracking, teledermatology screening, and the settings that are resource-limited and have a significant number of possible cases that should be prioritized to undergo confirmatory testing.

The deep-learning-based approaches have also emerged as appealing to this task since they can discover discriminative images of lesions without the use of descriptors that need to be created manually². The most popular review on the literature still belongs to convolutional neural networks due to their effectiveness in local texture, color range, and lesion margin recognition³. Recent work has further developed this paradigm with the help of transfer learning, attention mechanisms, and transformer modules, which try to model a larger spatial dependence of the image. Nonetheless, the literature on mpox has high reported performance that should be taken with caution. A large variety of studies have made use of sparse public datasets, have different sets of heterogeneous classes, or compare results with a variety of preprocessing and validation regimes⁴. Due to this, there is frequently a dilemma of whether performance increases are the result of a truly superior model, a desirable split, vigorous augmentation or disparities in the formulation of the underlying task.

Mpox remains one of the significant infectious diseases with adverse implications on public health, as its lesions involving skin infections can seemingly be confused with other vesicular and pustular lesions on the interface during eye screening in the course of the primary examination⁵. Practically, clinicians do not make their diagnosis of mpox based on morphology: pattern, distribution, history, epidemiology, and laboratory confirmation are components of the diagnosis decision⁶. Nevertheless, they can play a beneficial role in the triage process with the help of computational tools that rely on the images, particularly when the experience of the dermatologists is either inadequate, slow, or not available. They might be applicable in teledermatology, in cases where an outbreak is to be addressed with screening, and in places with resource limitations where suspected cases on the priorities list must be quickly addressed⁷.

Deep-learning-based methods, because they allow learning discriminative lesion representations, would be attractive to this task because they learn such representations without handcrafted descriptors, depending on the characteristics of the input images⁸. It has been a frenzy within the field of dermatological images to utilize convolutional neural networks (CNNs) in analyzing images, as they possess the ability to encode local texture, lesions' periphery, and exquisite morphology meaning. Later studies have provided this paradigm through transfer learning, attention, and transformer-based submodules to learn more general spatial relationships and contextual trends⁹. Nonetheless, the literature on the mpox imaging is still heterogeneous. It is not uncommon that performance is reported to be based not only on the architecture selection, but also based upon dataset structure, augmentation plan, classes definition, and validation structure¹⁰. Then, good headline performance does not necessarily indicate increased generalization.

In the current work, this issue is solved by using a more perceptible and internally consistent binary mpox image-classification benchmark¹¹. It does not discuss the contribution in the format of proposing an entirely new hybrid architecture, as CNN-transformer methods have already been sprinkled throughout the literature¹². Rather, the experiment introduces a standard comparison with explicit primary binary decomposition, where the extra multiclass pieces of evidence, preprocessing/training, and model performance are considered in a protocol-friendly way, and model performance is considered in a conservative way compared to experiment environment¹³. In this paradigm, a CNN-transformer model can still be of functional importance since CNNs are optimally structured to capture local lesion information, whereas transformer-based representations have the potential to represent long-range spatial structure of interest in dermatology as well.

Recent image-classification research of mpox can be divided into three widely streams of research that are founded on a methodology. The latter is based on the conventional CNNs and transfer-learning because of its maturity, computation speed, and recreation of lesion-specific textures¹⁴. The latter explores attention-based or transformer models, better suited to learn the global lesion context. The third integrates convolutional and attention in the hybrid pipelines to preserve the local sensitivity but use bigger body contextual modeling. Several of the studies report high performance which cannot be easily directly compared because the studies differ in the size of their dataset, curation strategy, structure of their classes, preprocessing and validation protocol¹⁵. They are binary classification oriented, multiclass dermatological discrimination oriented not all of which have the same measures of evaluation. Mpox image datasets made publicly accessible might also include near-duplicate or very similar images, which will require apparent performance to be inflated in case split control is not adequate¹⁶.

The given analysis is founded on a controlled binary set of 2,280 photographs of skin lesions, 1,020 photographs of mpox and 1,260 photographs of Other were planned. The Other category overlaps with the non-mpox lesions that appear to the naked eye, and this puts the task a clinically interesting early triage formulation, as compared to a complete dermatological localization system¹⁷. The images were all resized, normalized, standardized, and augmented and evaluated in a typical inner training-validation pipeline. The main objective was the comparison of the behavior of conventional and hybrid deep-learning models in a transparent benchmark and a conclusion on whether the hybrid architecture suggested offers a superior balance between discriminative and interpretability, relative to popular baselines.

The definite gap that the current revised version addresses is thus not the mere presentation of the hybrid model, since the strategies of hybrid CNN-transformer have already been covered in the literature. Instead, the disconnect is that what was needed was a more distinct, internally consistent metric that disregards the primary binary task and exploratory multiclass outputs, characterizes the preprocessing and training pipeline in a reproducible way, and presents the model performance in a manner that does not overstate novelty or clinical preparedness¹⁸. In that regard, a hybrid architecture is nevertheless worth considering, since CNNs excel at local feature extraction, and transformer-like attention can, in theory, model global lesion configurations and longer-range spatial connections to aid in visual diagnosis¹⁹. The practical query is not on whether a hybrid design will have a favorable balance of discrimination, stability, and interpretability as compared to typical baselines on the same curated data aspect, but on whether it provides a good balance of taxation, stability, and interpretability.

The latest works on mpox image analysis can be divided into three general streams of the study of methods. The former one uses traditional CNNs and transfer-learning architectures due to their maturity, computational efficiency, and ability to learn lesion-specific textures and local morphological features²⁰. The second stream offers the vision-transformer or attention-enhanced models to be able to capture the wider spatial associations over the lesion field, particularly when the texture patches are composed of contextual lesion distribution, and, thus, they may be as important as individual texture patches. The third stream adds the convolutional and the attention to support hybrid pipelines in an attempt at maintaining the local sensitivity of CNNs but utilizing the contextual modeling ability of attention blocks.

Even though most of these works claim excellent accuracy, cross-study comparison is not an easy task due to the differences in size, curation strategy, class composition, and validation design across datasets²¹. There are publications which assess binary discrimination, and those which assess multiclass dermatological categorization some are also balanced in reporting precision and recall, and some focus on accuracy²². Also, reported mpox datasets can be based on online image collections, curated collections, or augmented subsets, which can boost reported performance when train-validation leakage or near-duplicate images are not well managed. Table 1 is thus not kept as a mere compendium of previous work, but rather as a schematic map of model families, dataset constraints, and the opportunities offered by the methods available in this research.

The methodological value of the given work is reflected in a more limited and more defensible manner. This paper will discuss the necessity of the transparent binary benchmark with the help of the provided curated dataset, report the preprocessing and training procedure and directly compare the proposed hybrid model with popular baselines²³. These changes also separate the main binary experiment and the supplementation multiclass evidence in a way that allows performance claims to be associated with the appropriate experimental context²⁴. The difference makes a decisive difference in the discursive analysis of model conduct, reported measures, and practicality.

The main experiment of this script is a binary classification experiment on a smaller set of 2280 images of skin lesions, which was curated by hand. In this dataset, the number of images that were identified as mpox was 1,020, and those that were categorized as other were 1,260. The other category entails the presentation of non-mpox related dermatological appearances that are of a visual character, mostly lesions where the manuscript refers to them as chickenpox- and measles-like lesions²⁵. This dual nomenclature makes this binary grouping clinically significant as a primitive screening formulation because the model inquires whether the model is able to disaggregate suspected mpox and visually confusing alternatives, but is also less complex than a full multiclass dermatology benchmark.

The training of the models had to be standardized, with all images preprocessed prior to being trained. Resizing, intensity normalization, augmentation, and conversion of categorical to binary labels are also mentioned in the manuscript. However, in the updated meaning these steps are regarded as components of a reproducible pipeline and not as notes written at the periphery. Its array was further split into training and validation subsets consisting of directories, and the mentioned performance metrics should then be seen as internal validation performance impacts instead of as the indicators of a completely independent external testing group. To be clearer, this 2,280-image collection will be taken as the official data on which the main analysis will be conducted. The binary framing was chosen because it reflects the early triage question common in practice: is the image of a suspicious lesion more likely to be mpox than other visually similar conditions? This framing is limited compared to an extensive dermatology diagnosis, but it is a valid and technically interpretable initial starting point for comparative benchmarking.

The updated description of the dataset also considers the fact that the paper includes some additional outputs obtained with the help of different class structures or some enriched subsets, which are shown in Figures 1A–I. Instead of deleting those materials, the manuscript now contextualizes them directly to allow the readers to view the results that are part of the binary benchmark and the ones that are part of secondary experiments. This method will maintain the entire documentation of this study, but will not allow the inclusion of any incompatible experiments to be integrated into a single assertion.

Protocol

NOTE: Use the curated binary mpox image dataset described in the manuscript for the main experiment. The dataset contains 2,280 skin-lesion images, including 1,020 mpox images and 1,260 images labeled as Other. Use secondary image analysis only; do not perform direct patient recruitment, clinical intervention, animal experimentation, or generation of new biological specimens as part of this workflow.

1. Obtain and curate the dataset

1. Define a binary labeling scheme with two classes: mpox and Other.
2. Assign visually similar non-mpox lesions to the Other class.
3. Screen all images for readability, class identity, and suitability for supervised image classification.
4. Exclude unusable or ambiguous images during curation.
5. Organize the curated images into directory-based class folders for downstream loading with image generators.

2. Resize and normalize the images

1. Resize all images to 150 x 150 pixels before model training.
2. Apply bicubic interpolation during resizing to preserve smooth image transitions.
3. Rescale pixel intensities by dividing each pixel value by 255.
4. Normalize all image intensities to the range [0,1].
   NOTE: Use the same spatial resolution and normalization procedure for all models to maintain a fair comparison across architectures.

3. Augment the training images

1. Apply augmentation only to the training subset.
2. Set the rotation range to 20°.
3. Set the width-shift range to 0.2.
4. Set the height-shift range to 0.2.
5. Set the shear range to 0.2.
6. Set the zoom range to 0.2.
7. Enable horizontal flipping.
8. Use nearest-neighbor filling for pixels introduced during augmentation.
9. Perform augmentation online during training rather than by permanently duplicating image files.
   NOTE: Do not augment the validation images; rescale them only.

4. Encode the labels and split the dataset

1. Encode the two classes as binary labels.
2. Load the images using directory-based generators in binary class mode.
3. Set the batch size to 32.
4. Divide the dataset into training and validation subsets.
   NOTE: The manuscript reports internal training-validation benchmarking only and does not include an independent external test cohort.

5. Build the baseline CNN models

1. Construct a Sequential CNN baseline with three convolutional blocks.
2. Use 32 filters in the first convolutional block and follow the block with max pooling.
3. Use 64 filters in the second convolutional block and follow the block with max pooling.
4. Use 128 filters in the third convolutional block and follow the block with max pooling.
5. Flatten the extracted feature maps.
6. Add a dense layer with 512 units.
7. Apply dropout with a rate of 0.5.
8. Use a sigmoid output layer for binary classification.
9. Include InceptionV3, ResNetV2, ResNet50, and DenseNet121 as comparative baseline architectures.
   NOTE: Use the same curated binary dataset and preprocessing pipeline for all baseline comparisons wherever applicable.

6. Define the hybrid CNN-transformer model

1. Use a CNN module as the local feature encoder.
2. Extract lesion texture, edge, and local morphological patterns with the CNN component.
3. Use a transformer-based module as the context encoder.
4. Model broader spatial dependencies in the learned representation with the transformer component.
5. Project the CNN and transformer feature vectors to the same dimensionality and fuse them via element-wise addition.
6. Map the fused representation to a binary mpox-versus-Other output through a classification head.
   NOTE: Report the hybrid architecture at the module level if the number of attention heads, embedding dimensions, and transformer layers is not explicitly available.

7. Training process and appraisal practice

1. Assemble the binary classifier using the implemented model with the Adam optimizer, binary cross-entropy loss, and accuracy as the primary training metric.
2. Train the clearly defined baseline model for 15 epochs on the curated mpox-versus-other dataset.
3. Conduct all experiments in a Python-based deep-learning environment on a GPU-equipped system, and summarize the software resources and generalized computational environment in Table of Materials.
4. Monitor training and validation loss curves during training, and compute accuracy, precision, recall, F1-score, and AUC for each individual model when available.
5. Interpret model performance by emphasizing consistency of reporting and generalization behavior, and prioritize a balance between sensitivity and specificity over single peak accuracy values.
6. Formulate two evaluation questions for each model: (i) how well the model fits the training data and (ii) how well the learned representation transfers to held-out validation images from the same curated source.
   NOTE: Present these two aspects explicitly in the manuscript to distinguish representational capacity from useful generalization.
7. Interpret high training accuracy as evidence of sufficient representational capacity, and interpret high and stable validation performance as the more meaningful indicator of useful generalization.

8. Experimental analyses

1. Report only results that are directly supported by the recorded outputs, and avoid extrapolating beyond the documented evidence.
2. Explicitly state limitations wherever validation information is missing or where a table reflects a different class structure.
3. Treat the binary mpox-versus-other classification task as the primary analysis and use it as the internal reference for all experiments.
4. Translate the original implementation code into a narrative protocol and into Table 2 so that the dataset, software environment, model families, and training configuration are concisely summarized for reproducibility.
5. Generalize the descriptions of hardware and software so that the focus remains on methodological reproducibility rather than on a single vendor-specific platform.
6. Present model-wise results in the same sequence as the architectures are introduced, to simplify comparison of training behavior, validation behavior, and reporting completeness.
7. Describe the Sequential CNN baseline as achieving very high internal validation performance while cautioning that this may reflect favorable split characteristics and may not generalize externally.
8. Interpret the Sequential model as a strong internal reference rather than a definitive demonstration of deployable diagnostic robustness.
9. Justify retaining separate InceptionV3 panels because loss and accuracy trajectories together reveal effective training optimization but inconsistent validation behavior.
10. Describe ResNetV2 as an informative but partially reported baseline because comparable validation metrics were not available.
11. Treat the ResNet50 multiclass experiment as supplementary evidence distinct from the primary binary benchmark.
12. Interpret DenseNet121 as an intermediate benchmark with competitive but not dominant performance on the reported binary subset.
13. Describe the hybrid CNN-transformer as achieving balanced class discrimination rather than a one-sided gain in a single metric.
14. Avoid claiming universal superiority of the hybrid model over all baselines because some models were evaluated under different reporting conditions.
15. Position the hybrid model as a promising trade-off between high validation performance and metric balance, and recommend further evaluation on standardized externally tested benchmarks.

Results

Comparison beyond headline accuracy

These results show that comparisons of models in medical image analysis should not only focus on headline accuracy, but also on the behavior of the architectures with regard to a number of metrics and different experimental settings. Throughout the experiments, the models were not similarly behaving: the simple Sequential CNN was extremely high-performing on the internal validation split, InceptionV3 overfitted, ResNetV2 was not sufficiently...

Discussion

The key result of the study is that binary classification of mpox versus visually similar lesions, named automated classification, is possible with the curated dataset that was used in this study, but such a success largely relies on the interpretation of evidence²⁷. Separating the experiments appropriately, the manuscript presents evidence that both conventional and proposed hybrid architectures can be used to perform strong performance²⁸. It is thus inappropriate to view ...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Dataset	Curated mpox image dataset; 2,280 images for the main binary benchmark (1,020 mpox and 1,260 Other)	Primary training and internal validation source
Task definition	Binary classification for the core study; supplementary multiclass and augmented summaries retained separately	Ensures consistent interpretation of reported metrics
Programming environment	Python-based notebook workflow	Data handling, model training, and plotting
Deep-learning framework	TensorFlow / Keras implementation reported in the submitted code	Model development and optimization
Image utilities	ImageDataGenerator with rescaling and augmentation; plotting libraries for training curves	Preprocessing, augmentation, and visual reporting
Baseline architectures	Sequential CNN, InceptionV3, ResNetV2, ResNet50, DenseNet121	Comparative benchmarking
Proposed architecture	Hybrid CNN-transformer classifier	Primary methodological contribution
Training settings	Input size 150 x 150; batch size 32; Adam optimizer; binary cross-entropy; 15 epochs reported for supplied baseline code	Reproducibility of the core workflow
Compute environment	GPU-enabled computational system; hardware details generalized in the narrative	Model acceleration during training