HMP-MUNet: A Hybrid Deep Learning Framework for Automated Skin Lesion Segmentation in Dermoscopic Images

Summary

HMP-MUNet introduces a hybrid deep learning framework for automated skin lesion segmentation. By integrating State-Space Models with multi-scale attention mechanisms, it enhances segmentation accuracy while optimizing computational efficiency, offering a robust solution for skin cancer diagnosis in clinical settings.

Abstract

Computer-aided diagnosis systems for cutaneous lesions face significant challenges in achieving both high diagnostic accuracy and computational efficiency. Current deep learning approaches often require substantial computational resources while struggling to capture the complex morphological variations inherent in skin lesions across different scales. High-order Multi-scale Parallel Vision Mamba U-Net (HMP-MUNet), a novel deep learning framework that addresses these limitations through an innovative architectural design combining State-Space Models with advanced multi-scale processing capabilities.

The approach described in this study integrates a hybrid U-Net framework that combines a high-order vision state-space module for global context modeling, a multi-scale dilated attention fusion network for hierarchical feature extraction across multiple receptive fields, and a parallel multi-depth flexible network for computational optimization. This architecture uses a modified U-shaped encoder-decoder with increased channel dimensions and advanced attention mechanisms to enhance feature learning.

Comprehensive evaluation on benchmark datasets shows a Dice Similarity Coefficient of 95.85% on PH2 and 90.44% on ISIC2018. The model accomplishes this superior accuracy using only 7.61M parameters, representing a remarkable 72.2% reduction compared to existing Vision Mamba architectures while maintaining high segmentation accuracy. The lightweight design shows potential for deployment across various clinical settings and imaging modalities, pending clinical validation, from specialized dermatology centers to primary care facilities.

HMP-MUNet provides an automated solution for skin lesion segmentation that improves diagnostic consistency and supports clinical workflows, with potential for further validation in clinical use. The integration of high-order modeling capabilities with practical deployment considerations offers a potential solution for improving skin cancer screening protocols and expanding healthcare accessibility in computer-aided diagnosis applications.

Introduction

Computer-aided diagnosis (CAD) systems aim to improve medical imaging practices across clinical specialties, enhancing disease detection, diagnosis, and treatment planning approaches¹. In dermatology, the integration of artificial intelligence (AI) and advanced imaging techniques has emerged as a critical tool for improving diagnostic accuracy and clinical outcomes, particularly in early detection of skin cancer, which accounts for approximately 90% of all skin malignancies². The development of sophisticated algorithmic approaches for automated cutaneous lesion analysis contributes to the advancement of precision medicine, enabling standardized, reproducible diagnostic support that enhances clinical decision-making across diverse healthcare settings^3,4.

Modern dermatological imaging encompasses multiple modalities, including dermoscopy, digital photography, optical coherence tomography, and multispectral imaging systems⁵. Medical professionals utilize dermoscopic images to manually diagnose skin cancer through labor-intensive and time-consuming methods that demand considerable expertise⁶. The challenge of maintaining diagnostic consistency across different clinical settings and imaging conditions has driven the development of CAD systems providing objective, quantitative analysis of cutaneous lesions. The segmentation of cutaneous lesions from dermoscopic images constitutes a fundamental preprocessing step that directly impacts subsequent classification accuracy and clinical utility⁷. Generally, computer-aided skin cancer diagnosis involves five steps: image acquisition, preprocessing, segmentation, feature extraction, and classification⁸. Precise boundary delineation is essential for accurate lesion characterization, enabling clinicians to assess morphological features such as asymmetry, border irregularity, color variation, and diameter—key parameters in established diagnostic criteria⁹.

The emergence of State-Space Models (SSMs), particularly the Mamba architecture, offers linear computational complexity, improving computational efficiency while maintaining superior long-range dependency modeling capabilities¹⁰. Recent advancements in skin lesion segmentation¹¹, such as U-Net⁹, Att-UNet¹², UTNetV2¹³, and UNet++¹⁴ have demonstrated notable improvements in segmentation accuracy. For example, U-Net achieves a Dice Similarity Coefficient (DSC) of 87.55% on the ISIC2018 dataset, while UNet++ reaches 87.83%, and Att-UNet achieves 87.91%. On the PH2 dataset, U-Net reaches a DSC of 90.6%, while C2SDG¹⁵ and SCR-Net¹⁶achieve 90.3%and 89.89%, respectively. HMP-MUNet improves upon these models by integrating a multi-scale attention mechanism and parallel processing capabilities, reducing the parameter count by 72.2% while achieving superior accuracy with a DSC of 95.85% on the PH2 dataset and 90.44% on the ISIC2018 dataset. Given the importance of balancing visual representation learning capabilities with computing resource consumption, exploring generic medical image classification architectures that achieve optimal trade-offs is significant for intelligent clinical diagnosis system development¹⁷. The structured spatial SSMs framework enhances traditional Mamba architectures by optimizing state transition matrices, improving computational efficiency, and reducing memory overhead—characteristics essential for clinical deployment across diverse imaging modalities¹⁸.

Recent research demonstrates that Vision Mamba architectures, particularly those utilizing High-order Vision Mamba-based Switching Scheme (H-VSS), achieve superior performance with significantly reduced parameter counts compared to conventional transformers, making them particularly suitable for clinical workflow integration¹⁹. However, existing Vision Mamba implementations face challenges, including memory consumption limitations and scalability constraints under clinical deployment conditions²⁰. Current medical image segmentation methods are generally divided into CNNs and transformer-based models, where traditional CNNs are limited by receptive field constraints, making long-range dependency capture challenging²¹. In medical imaging, distinguishing critical regions such as lesions from healthy skin requires high precision, necessitating the adoption of advanced technological solutions to address these challenges²². Contemporary segmentation architectures have evolved to address specific clinical requirements, including computational efficiency for real-time applications, accuracy for critical diagnostic tasks, and robustness across diverse imaging protocols²³. Advanced U-Net variants incorporating attention mechanisms, multi-scale feature fusion strategies, and transformer-based encoders have demonstrated superior performance in complex medical imaging scenarios^12,24. These architectural innovations directly enhance clinical applications by enabling accurate delineation of anatomical structures, identification of pathological regions, and extraction of quantitative biomarkers, all of which are crucial for evidence-based medicine^25,26. However, integration challenges, including interface compatibility with existing electronic medical records (EMR), storage and retrieval of large images, and interoperability across different institutional systems, remain significant obstacles for widespread clinical adoption²⁷.

This paper introduces High-order Multi-scale Parallel Vision Mamba U-Net (HMP-MUNet), a novel CAD system specifically designed for automated cutaneous lesion segmentation in clinical practice. The framework here addresses critical gaps in current medical imaging systems by incorporating innovative architectural components: the Multi-Scale Dilated Attention Fusion Network (MSDAFN) and the Parallel Multi-depth Flexible Network (PMFlex). The MSDAFN implements sophisticated feature extraction strategies combining spatial and channel attention mechanisms with multi-scale dilated convolutions, enabling enhanced detection of subtle lesion characteristics across different scales—crucial for accurate diagnosis of varied lesion types encountered in clinical practice²⁸. The PMFlex introduces parallel processing capabilities enabling simultaneous analysis of multiple input streams, significantly improving computational efficiency while maintaining high-order modeling capabilities essential for clinical-grade accuracy²⁹. HMP-MUNet, through its innovative segmentation techniques and computational strategies, aims to push the boundaries of skin cancer diagnosis, enabling real-time and accurate clinical diagnosis^30,31.

The primary contributions of this research align with core objectives of advancing CAD in medical imaging: enhanced global context modeling through MSDAFN improving feature extraction and fusion efficiency for accurate lesion localization across diverse clinical presentations; efficient parallel processing via PMFlex reducing computational overhead while maintaining clinical-grade accuracy, facilitating deployment in resource-constrained clinical environments; clinical deployment optimization through hardware-aware design enabling efficient processing of high-resolution dermoscopic images supporting real-time clinical workflows; Clinical performance demonstrated through comprehensive evaluation on standard datasets with performance metrics suitable for clinical integration; and integrated diagnostic solution combining high-order modeling, multi-scale analysis, and parallel computation providing comprehensive solutions for precision dermatological diagnosis. This research advances computerized medical imaging by providing innovative, clinically applicable solutions addressing both technological and practical requirements for automated cutaneous lesion analysis in contemporary healthcare settings.

The framework is designed to handle high-resolution dermoscopic images, typically with input resolutions of 256 × 256, providing a balance between computational efficiency and image detail. However, variability in equipment and image quality, such as differences in lighting, skin tone, and the presence of hair, can affect segmentation performance. Despite these challenges, the system's design is optimized for real-time clinical workflows and is adaptable to different imaging devices, ensuring robust performance across diverse clinical environments.

Protocol

This research was performed in compliance with institutional guidelines for computational research and data processing. No human subjects or vertebrate animals were involved in this study.

Dataset preparation and preprocessing

Dataset collection and organization

Dermoscopic images were collected from the PH2, ISIC2018, and ISIC2017 datasets, with specific links provided in the Table of Materials. The PH2 dataset contains 200 high-resolution images (1000 × 1000 pixels), ISIC2018 includes 2,594 images with corresponding segmentation masks, and ISIC2017 includes 2,150 images with segmentation masks. The data were organized into training, validation, and testing sets following standard protocols, ensuring the images are in compatible formats (e.g., .jpg, .png, or .tiff) and include corresponding ground truth masks in binary format.

Data were organized into training, validation, and testing sets following standard protocols. The datasets were split as follows: for ISIC2018, 1,815 images were allocated for training, 259 for validation, and 520 for testing. For ISIC2017, 1,500 images were allocated for training, 220 for validation, and 430 for testing. For PH2, the dataset was divided into 160 training images, 10 validation images, and 30 test images. A consistent image resolution of 256 × 256 pixels was maintained throughout the preprocessing pipeline.

Data augmentation and normalization

Data augmentation techniques were applied to enhance model generalization. Horizontal flipping, vertical flipping, and random rotation within a ±15° range were implemented. Gamma correction and logarithmic transformation were applied with a probability of 0.3 each. Pixel values were normalized using ImageNet statistics (mean = [0.485, 0.456, 0.406], standard deviation = [0.229, 0.224, 0.225]).

Resize all input images to 256 × 256 pixels: To ensure computational efficiency, all input images were resized to a uniform resolution of 256 × 256 pixels. While high-resolution images, such as those in the dataset (1000 × 1000 pixels), contain fine-grained details and texture information crucial for accurate lesion boundary delineation, increasing the resolution did not significantly improve the model's performance. Therefore, the resolution was not increased to maintain a balance between computational efficiency and model accuracy. Future work may further investigate the effects of higher-resolution inputs to determine if the benefits in segmentation accuracy justify the increased computational cost. Images were converted to RGB format when necessary. The preprocessed data were saved in structured directories while maintaining the original dataset splits.

HMP-MUNet architecture implementation

Network architecture design

HMP-MUNet was configured following the hierarchical U-shaped encoder-decoder structure as shown in Figure 1. The network was configured with progressive channel expansion: 8→16→32→64→128→256 channels across encoder levels.

A significant architectural change was made by reducing the network depth from four hierarchical levels to two, which helps simplify the model and improve computational efficiency. Although the depth is reduced, the channel dimensions were increased at each level, enabling the network to capture richer and more expressive features. To further enhance the model's feature learning, attention mechanisms were introduced to focus the network on the most relevant features across multiple scales. These attention mechanisms effectively compensate for the potential loss of hierarchical feature abstraction due to the shallower architecture.

The encoder was initialized with dual Conv2D operations for initial feature extraction from input tensors X(C, H×W). The alternating configuration of three specialized modules was implemented: High-order Vision Mamba-based Switching Scheme (H-VSS), Parallel Multi-depth Flexible Network (PMFlex), and Multi-Scale Dilated Attention Fusion Network (MSDAFN).

The model uses ImageNet statistics (mean and standard deviation) for normalization, avoiding the need to recalculate for each dataset, thereby improving computational efficiency. It ensures stability and generalizability, leveraging ImageNet's extensive use in computer vision tasks. This choice simplifies design by reducing preprocessing complexity and enhances cross-dataset transferability, allowing the model to generalize effectively across diverse skin lesion images.

High-order vision mamba-based switching scheme (H-VSS) implementation                

The H-VSS module was implemented according to the architecture in Figure 2. Layer Normalization (LN) and Hardswish activation (HS) were configured with residual connections following Equation (1):

The Local Spatial Descriptor (LSD) component was implemented to maintain spatial coherence. The Spatial Selective 2D Scanning (SS2D) module was configured with multi-directional scanning patterns following Equation (2):

Multi-Layer Perceptron (MLP) processing was added with residual connections as specified in Equation (3):

The High-order 2D Selective Scanning (H-SS2D) mechanism projects input features to 2C-dimensional space for enhanced contextual modeling.

Parallel multi-depth flexible network (PMFlex) implementation

The PMFlex module was configured according to Figure 3 specifications. Layer Normalization was applied to input feature maps X(C, H×W), then partition into four segments along the channel dimension following Equation (4):

Each segmented feature Y_i was processed through shared Visual Mamba (VMamba) modules. Processed outputs were concatenated, and refinement was applied through Layer Normalization and projection as described in Equation (5):

Multi-Scale dilated attention fusion network (MSDAFN) implementation

The MSDAFN module was implemented following the Figure 4 architecture. Parallel convolution operations were configured with dilation rates of 6, 12, and 18 for multi-scale feature extraction according to Equation (6):

Multi-scale features were integrated through channel concatenation as specified in Equation (7):

Dual attention mechanisms were implemented for feature recalibration. Channel attention weights were calculated using global average pooling following Equation (8):

Channel attention weighting was applied as described in Equation (9):

Spatial attention was configured through convolution operations following Equation (10):

Channel and spatial attention were combined as specified in Equation (11):

Training configuration and optimization

Environment setup

The experimental environment was configured on an Ubuntu 20.04 system with a GPU (32 GB VRAM). Python 3.8, deep learning framework (RRID: SCR_018536), and CUDA 11.8 were installed. The input image size was set to 256 × 256 pixels for cutaneous lesion tasks.

Loss function and optimizer configuration

The BceDice loss function was implemented for training optimization. AdamW optimizer was configured with initial learning rate of 0.001, batch size of 8, and 250 training epochs. Weight decay of 1 × 10⁻⁵ was applied for regularization.

For the baseline configuration, this study refers to the work by Liu et al. (2024) on the Vmamba model, which was used as a visual state space model for the medical image classification task¹¹. The exact configurations and scripts used to implement Vmamba are based on their published work and adapted for this study, with specific adjustments to better suit the medical imaging dataset.

Learning rate scheduling was set up using cosine annealing with warm restarts. Configure T_0 = 10 epochs for initial restart period, T_mult = 2 for period multiplication, and η_min = 1 × 10⁻⁶ for minimum learning rate.

Hyperparameter optimization

To ensure reproducibility and consistency, all experiments were conducted using a fixed random seed of 42. The systematic hyperparameter optimization was conducted focusing on batch size, learning rate, and drop path rate. Batch sizes of 4, 8, 16, and 32 were evaluated. The learning rates of 0.0005, 0.001, 0.0015, and 0.002 were tested. The validation performance was monitored using Dice Similarity Coefficient (DSC) as the primary metric. Batch sizes larger than 8 may lead to memory overflow on GPUs with less than 32 GB VRAM.

Model evaluation and performance assessment

Evaluation metrics configuration

Comprehensive evaluation metrics, including Mean Intersection over Union (mIoU), Dice Similarity Coefficient (DSC), Sensitivity (Sen), Specificity (Spe), and Accuracy (Acc), were implemented. The metrics were calculated according to the following formulations:

Mean Intersection over Union (mIoU) quantifies overlap between predicted and ground truth segmentation:

Dice Similarity Coefficient (DSC) measures segmentation consistency. Values range from 0 to 1, with higher values indicating better performance:

Sensitivity (Sen) measures the model's ability to detect positive samples:

Specificity (Spe) assesses correct negative sample recognition:

Accuracy (Acc) measures overall prediction correctness:

Parameters (M) reflect model complexity, measuring total trainable parameters:

where P_i is the number of parameters in the i-th layer, and N is the total number of layers. Smaller parameter counts indicate more lightweight models suitable for clinical deployment.

Ablation study protocol

Comprehensive ablation studies were conducted to validate architectural component contributions following the experimental design in Table 1. Four architectural variants were evaluated: H-MUNet (baseline without MSDAFN and PMFlex), HP-MUNet (without MSDAFN), HM-MUNet (without PMFlex), and HMP-MUNet (complete model).

Identical training parameters were configured across all variants: learning rate 0.001, batch size 8, 250 epochs. Training convergence was monitored, and performance improvements were validated for each component addition.

Computational efficiency analysis

The computational efficiency metrics, including parameter count, FLOPs, and inference time across different architectures, were measured. The evaluation of the model's inference time on a standard clinical-grade GPU shows that, although it performs best on high-performance GPUs, the model is approximately 70% slower on more commonly used hardware. Nevertheless, it is still capable of efficient inference speeds, making it suitable for practical clinical deployment. Monitor GPU memory usage during training to ensure system stability. Recommended minimum 16 GB GPU memory for batch size 8.

Validation and analysis

Performance benchmarking

The performance of HMP-MUNet was compared against state-of-the-art methods on both datasets. All compared models were implemented and trained under the same data splits, preprocessing, data augmentation, and training protocols to ensure that performance differences were solely due to architectural differences. The quantitative results, including DSC improvements of 2.89% and 5.25% on the ISIC2018 and PH2 datasets, were documented respectively. The study validated that the parameter count is reduced by 4.55 times compared to the U-Net baseline.

Statistical analysis

Statistical significance testing was performed using paired t-tests for performance comparisons. Confidence intervals were calculated for the reported metrics. Reproducibility was ensured through multiple training runs with different random seeds.

The optimal hyperparameter configurations were recorded as batch size 8 and learning rate 0.001, achieving a peak DSC of 0.9585 on the PH2 dataset and 0.9044 on the ISIC2018 dataset, as documented in the experimental results. Sufficient validation data were ensured to prevent overfitting. Validation loss curves were monitored to apply early stopping criteria.

Results

HMP-MUNet architecture design for medical image segmentation

The proposed HMP-MUNet architecture demonstrated exceptional performance in automated cutaneous lesion segmentation tasks. Figure 1 illustrates the complete network architecture, showcasing the hierarchical U-shaped encoder-decoder structure with progressive channel expansion from 8 to 256 channels. The integration of three specialized modules in alternating configuration proved highly e...

Discussion

This study presents HMP-MUNet (High-order Multi-scale Parallel Vision Mamba U-Net), a novel deep learning framework that addresses fundamental challenges in computer-aided diagnosis (CAD) for cutaneous lesion segmentation through innovative integration of State-Space Models (SSMs) with advanced architectural components¹. The approach described here demonstrates that combining high-order feature interaction mechanisms with multi-scale attention and parallel processing can achieve superior diagnosti...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
CUDA 11.8	NVIDIA Corporation, Santa Clara, CA, USA		Software
GPU (32 GB VRAM)	NVIDIA		Operating system
ISIC2017 Dataset	ISIC Challenge	https://challenge.isic-archive.com/data	Datasets
ISIC2018 Dataset	ISIC Challenge	https://challenge.isic-archive.com/data	Datasets
NVIDIA V100 GPU	NVIDIA Corporation, Santa Clara, CA, USA		Hardware
PH2 Dataset	Universidade Do Porto	https://www.fc.up.pt/addi/ph2	Datasets
Python 3.8	Python	RRID:SCR_018536	Software
PyTorch 1.13.0	PyTorch	RRID:SCR_018536	Software
Ubuntu 20.04	Canonical		Hardware