Multi-view Vision Mamba U-Shaped Network Framework for Medical Image Segmentation

Summary

This protocol describes how to construct, train, and evaluate a multiview Vision Mamba U-Shaped Network framework for medical image segmentation, enabling reproducible segmentation of skin lesions and abdominal organs through standardized dataset preparation, model implementation, and performance evaluation.

Abstract

Medical image segmentation requires computational methods that accurately capture global context, local boundaries, and multiscale anatomical structures while remaining reproducible across different applications. This article presents a protocol for constructing, training, and evaluating a Multi-view Vision Mamba U-Shaped Network framework for two-dimensional medical image segmentation. The protocol provides a reproducible workflow that includes public dataset acquisition, image and mask preprocessing, network construction, model training, checkpoint selection, and quantitative and qualitative performance evaluation. The framework incorporates multiview feature scanning to capture complementary spatial, contour, scale, and boundary information and applies multistage feature fusion within a U-shaped encoder-decoder architecture to improve feature integration during segmentation. The protocol is demonstrated using publicly available skin lesion and abdominal organ segmentation datasets. Under the described implementation workflow, the framework achieves competitive segmentation performance using standard evaluation metrics. By following the procedures presented in this protocol, researchers can reproduce the model implementation, train the network using defined experimental settings, evaluate segmentation performance, and adapt the workflow for related medical image segmentation tasks requiring reproducible deep learning–based analysis.

Introduction

Medical image segmentation is a fundamental task in the fields of computer vision and medical image analysis^1,2,3. It involves partitioning an image into multiple regions or objects for further analysis and processing. This technology is particularly important in medical imaging because it assists clinicians in identifying and localizing pathological regions, thereby improving diagnostic accuracy and treatment planning. With the advancement of medical imaging technologies, such as magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PE....

Protocol

This study used only publicly available and de-identified medical image datasets, including ISIC 2017, ISIC 2018, and Synapse. No new human participants, animal subjects, or identifiable private medical records were collected in this study. The ISIC 2017 dataset used in this study was the ISIC 2017 Challenge skin lesion segmentation dataset, obtained from the official International Skin Imaging Collaboration challenge data repository (https://challenge.isic-archive.com/data/#2017). The dataset version used in this study corresponds to the ISIC 2017 lesion segmentation task, including the official training, validation, and testing partitions. The dataset was downloaded....

Results

Expected outcomes and interpretation
When this protocol is implemented correctly, the trained MVM-UNet model is expected to produce stable segmentation performance across repeated runs, with only small variations among different random seeds for most evaluation metrics. For ISIC 2017 and ISIC 2018, successful outcomes are reflected by high DSC, mIoU, Acc, Sen, and Spe values, together with predicted lesion masks that closely follow the ground-truth lesion boundaries (Figures 6 and 7).......

Discussion

This protocol describes MVM-UNet, a Mamba-based medical image segmentation framework. The method was designed to address two limitations of existing segmentation models. First, conventional CNN-based methods have limited ability to model long-range dependencies and hierarchical contextual information in complex medical images⁴³. Second, Transformer-based methods can model global context but usually require higher computational cost^23,25. M.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Hardware - GPU workstation or GPU server	Institutional computing platform / local workstation	Custom-built local GPU workstation; Ubuntu 22.04.1 with Linux kernel 6.8.0; Intel Core i9-13900K CPU; NVIDIA A800 GPU; 128 GB RAM; 512 GB local storage.	Computational platform for training, validation, testing, ablation, and inference-only experiments.
Hardware - Graphics processing unit (GPU)	NVIDIA Corporation	NVIDIA A800 GPU (80 GB memory).	GPU-accelerated model training and inference.
Hardware - Central processing unit (CPU)	Intel Corporation / AMD	13th Gen Intel Core i9-13900K CPU.	Host processor for data loading, preprocessing, and experiment execution.
Hardware - System memory (RAM)	Institutional computing platform / local workstation	128 GB system RAM	Memory for dataset loading, preprocessing, and training.
Hardware - Storage	Institutional computing platform / local workstation	2 TB NVMe solid-state drive.	Storage for datasets, checkpoints, logs, and generated prediction figures.
Software environment - Operating system	Canonical Ltd.	Recommended: Ubuntu 22.04.1 LTS	Operating system for the computational environment.
Software environment - Conda environment	Anaconda, Inc. / Miniconda	Environment name: mvmunet	Python environment used to install and isolate dependencies.
Software environment - Python	Python Software Foundation	Python 3.8	Programming language used for implementation and experiment execution.
Software environment - CUDA toolkit	NVIDIA Corporation	CUDA Toolkit 11.8	GPU computing backend required by PyTorch and Mamba-related packages.
Software environment - cuDNN	NVIDIA Corporation	cuDNN 8.7.0.	GPU-accelerated deep-learning primitives used through PyTorch.
Python package - PyTorch	PyTorch	torch == 2.0.1	Deep-learning framework for model training, loss calculation, optimization, and inference.
Python package - Torchvision	PyTorch	torchvision == 0.14.0	Image transform utilities used in preprocessing and augmentation.
Python package - Torchaudio	PyTorch	torchaudio == 0.13.0	Installed with the recommended PyTorch environment.
Python package - timm	timm developers	timm == 0.4.12	Model-component or utility dependency listed in the repository environment instructions.
Python package - triton	OpenAI / Triton developers	triton == 2.0.0	Dependency used by GPU-accelerated sequence modeling components.
Python package - causal-conv1d	causal-conv1d developers	causal_conv1d == 1.0.0	Efficient causal convolution dependency required by the Mamba implementation.
Python package - mamba-ssm	Mamba SSM developers	mamba_ssm == 1.0.1	State-space sequence modeling package used for Mamba/S6-related components.
Python package - NumPy	NumPy developers	NumPy version 1.24.3.	Numerical computation and array operations.
Python package - SciPy	SciPy developers	SciPy version 1.10.1.	Scientific computation; scipy.ndimage.zoom is imported in utils.py.
Python package - SimpleITK	Insight Software Consortium	SimpleITK version 2.2.1.	Medical image input/output and preprocessing utility imported in utils.py.
Python package - MedPy	MedPy developers	MedPy version 0.4.0.	Medical image metric calculation package imported in utils.py.
Python package - scikit-image	scikit-image developers	scikit-image version 0.21.0.	Image-processing dependency listed in README.
Python package - scikit-learn	scikit-learn developers	scikit-learn version 1.3.2.	Machine-learning utility package listed in README.
Python package - matplotlib	Matplotlib developers	Matplotlib version 3.7.2.	Used for saving qualitative visualization figures.
Python package - h5py	h5py developers	h5py version 3.9.0.	HDF5 file support for Synapse test volumes.
Python package - thop	THOP developers	THOP version 0.1.1.post2209072238.	Used when calculating FLOPs and parameter-related computational cost.
Python package - packaging	Python Packaging Authority	packaging version 23.1.	Dependency listed in README.
Python package - pytest	pytest developers	pytest version 7.4.0.	Dependency listed in README.
Python package - chardet	chardet developers	chardet version 5.2.0.	Dependency listed in README.
Python package - yacs	YACS developers	yacs version 0.1.8.	Configuration utility dependency listed in README.
Python package - termcolor	termcolor developers	termcolor version 2.3.0.	Logging/terminal utility dependency listed in README.
Python package - submitit	submitit developers	submitit version 1.4.5.	Experiment/job utility dependency listed in README.
Python package - tensorboardX	tensorboardX developers	tensorboardX version 2.6.2.2.	Training log visualization utility listed in README.
Python package - ml-collections	ml_collections developers	ml-collections version 0.1.1.	Imported by configs/config_setting_synapse.py.
Dataset - ISIC 2017 Challenge dataset	International Skin Imaging Collaboration	ISIC 2017 skin lesion segmentation dataset	Public de-identified dermoscopic skin lesion images and masks used for binary segmentation.
Dataset - ISIC 2018 Challenge Task 1 dataset	International Skin Imaging Collaboration	ISIC 2018 Task 1: Lesion Boundary Segmentation	Public de-identified dermoscopic skin lesion images and masks used for binary segmentation.
Dataset - Synapse Multi-Atlas Labeling Beyond the Cranial Vault dataset	Synapse / Sage Bionetworks	Accession identifier: syn3193805	Public abdominal CT multi-organ segmentation dataset.
Data organization - ISIC 2017 data folder	Authors / repository layout	data/isic2017/	Expected local folder containing train and validation images/masks.
Data organization - ISIC 2018 data folder	Authors / repository layout	data/isic2018/	Expected local folder containing train and validation images/masks.
Data organization - Synapse data folder	Authors / repository layout	data/Synapse/	Expected local folder for Synapse lists, train_npz, and test_vol_h5.
Source code - MVM-UNet source-code repository	Authors / GitHub	Branch: master;Git commit hash: ee891b42c2f083c4990eed72f1d4463adc5e103e.	Complete source-code implementation of the protocol.
Source code - ISIC configuration file	Authors	configs/config_setting.py	Configuration file for ISIC-style binary segmentation.
Source code - Synapse configuration file	Authors	configs/config_setting_synapse.py	Configuration file for Synapse multi-organ segmentation.
Source code - ISIC training script	Authors	train.py	Training and validation entry point for ISIC-style binary segmentation.
Source code - Synapse training script	Authors	train_synapse.py	Training and validation entry point for Synapse multi-class segmentation.
Source code - ISIC engine file	Authors	engine.py	Training/validation engine for ISIC-style experiments.
Source code - Synapse engine file	Authors	engine_synapse.py	Training/validation engine for Synapse experiments.
Source code - Losses and utility functions	Authors	utils.py	Implements seed setting, optimizer/scheduler utilities, BCE-DICE loss, CE-DICE loss, Dice loss, data transforms, and qualitative image saving.
Source code - MVM-UNet architecture implementation	Authors	models/mvmunet/mvmunet.py	Main MVM-UNet network definition.
Source code - Custom module implementation	Authors	models/mvmunet/core.py	Implements MV4D, MVV Block, SFusion Mamba, MFusion Mamba, S6/Mamba-related components, and scan-index generators.
Training configuration - ISIC loss function	Authors / PyTorch	BceDiceLoss(wb=1, wd=1)	Combined BCE-DICE loss for binary skin lesion segmentation.
Training configuration - Synapse loss function	Authors / PyTorch	CeDiceLoss(num_classes=9, loss_weight=[1,1])	Combined CE-DICE loss for multi-class organ segmentation.
Training configuration - Optimizer	PyTorch	AdamW	Optimizer used in default configuration files.
Training configuration - Learning-rate scheduler for ISIC	PyTorch	CosineAnnealingLR	Learning-rate scheduler in default ISIC configuration.
Training configuration - Learning-rate scheduler for Synapse	PyTorch	CosineAnnealingLR	Learning-rate scheduler in default Synapse configuration.
Training configuration - Random seed control	Authors / PyTorch / NumPy	set_seed(seed) in utils.py	Function used to set Python, NumPy, PyTorch CPU, PyTorch CUDA, and cuDNN deterministic behavior.
Training configuration - Mixed-precision setting	Authors / PyTorch	amp = False	Automatic mixed-precision training flag.
Output - Training result directory	Authors / repository layout	results/mvmunet_*	Directory used to store checkpoints, logs, and generated outputs.
Output - Inference-only output path	Authors / configuration files	img_save_path	Directory for saved qualitative prediction figures during inference-only evaluation.
Output - Best checkpoint path	Authors / configuration files	best_ckpt_path	Checkpoint path used for inference-only evaluation.