Semantic Segmentation–Guided Reconstruction and Artistic Style Synthesis of Intangible Cultural Heritage Patterns Using a Deep Learning Framework

Summary

This protocol describes a deep learning workflow for semantic segmentation, reconstruction, and artistic style synthesis of intangible cultural heritage patterns using transformer-based feature extraction, adversarial reconstruction, and spatially adaptive image generation to support digital preservation and creative heritage applications.

Abstract

The digital preservation and reconstruction of intangible cultural heritage (ICH) patterns have attracted increasing attention with the advancement of deep learning–based image synthesis techniques. Existing SegCycle-SPADE-based approaches have demonstrated potential for the structural segmentation and artistic reconstruction of traditional craft patterns; however, limitations remain, including limited dataset diversity, insufficient incorporation of cultural semantics, and inadequate preservation of structural pattern features during reconstruction. To address these challenges, this study proposes an improved SegCycle-SPADE framework for the semantic segmentation and artistic reconstruction of ICH pattern types. A Transformer-based segmentation model, SegFormer, is employed to accurately identify motif boundaries and pattern regions. In addition, a Cross-Attention Cultural Feature Fusion Module is introduced to enhance culturally significant motifs and improve feature representation. Pattern translation and reconstruction are performed using CycleGAN, while Spatially-Adaptive Denormalization (SPADE) is used for artistic style synthesis to maintain semantic consistency between segmentation maps and generated images. To improve model generalization and artistic diversity, the framework is trained using both the Miao Batik cultural motif dataset and the WikiArt dataset. Experimental results demonstrate that the proposed attention-guided SegCycle-SPADE framework improves segmentation accuracy and heritage-pattern reconstruction performance compared with existing generative adversarial network (GAN)-based reconstruction methods. The proposed framework provides a scalable solution for artificial intelligence–assisted cultural heritage documentation, reconstruction, and artistic revitalization of traditional pattern designs.

Introduction

Cultural heritage, which includes both intangible elements such as customs, languages, and beliefs, and tangible elements such as artworks, buildings, and written records, is a fundamental manifestation of human history, creativity, and identity¹. Heritage conservation seeks to enable communities to reconnect with their origins and allows future generations to benefit from their collective cultural memory. It also promotes intercultural understanding, appreciation of diverse traditions and customs, and recognition of different historical narratives. These cultural assets help us understand the values, perspectives, and experiences of previous g....

Protocol

The proposed framework of SegCycle-SPADE is designed to reconstruct and improve traditional cultural heritage patterns through semantic segmentation, feature enhancement using attention, generative reconstruction, and artistic style synthesis. This study utilized publicly available cultural heritage and artistic image datasets, including the Miao Batik dataset and the WikiArt dataset. No human participants, patient data, personal information, animal subjects, or clinical records were involved in the research; therefore, institutional ethics committee approval and informed consent were not required. To start with, the Miao Batik cultural motif dataset and the WikiArt d....

Results

The proposed SegCycle-SPADE framework was evaluated using two datasets: the Miao Batik cultural motif dataset and the WikiArt dataset. The Miao Batik dataset contains traditional cultural heritage images and was primarily used for semantic segmentation and structural reconstruction tasks, whereas the WikiArt dataset contains diverse artistic styles and was used for artistic style learning and generation. Images from both datasets were processed through the complete SegCycle-SPADE pipeline, including semantic segmentation.......

Discussion

The preservation and digital reconstruction of ICH patterns have gained increasing attention in recent years because of the gradual loss of traditional crafts. Previous studies have attempted to address cultural heritage loss through DL-based image processing techniques. For example, Quan et al.³² proposed a cultural heritage preservation framework based on knowledge graphs and DL for Guizhou Miao batik culture. Although the study focused on the digital reconstruction of cultural patterns, its met.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Adam Optimizer	PyTorch Optimizer Library	Adam	Optimization algorithm used during model training.
CUDA Toolkit	NVIDIA Corporation	CUDA 11.3	GPU acceleration for deep learning computations.
cuDNN	NVIDIA Corporation	cuDNN 8.2	Deep neural network acceleration library used to accelerate training and inference.
CycleGAN	University of California, Berkeley / Open Source	Official PyTorch Implementation (https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix)	Generative adversarial network used for cultural motif reconstruction.
ImageNet Pretrained Weights	ImageNet / Open Source	mit_b2.pth (ImageNet-1K Pretrained Checkpoint)	Used to initialize the SegFormer-B2 backbone before fine-tuning on the Miao Batik dataset.
Intel Processor	Intel Corporation	Intel Core i7 (11th Generation)	CPU used for image preprocessing, model training support, and inference.
Matplotlib	Matplotlib Development Team	Matplotlib 3.5.1	Visualization of training curves and evaluation results.
Miao Batik Dataset	Zenodo Repository	DOI: 10.5281/zenodo.20807658	Cultural motif dataset containing 15,148 images used for semantic segmentation and pattern reconstruction.
NumPy	NumPy Developers	NumPy 1.21.5	Numerical computation and dataset processing.
NVIDIA GPU	NVIDIA Corporation	NVIDIA RTX 3090 (24 GB VRAM)	Hardware platform used for model training and inference.
OpenCV	OpenCV Foundation	OpenCV 4.5.5	Image preprocessing, resizing, interpolation, and Gaussian denoising.
Operating System	Canonical Ltd.	Ubuntu 20.04 LTS	Experimental execution environment.
Pillow (PIL)	Python Imaging Library	Pillow 8.4.0	Image loading and manipulation.
Python	Python Software Foundation	Python 3.8.10	Programming language used for implementation and experimentation.
PyTorch	Meta AI	PyTorch 1.10.0	Deep learning framework used for model training and inference.
Random Seed	Experimental Setting	42	Fixed seed used for dataset partitioning, model initialization, and experimental reproducibility.
Scikit-learn	Scikit-learn Developers	Scikit-learn 1.0.2	Dataset partitioning, statistical analysis, and evaluation metrics.
Seaborn	Open Source Community	Seaborn 0.11.2	Statistical data visualization.
SegFormer-B2	NVIDIA Research / Open Source	SegFormer-B2 (MIT-B2 Backbone)	Transformer-based semantic segmentation model used for cultural motif segmentation.
Segmentation Masks / Annotations	Miao Batik Dataset	Included with Dataset	Pixel-level semantic segmentation labels used for motif classification and training.
SPADE	NVIDIA Research / Open Source	Official PyTorch Implementation (https://github.com/NVlabs/SPADE)	Segmentation-guided image synthesis model used for artistic pattern generation.
TorchVision	Meta AI	TorchVision 0.11.1	Image-processing utilities and dataset transformations used with PyTorch.
WikiArt Dataset	WikiArt	https://www.wikiart.org/	Artistic style dataset containing more than 80,000 artworks across 27 artistic styles, used for style learning and synthesis.