Semantic Segmentation–Guided Reconstruction and Artistic Style Synthesis of Intangible Cultural Heritage Patterns Using a Deep Learning Framework

Summary

This protocol describes a deep learning workflow for semantic segmentation, reconstruction, and artistic style synthesis of intangible cultural heritage patterns using transformer-based feature extraction, adversarial reconstruction, and spatially adaptive image generation to support digital preservation and creative heritage applications.

Abstract

The digital preservation and reconstruction of intangible cultural heritage (ICH) patterns have attracted increasing attention with the advancement of deep learning–based image synthesis techniques. Existing SegCycle-SPADE-based approaches have demonstrated potential for the structural segmentation and artistic reconstruction of traditional craft patterns; however, limitations remain, including limited dataset diversity, insufficient incorporation of cultural semantics, and inadequate preservation of structural pattern features during reconstruction. To address these challenges, this study proposes an improved SegCycle-SPADE framework for the semantic segmentation and artistic reconstruction of ICH pattern types. A Transformer-based segmentation model, SegFormer, is employed to accurately identify motif boundaries and pattern regions. In addition, a Cross-Attention Cultural Feature Fusion Module is introduced to enhance culturally significant motifs and improve feature representation. Pattern translation and reconstruction are performed using CycleGAN, while Spatially-Adaptive Denormalization (SPADE) is used for artistic style synthesis to maintain semantic consistency between segmentation maps and generated images. To improve model generalization and artistic diversity, the framework is trained using both the Miao Batik cultural motif dataset and the WikiArt dataset. Experimental results demonstrate that the proposed attention-guided SegCycle-SPADE framework improves segmentation accuracy and heritage-pattern reconstruction performance compared with existing generative adversarial network (GAN)-based reconstruction methods. The proposed framework provides a scalable solution for artificial intelligence–assisted cultural heritage documentation, reconstruction, and artistic revitalization of traditional pattern designs.

Introduction

Cultural heritage, which includes both intangible elements such as customs, languages, and beliefs, and tangible elements such as artworks, buildings, and written records, is a fundamental manifestation of human history, creativity, and identity¹. Heritage conservation seeks to enable communities to reconnect with their origins and allows future generations to benefit from their collective cultural memory. It also promotes intercultural understanding, appreciation of diverse traditions and customs, and recognition of different historical narratives. These cultural assets help us understand the values, perspectives, and experiences of previous generations and contribute to the formation of contemporary social identities and cultural continuity. The basic principles of cultural heritage preservation are illustrated in Figure 1.

Figure 1. Conceptual Overview of Cultural Heritage Pattern Reconstruction Using the SegCycle-SPADE Framework. Schematic illustration of the proposed approach for reconstructing and enhancing intangible cultural heritage patterns. The workflow includes dataset collection from the Miao Batik and WikiArt datasets, image preprocessing, semantic segmentation using SegFormer-B2, feature fusion using the Cross-Attention Cultural Feature Fusion Module (CAFFM), pattern reconstruction using CycleGAN, artistic style synthesis using SPADE, and final evaluation using segmentation and reconstruction metrics. Arrows indicate the flow of data and feature representations throughout the framework. Please click here to view a larger version of this figure.

The collective memory and cultural legacy of human society are embodied in intangible cultural heritage (ICH), which serves as an important medium for artistic expression and cultural identity. However, ICH faces significant challenges due to the effects of globalization and digitalization^2,3,4. Many endangered ICH practices require new approaches to preservation and development because of declining numbers of skilled artisans and limited adaptability to modern markets^5,6. As an important area of ICH research, sustainable design emphasizes the integrated development of culture, society, and the environment and provides a practical pathway for the continued preservation of ICH. Through sustainable design approaches, ICH can be revitalized while adapting to the needs of contemporary society^7,8.

In this study, the term “intangible cultural heritage patterns” refers to visual motif representations that communicate and preserve underlying intangible cultural values. Consequently, the proposed framework aims to preserve and document the cultural information associated with these motifs while reconstructing their visual forms.

Miao embroidery and batik are significant forms of Chinese ICH, reflecting the history, beliefs, and traditions of the Miao community through symbolic motifs such as birds, butterflies, and ancestral totems⁹. These motifs are deeply embedded in ritual clothing and festival practices and contribute to local cultural and economic development^10,11. Advances in digital technologies, particularly artificial intelligence (AI) and deep learning (DL), have created new opportunities for cultural heritage preservation, analysis, reconstruction, and documentation. Guizhou Miao batik, one of the best-preserved batik traditions in China, serves as an important medium for transmitting the cultural knowledge, beliefs, customs, and rituals of the Miao people^{12,13,14,15,16}.

Recent studies have demonstrated the potential of DL for cultural heritage preservation and pattern reconstruction, achieving improved segmentation accuracy (pixel accuracy = 88.6%, mean intersection over union [IoU]= 78.1%) and reconstruction performance compared with U-Net and DeepLabV3+1. However, several challenges remain. Existing generative adversarial network (GAN)-based approaches often struggle to preserve fine structural details in complex patterns¹⁷, while limited dataset diversity restricts model generalization across cultural domains¹⁸. In addition, image-to-image translation models such as CycleGAN may fail to maintain semantic consistency between segmentation maps and generated images¹⁹, and the absence of attention mechanisms can result in the loss of culturally significant motif details during reconstruction²⁰. Therefore, an enhanced framework integrating transformer-based segmentation, attention-guided feature learning, and multidataset training is needed for effective ICH pattern reconstruction.

New opportunities for cultural heritage preservation have emerged through the digitalization of Miao embroidery and the rapid advancement of generative AI. Diffusion models, an emerging class of deep generative models, have demonstrated remarkable performance in computer vision tasks due to their powerful content-generation capabilities^{21,22,23,24,25}. During the reverse diffusion process, the model gradually learns to reconstruct the original input data from noisy representations^26,27,28. Previous studies have also explored the application of three-dimensional reconstruction and computer-aided design (CAD) technologies for the preservation and dissemination of cultural heritage.

Even though convolutional neural networks (CNNs) and GANs perform well in image generation and feature preservation, they still face challenges related to limited receptive fields, mode collapse, and training instability²⁹. Transformer-based architectures, such as SegFormer and Segmenter, excel at capturing global context and semantic relationships; however, they are computationally intensive and may struggle with fine-grained details. Although diffusion models such as Stable Diffusion demonstrate strong image-generation capabilities, they often lack precise control over the structural and artistic characteristics of generated designs. Furthermore, most existing approaches employ independent training strategies that limit effective information sharing and collaborative optimization between segmentation and generation components, resulting in a trade-off between structural accuracy and artistic authenticity³⁰.

Recent diffusion-based models, such as latent diffusion and Stable Diffusion, achieve high-quality image synthesis but require iterative denoising, leading to increased computational cost and inference time. Moreover, preserving fine cultural motifs and structural consistency remains challenging without specialized conditioning mechanisms. Transformer-based generative models effectively capture global contextual relationships but often require large-scale datasets and substantial computational resources. In contrast, the proposed SegCycle-Spatially Adaptive Denormalization (SegCycle-SPADE) framework combines SegFormer-based segmentation, cross-attention feature fusion, CycleGAN reconstruction, and SPADE-guided synthesis to provide explicit structural guidance, thereby improving motif preservation, semantic consistency, and controllable reconstruction of cultural heritage patterns.

The majority of state-of-the-art semantic segmentation techniques rely on fully CNNs (FCNNs) with U-shaped architectures³¹. During the encoding stage, deep features with large receptive fields are extracted through a series of convolution and downsampling operations. The decoding stage progressively restores spatial resolution through upsampling, ultimately enabling pixel-level semantic prediction. By compensating for spatial information loss during downsampling and improving segmentation performance across a wide range of applications, skip connections allow high-resolution information from different encoder levels to be directly integrated into the decoder³².

While recent GAN-based, CNN-based, and diffusion-based methods have shown promising results in image generation and cultural heritage restoration, they often struggle to maintain semantic consistency, preserve fine structural motifs, and ensure reproducible reconstruction across diverse cultural patterns. Most existing approaches treat segmentation, reconstruction, and style synthesis as separate processes, which can result in the loss of culturally significant elements and inconsistent outputs. To address this methodological gap, this study proposes a unified SegCycle-SPADE framework that integrates SegFormer-based semantic segmentation, a novel Cross-Attention Cultural Feature Fusion Module (CAFFM), CycleGAN-based reconstruction, and SPADE-guided style synthesis. By explicitly integrating structural segmentation information with generative feature learning, the proposed framework enables more reliable, semantically consistent, and reproducible reconstruction of ICH motifs while preserving both cultural authenticity and artistic quality.

Three major limitations of current cultural heritage reconstruction approaches are identified in this study: (i) insufficient preservation of fine structural motifs in GAN-based reconstruction methods; (ii) limited generalization resulting from training on small or domain-specific datasets; and (iii) inadequate semantic consistency between segmentation outputs and generated images in image-to-image translation frameworks. Additionally, segmentation, reconstruction, and style synthesis are commonly treated as separate processes, leading to the loss of culturally important elements during reconstruction. By combining transformer-based semantic segmentation, cross-attention feature fusion, CycleGAN reconstruction, and SPADE-guided artistic synthesis within a unified architecture, the proposed SegCycle-SPADE framework addresses these limitations and improves motif preservation, semantic consistency, and artistic authenticity in the reconstruction of ICH patterns.

The main objective of this study is to develop an enhanced SegCycle-SPADE framework for semantic segmentation-guided artistic reconstruction of ICH patterns. The framework integrates SegFormer for accurate cultural motif segmentation, CycleGAN for pattern reconstruction, and SPADE for style-consistent artistic synthesis. To improve feature representation and preserve fine structural details, a CAFFM is introduced to facilitate effective interaction between segmentation and generative features. Trained on the Miao Batik and WikiArt datasets, the proposed framework enhances reconstruction authenticity, style consistency, and the preservation of culturally significant motifs.

By combining semantic segmentation, attention-guided feature fusion, pattern reconstruction, and style synthesis within a unified architecture, this study presents a novel SegCycle-SPADE framework for the artistic reconstruction of ICH patterns. The main contributions of this work are as follows. First, a novel semantic segmentation-guided reconstruction framework is developed through the integration of SegFormer, enabling the accurate extraction and preservation of intricate cultural motifs and structural elements. Second, a new CAFFM is introduced to effectively fuse segmentation features with generative representations, allowing the model to capture long-range relationships between cultural motifs and artistic style patterns. Third, the proposed CAFFM enhances the preservation of culturally significant elements while improving the visual realism and structural coherence of reconstructed heritage patterns. Fourth, by integrating CycleGAN for cultural pattern reconstruction and SPADE for segmentation-guided artistic synthesis, the framework ensures both semantic accuracy and stylistic authenticity in the generated outputs. Fifth, to improve generalization and artistic diversity, the model is trained using the Miao Batik cultural motif dataset for cultural pattern learning and the WikiArt dataset for artistic style representation. WikiArt serves as an auxiliary dataset to evaluate the framework’s generalization capability, feature-learning robustness, and style-control effectiveness across diverse visual contexts rather than as a target style for direct application in Miao cultural heritage products. Finally, compared with existing GAN-based cultural heritage reconstruction methods, experimental results demonstrate that the proposed SegCycle-SPADE framework achieves improved segmentation accuracy, reconstruction quality, and stylistic consistency.

Protocol

The proposed framework of SegCycle-SPADE is designed to reconstruct and improve traditional cultural heritage patterns through semantic segmentation, feature enhancement using attention, generative reconstruction, and artistic style synthesis. This study utilized publicly available cultural heritage and artistic image datasets, including the Miao Batik dataset and the WikiArt dataset. No human participants, patient data, personal information, animal subjects, or clinical records were involved in the research; therefore, institutional ethics committee approval and informed consent were not required. To start with, the Miao Batik cultural motif dataset and the WikiArt dataset are utilized for the evaluation. The Miao Batik dataset was described in Quan et al. (2024)32 and contains 15,148 annotated images of traditional Miao batik motifs. Existing annotations provided by the dataset creators were used directly, and no additional manual annotations were generated in this study. Then, preprocessing of the images is performed by resizing, normalizing, denoising, and creating a segmentation mask. The exact preprocessing parameters are described in the Data Preprocessing subsection. The proposed framework utilizes a transformer-based model named SegFormer-B2 for the semantic segmentation of the data. The method is designed to detect key regions of cultural motifs and learn their structures. The segmented features are then enhanced through an attention mechanism, where important information related to cultural motifs is highlighted and irrelevant information is discarded. The attention mechanism configuration is described in the CAFFM subsection. The enhanced features are then passed through CycleGAN, where damaged structures are reconstructed through cyclic learning. During training, incomplete motif samples were artificially created by applying random masking and partial occlusion operations to the original Miao Batik images. These degradation procedures simulated missing motif regions and structural damage commonly observed in deteriorated cultural heritage artifacts. The resulting incomplete images were used as inputs to the CycleGAN reconstruction module, while the corresponding original images served as reconstruction targets. The reconstructed cultural heritage patterns are then enhanced through SPADE, where artistic enhancement is carried out based on the generated segmentation maps. The performance of the proposed framework is evaluated using quantitative segmentation and image-quality metrics, while the visual authenticity of the reconstructed cultural motifs is assessed qualitatively. Visual authenticity was assessed through visual inspection of the generated cultural patterns and was not used as an independent quantitative metric. The assessment focused on motif preservation, semantic consistency, cultural characteristics, structural coherence, and artistic appearance relative to the corresponding reference cultural motifs. Quantitative evaluation of model performance was performed using Segmentation Accuracy, IoU, Dice Coefficient, Structural Similarity Index Measure (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Fréchet Inception Distance (FID). Figure 2 illustrates the overall workflow of the proposed SegCycle-SPADE framework and summarizes the evaluation metrics used in this study.

Figure 2. Overall Architecture Workflow of the Proposed SegCycle-SPADE Framework. Overview of the complete SegCycle-SPADE workflow. Images from the Miao Batik and WikiArt datasets undergo preprocessing before being processed through semantic segmentation using SegFormer-B2, feature enhancement using the CAFFM, pattern reconstruction using CycleGAN, and artistic style generation using SPADE. Model performance is evaluated using Segmentation Accuracy, Intersection over Union (IoU), Dice Coefficient, Structural Similarity Index Measure (SSIM), PSNR, Fréchet Inception Distance (FID), while visual authenticity is assessed qualitatively. Please click here to view a larger version of this figure.

The SegCycle-SPADE framework for cultural heritage pattern reconstruction is designed to integrate multiple DL components for accurate motif segmentation, reconstruction, and artistic enhancement, as illustrated in Figure 2. Images from the Miao Batik cultural motif dataset and the WikiArt dataset are used to provide diverse cultural patterns and artistic styles. Initially, the images are processed using a SegFormer-based semantic segmentation module to identify and delineate cultural motifs from the background. The overall architecture consists of four main stages. First, the SegFormer model extracts semantic segmentation maps from the cultural pattern images. Second, the CAFFM integrates segmentation features with generative representations to enhance feature learning. Third, the CycleGAN module reconstructs cultural patterns using the fused feature representations. Finally, the SPADE module generates stylistically enhanced outputs while preserving structural consistency through segmentation-guided synthesis. The refined feature maps are subsequently processed by the CycleGAN reconstruction module, which learns to restore incomplete cultural motifs by mapping degraded patterns to their corresponding complete forms. Incomplete motif samples were generated by applying random masking and partial occlusion operations to the original Miao Batik images, thereby simulating missing pattern regions and structural degradation commonly observed in damaged cultural artifacts. Each degraded image was paired with its corresponding original image, which served as the reconstruction target during training. This strategy enabled the CycleGAN module to learn the restoration of missing motif structures while preserving semantic consistency and culturally significant characteristics. Finally, the SPADE generator produces visually enhanced artistic outputs by conditioning image synthesis on the generated segmentation maps, thereby maintaining the structural integrity of the cultural motifs throughout the artistic enhancement process.

The following steps are involved in proposed SegCycle-SPADE framework.

Step 1: Dataset Collection
Two datasets were used to achieve the objectives of cultural pattern learning and artistic style generation. The Miao Batik Cultural Motif Dataset was used to provide traditional cultural pattern information. The dataset is publicly available through Zenodo (https://doi.org/10.5281/zenodo.20807658) and contains 15,148 annotated images of traditional Miao batik motifs. Existing annotations provided by the dataset creators were used directly, and no additional manual annotations were generated in this study. The WikiArt dataset was used to provide diverse artistic style information for artistic style generation and was obtained from the publicly available WikiArt repository (https://www.wikiart.org/).

Step 2: Data Preprocessing
All images were preprocessed through resizing, normalization, and noise reduction. Existing cultural motif annotations obtained from the original dataset sources were used to support the semantic segmentation stage. No additional annotation or relabeling procedures were performed as part of this study.

Step 3: Semantic Pattern Segmentation using SegFormer
The SegFormer model was used for cultural motif segmentation. The exact SegFormer backbone and initialization strategy are described in the Semantic Pattern Segmentation Using SegFormer subsection. Specifically, the SegFormer-B2 architecture with an ImageNet-pretrained MIT-B2 backbone was employed for semantic motif segmentation. This model effectively captures global contextual information while preserving the complex structural details of traditional cultural patterns.

Step 4: CAFFM
The CAFFM combines semantic segmentation features with generative representations, enabling the framework to learn both structural motif characteristics and artistic style information. Integration details of CAFFM are provided in the CAFFM subsection. The module is positioned between the SegFormer-based semantic segmentation stage and the generative reconstruction stage, where it fuses segmentation-derived structural features with reconstruction features through multi-head cross-attention. This process improves cultural motif preservation and enhances the realism of the reconstructed outputs.

Step 5: Pattern Reconstruction (CycleGAN)
The fused features are subsequently processed by the CycleGAN module to reconstruct cultural pattern structures. The CycleGAN architecture and training configuration are described in the Pattern Reconstruction Using CycleGAN subsection. The reconstruction module consists of two generators and two discriminators, utilizes a residual-network generator architecture with nine residual blocks, employs a PatchGAN discriminator, and is trained using adversarial and cycle-consistency losses. This configuration enables bidirectional domain mapping and facilitates the reconstruction of incomplete cultural pattern structures.

Step 6: Artistic Style Generation (SPADE)
The reconstructed patterns are then processed through the SPADE module to perform segmentation-guided artistic style synthesis. The SPADE generator configuration is described in the Artistic Style Generation Using SPADE subsection. The module employs SPADE residual blocks, spatially adaptive normalization layers, and semantic conditioning derived from SegFormer segmentation maps and CAFFM feature representations. By conditioning image generation on segmentation maps, the synthesized outputs maintain structural alignment with the original cultural motifs while incorporating artistic style characteristics.

Step 7: Performance Estimation
The proposed framework was evaluated using multiple segmentation and image-quality assessment metrics, including Segmentation Accuracy, IoU, Dice Similarity Coefficient (Dice), SSIM, PSNR, and FID. To assess experimental consistency, all experiments were repeated five times using different random seeds, and the results are reported as mean ± standard deviation. Statistical significance was evaluated using paired t-tests, with a significance threshold of p < 0.05.

Dataset Collection
In the proposed framework of SegCycle-SPADE, two datasets are employed for cultural pattern understanding and style learning. The first dataset employed in the proposed framework is the Miao Batik cultural motif dataset. This dataset contains cultural motifs of Miao Batik, which are generally images of traditional Miao Batik containing motifs like animals, plants, and geometric shapes, which are used in Miao ethnicity art. This dataset contains images with rich texture information, which are generally important for preserving cultural heritage. The second dataset employed in the proposed framework of SegCycle-SPADE is the WikiArt dataset. This dataset contains a huge number of artworks, which are generally from different styles. This dataset is employed for complex style learning, which is generally useful for enhancing artworks. This dataset has 80,000 artworks in HD, with 27 different designs, making it more useful for DL-based style generation or transformation tasks.

Miao Batik Dataset:
The Miao Batik dataset (https://doi.org/10.5281/zenodo.20807658) comprises 15,148 images of batik patterns depicting culturally significant motifs. The images include a variety of traditional designs, such as animal motifs (e.g., butterflies and fish), plant-based patterns, and geometric motifs that carry cultural symbolism. This dataset is an important component of the proposed framework because it provides authentic cultural structures that enable the segmentation module to accurately identify and preserve motif boundaries during pattern reconstruction (Table 1). In this study, culturally important motifs refer to symbolic visual elements commonly documented in Miao cultural heritage literature and represented within the dataset annotations, including birds, butterflies, floral patterns, and ancestral totems. These motifs were selected based on their documented cultural significance and recurring presence in traditional Miao batik and embroidery designs rather than solely on their frequency of occurrence or spatial composition. The expert-guided motif annotations and segmentation labels were obtained from the original Miao Batik dataset source and were used directly in this study. No additional manual annotation, relabeling, or expert-guided annotation procedures were performed by the authors. The existing annotations provided by the dataset creators were utilized for semantic segmentation training and evaluation.

Parameter	Description
Dataset Name	Miao Batik Cultural Pattern Dataset
Dataset Source	Zenodo Repository (DOI: 10.5281/zenodo.20807658)
Domain	Intangible Cultural Heritage (ICH) / Textile Pattern Analysis
Total Images	15,148 images
Image Type	Digital images of traditional Miao batik textile patterns
Pattern Categories	Animal motifs, plant motifs, and geometric motifs
Cultural Origin	Miao ethnic culture, Guizhou Province, China
Original Image Resolution	High-resolution images (typically ≥ 1,000 pixels on the shorter side) captured as raw scans or photographs prior to preprocessing
Training Resolution	Images resized to 256 × 256 pixels during preprocessing
Annotation Availability	Existing segmentation annotations provided by the dataset creators were used without modification for training and evaluation. No additional manual annotation, relabeling, or expert-confirmation procedures were performed in this study.
Application in This Study	Cultural motif segmentation, feature extraction, motif preservation, and pattern reconstruction

Table 1: Characteristics of the Miao Batik Cultural Pattern Dataset. Summary of the Miao Batik cultural motif dataset used for semantic segmentation, cultural pattern analysis, and motif reconstruction. The table describes the dataset source, image characteristics, motif categories, cultural origin, image resolution, and its role within the proposed SegCycle-SPADE framework.

WikiArt Dataset:
The WikiArt dataset [https://www.wikiart.org/] is a popular large-scale digital art repository that comprises over 80,000 images from 27 different artistic styles in Table 2. The styles include Renaissance art, Impressionism, Cubism, and Surrealism. The dataset is very important in the suggested framework since it offers knowledge that allows the SPADE module to create culturally enriched patterns while maintaining structural information obtained from the segmentation process. The dataset comprises 56,000 pictures for learning and 12,000 pictures for both validation and testing. The images in the dataset help in training the DL model effectively.

Parameter	Description
Dataset Name	WikiArt Dataset
Dataset Source	WikiArt Repository (https://www.wikiart.org/)
Domain	Digital Art and Paintings
Total Images	Approximately 80,000 artworks
Training Images	56,000
Validation Images	12,000
Test Images	12,000
Artistic Styles	27 artistic styles, including Renaissance, Impressionism, Cubism, Surrealism, Expressionism, Realism, and Abstract Art
Original Image Resolution	Original image resolutions vary across artworks and sources. Images were resized to a uniform resolution of 256 × 256 pixels during preprocessing.
Training Resolution	Images resized to 256 × 256 pixels during preprocessing prior to artistic style learning and segmentation-guided image synthesis.
Dataset Partitioning	Stratified random partitioning (70% training, 15% validation, and 15% testing) using a fixed random seed of 42
Style Sampling Strategy	Stratified proportional sampling was employed to preserve the distribution of the 27 artistic styles across the training, validation, and testing subsets
Application in This Study	Artistic style learning, style representation, and segmentation-guided artistic synthesis

Table 2: Characteristics of the WikiArt Dataset. Summary of the WikiArt dataset used for artistic style learning and style-guided image synthesis. The table describes the dataset source, image distribution, artistic-style coverage, dataset partitioning, image resolution, and its application within the proposed SegCycle-SPADE framework.

The Miao Batik dataset contains 15,148 images representing traditional Miao cultural motifs.32 The dataset is publicly available through Zenodo (https://doi.org/10.5281/zenodo.20807658). Prior to model training, all images were visually inspected, resized to 256 × 256 pixels, normalized, and screened for corrupted or duplicate samples. Quality-control screening did not result in the exclusion of any images; therefore, all 15,148 images were retained for subsequent analysis. The dataset was randomly partitioned into training (70%), validation (15%), and testing (15%) subsets using a fixed random seed of 42 to ensure reproducibility of the dataset splits. The WikiArt dataset was accessed through the official WikiArt repository (https://www.wikiart.org/) and contains more than 80,000 artworks spanning 27 artistic styles. For style-learning experiments, 56,000 images were used for training, 12,000 for validation, and 12,000 for testing.

Data Preprocessing
Before training the proposed SegCycle-SPADE framework, the Miao Batik cultural motif dataset and the WikiArt dataset were subjected to several preprocessing steps to ensure consistent image quality and accurate feature representation. The same fundamental preprocessing pipeline, including Gaussian denoising, normalization, resizing to a consistent input resolution, and image-quality verification, was applied to both datasets. However, the datasets served distinct roles within the framework. The WikiArt dataset was used for artistic style learning and synthesis, whereas the Miao Batik dataset was used primarily for cultural motif segmentation and reconstruction. No dataset-specific preprocessing changes were made beyond these task-specific roles. To ensure consistent processing by the DL model, images were first resized to a fixed resolution. This step was necessary because images in both datasets varied in resolution depending on their source. Resizing improved computational efficiency and prevented dimensional inconsistencies from affecting training. The second preprocessing step was normalization, in which pixel values were scaled to a standardized range to stabilize gradient updates during training. The images were then processed using Gaussian filtering to reduce noise that could interfere with cultural motif structures. For the Miao Batik dataset, existing cultural motif segmentation masks obtained directly from the original dataset source were used without modification for semantic segmentation training and evaluation. No new segmentation masks were generated specifically for this study.

Unless otherwise indicated, equations describing established methods were adapted from previous studies and are cited accordingly. Equations describing the proposed CAFFM and the composite SegCycle-SPADE optimization framework were developed by the authors for this study.

Let an input image from the dataset be represented as Equation 1:

  (1)

Here, H, W, and C refer to the image height, width, and number of color channels, respectively. All images are resized to a fixed dimension H′ × W′ as shown in Equation 2:

Here, I_r is the resized image. The normalized pixel values are computed according to Equation 3:

   (3)

This helps stabilize the training procedure. Noise filtering is performed using a Gaussian filter as shown in Equation 4:

Here, G(σ) is a Gaussian filter and * represents convolution. A 5 × 5 kernel Gaussian filter with a standard deviation of σ = 1.0 was used. Reflective padding was applied to border pixels in order to reduce edge artifacts. Prior to scaling and normalization, the denoising process was carried out immediately following image loading. To guarantee uniform preprocessing throughout the study, the same filter configuration was applied to every image in the Miao Batik and WikiArt datasets. For the Miao Batik dataset, the segmentation masks are created according to Equation 5:

Here, M is a segmentation mask used to guide the SegFormer model.

The preprocessing pipeline starts with the acquisition of images from the Miao Batik dataset and the WikiArt dataset, as shown in Figure 3. No data augmentation techniques were employed during training. After preprocessing, which included resizing, normalization, Gaussian denoising, and segmentation-mask preparation, the images were directly used for training, validation, and testing of the proposed SegCycle-SPADE framework. The initial step of the preprocessing pipeline involves resizing the images to a fixed size, allowing the DL model to process the images more efficiently. The second step involves normalization, which converts pixel values to a standardized numerical range and facilitates model convergence. The third step involves noise removal, whereby the images are cleared of unwanted noise to improve pattern recognition. In addition, for the Miao Batik dataset, cultural motif regions are annotated, allowing the generation of masks that are used in the segmentation module of the proposed model, ensuring that cultural motifs are preserved during generation. The final output of this stage is a clean dataset that is ready for training the proposed model's segmentation and generation modules.

Figure 3. Data-Preprocessing Pipeline for Cultural Heritage Images. Workflow illustrating image preprocessing procedures applied prior to model training. The pipeline includes dataset acquisition, image resizing, normalization, Gaussian denoising, existing cultural-motif annotations and segmentation-mask preparation. The resulting processed images and masks are used as inputs for semantic segmentation and pattern reconstruction. Please click here to view a larger version of this figure.

Every image was subjected to a quality-control process before model training. The Miao Batik dataset contained 15,148 images. Corrupted, duplicate, and low-quality samples had already been addressed by the original dataset creators; therefore, no additional images were excluded during quality-control screening in this study. Consequently, all 15,148 images were retained for analysis. Images were loaded in RGB format during preprocessing and resized to 256 × 256 pixels using bilinear interpolation. Pixel values were scaled from the range [0, 255] to [0, 1] by dividing by 255. Channel-wise normalization was then applied using mean values of [0.485, 0.456, 0.406] and standard deviation values of [0.229, 0.224, 0.225] for the red, green, and blue channels, respectively. The preprocessing pipeline included image loading, RGB conversion, resizing, pixel-value scaling, normalization, and tensor conversion. When necessary, grayscale images were converted to three-channel RGB format to maintain compatibility with the DL models. Following preprocessing, the images were partitioned into training, validation, and testing subsets. All stages of the SegCycle-SPADE framework—including semantic segmentation, feature fusion, motif reconstruction, and SPADE-based artistic synthesis—used a consistent input resolution of 256 × 256 pixels.

Semantic Pattern Segmentation using SegFormer
After this preprocessing step, the cleaned and normalized images of the Miao Batik cultural motif dataset and the WikiArt dataset are fed into the semantic segmentation module based on the proposed framework of SegFormer-B2. The purpose of this step is to correctly identify and separate cultural motifs that are present in heritage patterns. The proposed framework of SegFormer is based on a transformer architecture that incorporates a hierarchical Transformer encoder and a lightweight MLP decoder to accurately capture global contextual information as well as detailed structural information from complex heritage patterns. The model first extracts feature representations at multiple scales from the input image, followed by fusion of these representations through the decoder to generate accurate segmented patterns. This ensures that the patterns in the heritage image are correctly segmented into motifs such as geometric patterns, floral patterns, and symbolic patterns, thereby separating them from the background while maintaining their structural integrity. This structural information is subsequently used during the later stages of pattern generation within the SegCycle-SPADE framework.

Let the preprocessed input image be represented as Equation 6:

    (6)

The SegFormer encoder splits the image into patches and extracts hierarchical features using transformer layers, as shown in Equation 7¹:

Here, F denotes the multiscale feature maps obtained from the transformer encoder. These features encode both local texture patterns and global contextual relationships among motif regions. The SegFormer model extracts feature maps at different scales as defined in Equation 8:

The use of multiscale representations facilitates the detection of patterns of different sizes within cultural motifs. Finally, the features are combined using the MLP decoder as shown in Equation 9¹:
 

Here, S denotes the final predicted segmentation map.

The SegFormer-B2 backbone was initialized using ImageNet-pretrained weights and subsequently fine-tuned on the Miao Batik dataset for cultural motif segmentation. The pretrained checkpoint was obtained from the official SegFormer implementation. Specifically, the ImageNet-1K pretrained MIT-B2 checkpoint (mit_b2.pth) provided by the official SegFormer repository was used to initialize the SegFormer-B2 backbone prior to fine-tuning. The pretrained weights correspond to the MIT-B2 backbone released by the SegFormer authors and served as the initialization model for semantic segmentation training. The remaining task-specific layers were initialized using the default PyTorch weight-initialization procedures prior to training.

A four-stage hierarchical Transformer encoder with overlapping patch embeddings is used in the architecture. In the initial stage, the patch size was set to 7 × 7 with a stride of 4. For each of the four encoder stages, the embedding dimensions were 64, 128, 320, and 512. Across the four stages, there were 1, 2, 5, and 8 multihead self-attention heads, and the corresponding encoder depths were 3, 4, 6, and 3 layers. Multiscale features retrieved from the encoder were aggregated using a lightweight all-MLP decoder. Before creating the final segmentation map, the decoder projected features from every encoder stage into a single embedding space. All Transformer blocks used the Gaussian Error Linear Unit (GELU) activation function. A dropout rate of 0.1 was used during training to improve generalization and reduce overfitting.

During the training phase, the SegFormer framework receives the preprocessed images from both datasets. The images from the Miao Batik dataset contain motif regions that serve as labels for supervised learning of the segmentation model. The Transformer encoder processes the entire image and captures long-range relationships among image components, which is particularly important for traditional batik patterns containing spatially distributed motifs. The hierarchical feature extraction mechanism simultaneously captures local structures such as repeated geometric textures. The MLP decoder processes these extracted features and produces a segmentation mask highlighting motif regions. The segmentation maps generated during this stage are subsequently used as structural inputs for the attention module and pattern reconstruction stage, thereby helping preserve the authenticity of generated patterns.

Cultural motifs were divided into different classes for semantic segmentation according to their visual and cultural characteristics. The segmentation taxonomy comprised animal motifs (e.g., butterflies, fish, birds, and other symbolic fauna), plant motifs (e.g., flowers, leaves, vines, and botanical patterns), geometric motifs (e.g., repeating shapes, borders, and abstract geometric structures), and background regions representing non-motif areas. Pixel-level segmentation masks were used to assign each pixel to one of the predefined classes. Integer labels were encoded as follows: Label 0 = background, Label 1 = animal motifs, Label 2 = plant motifs, and Label 3 = geometric motifs. The segmentation annotations and motif labels were obtained directly from the original Miao Batik dataset source. No additional manual annotation, relabeling, boundary verification, or expert-confirmation procedures were performed in this study. The existing annotations provided by the dataset creators were used without modification for training and evaluation.

The SegFormer model for cultural motif segmentation processes the preprocessed images from the Miao Batik dataset and the WikiArt dataset using a transformer-based mechanism for accurate detection of cultural pattern regions, as shown in Figure 4. First, the input image containing traditional cultural motifs is provided to the SegFormer model. The Transformer encoder extracts both global contextual information and local structural features from image patches. The resulting multiscale features are provided to the segmentation decoder, which utilizes a Mix Feed-Forward Network to refine feature representations and improve motif detection. The output of the SegFormer model is a segmentation mask that highlights pixels corresponding to cultural patterns while suppressing irrelevant background information. The isolated cultural patterns then serve as structural guidance for the subsequent stages of the SegCycle-SPADE framework.

Figure 4. SegFormer-B2-Based Semantic Segmentation of Cultural Motifs. Architecture of the SegFormer-B2 semantic segmentation module used for cultural motif extraction and trained exclusively on the Miao Batik dataset. The framework consists of a Transformer-based encoder for multiscale feature extraction and a lightweight segmentation decoder for generating motif masks. The model predicts four semantic classes—animal, plant, geometric, and background—where the resulting segmentation masks identify culturally significant motif regions while suppressing irrelevant background information. These segmentation outputs provide structural guidance for the subsequent feature-fusion, reconstruction, and artistic-synthesis stages of the proposed SegCycle-SPADE framework. Please click here to view a larger version of this figure.

There is no need to create new segmentation annotations in the suggested framework. The Miao Batik dataset was acquired from an already existing public source, and the model was trained using the related motif information supplied by the dataset creators. During the learning process, the SegFormer model produced semantic segmentation maps. As a result, the current study did not require any additional manual annotation software, annotation guidelines, or annotation quality-control procedures.

CAFFM
To improve the interaction between the features of semantic segmentation and the representations of generative reconstruction, the study also proposed a CAFFM. The SegFormer-B2 encoder provides semantic feature maps to the CAFFM module, whereas the CycleGAN reconstruction step provides generative feature maps. First, linear projection layers are used to project both feature streams into a common embedding space. For cross-attention computation, query (Q), key (K), and value (V) representations are created using the generated feature tensors. The relationships between structural motif information and artistic style elements are then modeled using multihead cross-attention. Layer normalization, residual connections, and feed-forward layers with GELU activation are then applied to the fused feature representations. The SPADE-based synthesis stage receives the CAFFM module’s output, allowing for the preservation of culturally meaningful themes without sacrificing artistic coherence.

To guarantee feature compatibility, the input features are first projected using linear projection layers onto a shared embedding space with an embedding dimension of 256. The interconnections between semantic structural information and generative style representations are then modeled using a MultiHead Cross-Attention mechanism with eight attention heads. Cross-attention computation uses the Query (Q), Key (K), and Value (V) vectors that are produced by specific linear transformation layers. To increase training stability and maintain feature integrity, residual connections and Layer Normalization are employed to further enhance the fused feature representations. Non-linear feature learning is then improved by applying a feed-forward network with the Gaussian Error Linear Unit (GELU) activation function. An output feature representation of dimension 256 is generated by the module and sent to other stages for creative synthesis. The CAFFM successfully combines artistic style data with cultural motif structures by using a cross-attention-based fusion technique, which enhances motif preservation and visual coherence in the reconstructed cultural heritage patterns.

In the proposed system, the structural features are derived from the Miao Batik dataset, while the stylistic features are learned from the WikiArt dataset. Let the feature map derived from the SegFormer segmentation network using images from the Miao Batik dataset be denoted by F_s, while the feature map derived from the style-feature extraction branch using WikiArt images is denoted by F_a. The proposed CAFFM combines the two feature domains using a cross-attention mechanism to learn both structural and stylistic dependencies of cultural patterns. Table 3 reports all architectural characteristics, including embedding dimensions, normalization layers, activation functions, and attention-head configuration. For an input image size of 256 × 256 pixels, the SegFormer-B2 encoder produced semantic feature maps of size 64 × 64 × 128, while the style-feature extraction branch produced feature maps of size 64 × 64 × 128. Both feature maps were projected through learnable linear layers into a shared embedding space of dimension 256 prior to cross-attention fusion.

Parameter	Configuration
Proposed Framework	SegCycle-SPADE
Semantic Segmentation Model	SegFormer-B2
SegFormer Encoder Stages	4
Weight Initialization	ImageNet-1K Pretrained SegFormer-B2 (MIT-B2 Backbone)
Checkpoint Source	Official NVIDIA SegFormer Repository (https://github.com/NVlabs/SegFormer); checkpoint: segformer.b2.512x512.ade.160k
Fine-Tuning Strategy	End-to-end fine-tuning on the Miao Batik dataset
Patch Embedding Size	7 × 7
Patch Stride	4
Embedding Dimensions	64, 128, 320, and 512
Encoder Depths	3, 4, 6, and 3
Attention Heads	1, 2, 5, and 8
Decoder Type	All-MLP Decoder
Activation Function	GELU
Dropout Rate	0.1
Feature Enhancement Module	Cross-Attention Cultural Feature Fusion Module (CAFFM)
CAFFM Embedding Dimension	256
CAFFM Attention Heads	8
CAFFM Fusion Scales	4
Reconstruction Model	CycleGAN
Style Generation Model	SPADE
Cultural Dataset	Miao Batik Dataset
Style Dataset	WikiArt Dataset
Miao Batik Images	15,148
WikiArt Images	Approximately 80,000
Input Image Resolution	256 × 256 pixels
Color Format	RGB
Interpolation Method	Bilinear Interpolation
Denoising Method	Gaussian Filtering
Gaussian Kernel Size	5 × 5
Gaussian Sigma (σ)	1
Normalization Range	[0, 1]
Batch Size	16
Number of Epochs	100
Optimizer	Adam
Learning Rate	0.0002
Adam Parameters	β₁ = 0.5, β₂ = 0.999, ε = 1 × 10⁻⁸, weight decay = 0
Loss Functions	Segmentation Loss, Adversarial Loss, Cycle-Consistency Loss, Reconstruction Loss, Motif-Preservation Loss, and Style-Consistency Loss
Dataset Split Strategy	Training / Validation / Testing
Training Set	70%
Validation Set	15%
Testing Set	15%
Random Seed	42
Evaluation Metrics	Segmentation Accuracy, IoU, Dice Coefficient, SSIM, PSNR, and FID
Programming Language	Python 3.8.10
Deep Learning Framework	PyTorch 1.10.0
Hardware Platform	NVIDIA RTX 3090 GPU (24 GB VRAM), Intel Core i7 (11th Generation), 32 GB RAM
Operating Environment	Ubuntu 20.04 LTS

Table 3: Experimental Setup and Model Configuration. Summary of the architectural components, training configuration, preprocessing settings, datasets, optimization parameters, evaluation metrics, and computational environment used for implementation and evaluation of the proposed SegCycle-SPADE framework.

The attention module first maps the feature map into query, key, and value representations using Equation 10:

Here, W_q, W_k, and W_v are learnable weight matrices. The attention weights are calculated based on the similarity between the query vector and the key vector using Equation 11¹⁷: 

Here, d_k is the dimension of the key vector. The enhanced feature representation is computed by multiplying the attention weights with the value matrix using Equation 12:

Here, F_a is the enhanced feature representation of the feature map. The enhanced feature representation is computed by combining the original segmentation features with the enhanced features obtained using the attention mechanism using Equation 13:

                                

Here, F_e is the enhanced feature map used for pattern reconstruction. The CAFFM is extended with a multiscale cross-attention mechanism to extract features for cultural motifs at different scales. Let F_sⁱ denote the segmentation features at scale i, while F_a^j denotes the artistic-style features at the same scale. The final feature representation is defined using Equation 14:

  

Here, Q_i, K_i, and V_i represent the query, key, and value matrices at scale i, respectively, and N denotes the number of feature scales. In this study, N = 4, corresponding to feature maps extracted at spatial resolutions of 1/4, 1/8, 1/16, and 1/32 of the input image size. These multiscale feature representations were fused using learnable attention weights prior to reconstruction and artistic synthesis. The above extension enables the method to extract global cultural motifs and textures to reconstruct cultural patterns. CAFFM is situated between the SegFormer-based segmentation stage and the SPADE-based creative synthesis stage in the suggested SegCycle-SPADE system. First, while CycleGAN concurrently creates reconstruction features with stylistic and pattern-related information, SegFormer-B2 recovers semantic feature maps that depict the structural properties of cultural motifs. The CAFFM module receives these two feature streams and uses cross-attention-based fusion to identify relationships between structural and artistic representations. In order to create culturally authentic patterns while maintaining semantic consistency and structural features, the resulting fused feature maps are then sent to the SPADE module for segmentation-guided creative synthesis.

Pattern Reconstruction using CycleGAN
Once the attention-based feature enhancement stage is done, the feature maps will contain the enhanced cultural motif structures. However, the feature maps may still contain incomplete or damaged pattern regions. Therefore, to address the problem of reconstructing the patterns, the proposed framework will make use of the CycleGAN model. In the proposed framework, the two domains will be the original cultural motif patterns and the reconstructed cultural motif patterns. Using the enhanced features from the previous stages, the CycleGAN model will reconstruct the cultural patterns while maintaining structural consistency. This will ensure that cultural patterns such as geometric patterns, floral patterns, and symbolic patterns maintain their spatial arrangement. The original Miao Batik images were subjected to random masking and partial occlusion operations to generate incomplete or damaged cultural motifs. These degradation procedures simulated missing pattern regions and structural damage commonly observed in deteriorated cultural heritage artifacts. The degraded images were used as inputs to the reconstruction module, while the corresponding original images served as reference images for performance evaluation and reconstruction-quality assessment. This strategy enabled the model to learn the restoration of missing motif structures while preserving semantic consistency and cultural characteristics.

Let X be the domain of incomplete cultural patterns and Y be the domain of reconstructed cultural patterns. The two generators learn to perform bidirectional mapping using Equation 15:

 

Here, G_E is used to reconstruct motif structures and F_M is used to map the patterns back to the original domain. The adversarial loss is used to ensure that the reconstructed patterns are realistic (Equation 16)³⁰

Here, D_Y is the discriminator used to distinguish real and reconstructed patterns, and G_E(x) denotes the generated reconstructed pattern. The CycleGAN also enforces cycle consistency to preserve the structural layout of cultural motifs (Equation 17)³⁰.

The final loss function is defined using Equation 18³⁰:

Here, λ_i denotes the weighting coefficient for the cycle-consistency loss and was set to 10 during training, consistent with the original CycleGAN formulation. This value was selected to balance adversarial learning and reconstruction consistency between the source and target domains.

The CycleGAN reconstruction module consists of two generators and two discriminators for bidirectional image translation. Each generator follows a residual-network architecture comprising an initial 7 × 7 convolution layer, followed by two downsampling convolution layers, nine residual blocks, and two upsampling layers. All convolution operations employ a kernel size of 3 × 3, except for the input layer, which utilizes a 7 × 7 kernel for enhanced feature extraction. Instance Normalization is applied after each convolution layer to improve training stability, while ReLU activation is used throughout the generator network. The output layer employs a Tanh activation function to generate normalized image outputs. The discriminator follows a 70 × 70 PatchGAN architecture consisting of a series of convolutional layers with kernel sizes of 4 × 4 and progressively increasing feature channels. Instance Normalization and LeakyReLU activation (negative slope = 0.2) are employed in the discriminator to improve feature discrimination and adversarial learning. The CycleGAN module receives preprocessed cultural motif images as input and generates reconstructed pattern representations, which are subsequently forwarded to the CAFFM for integration with segmentation features. CycleGAN training was performed using the Adam optimizer (β₁ = 0.5, β₂ = 0.999) with an initial learning rate of 0.0002. The learning rate was maintained for the first 100 epochs and subsequently linearly decayed to zero over the remaining training period. A replay buffer containing 50 previously generated images was employed to stabilize discriminator training. The generators and discriminators were updated using a 1:1 update ratio, with one generator update followed by one discriminator update during each training iteration.

The reconstruction process of the CycleGAN is based on the enhanced feature maps obtained using the CAFFM mechanism in Figure 5. The feature maps contain enhanced cultural motifs obtained from the previous stages of segmentation and CAFFM. The feature maps are used as input to the first generator G_E. The first generator G_E learns the mapping required to reconstruct cultural patterns. The outputs are then fed to the discriminator D_Y to distinguish between real cultural patterns and reconstructed patterns. The discriminator D_Y helps the generator G_E produce realistic outputs. To ensure that cultural patterns are not distorted during reconstruction, the second generator F_M performs cycle-consistency mapping. The second generator F_M maps the outputs back to the original domain. The outputs are then evaluated by the discriminator to ensure realism. The final outputs are subsequently forwarded to the SPADE module for artistic style generation in the next stage of the proposed SegCycle-SPADE framework.

Figure 5. CycleGAN-Based Pattern Reconstruction Workflow. Workflow of the CycleGAN reconstruction module. Attention-enhanced feature representations are provided as inputs to the generator network to reconstruct incomplete cultural motifs. The discriminator evaluates reconstructed outputs against reference patterns, while cycle-consistency mapping preserves structural integrity during bidirectional image translation. Reconstructed motifs are subsequently forwarded to the SPADE module for artistic style synthesis. Please click here to view a larger version of this figure.

Artistic Style Generation using SPADE
Subsequently, the reconstructed cultural motifs are subjected to the SPADE module for the generation of artistic style. The main goal of the SPADE module is to add more visually rich textures of artistic style to the reconstructed cultural motifs without compromising the structural layout of the motifs. The SPADE module is a conditional image generation technique that makes use of the concept of image segmentation for the generation of images. Multichannel semantic maps that corresponded to the predetermined motif classes were encoded from the semantic segmentation masks produced by SegFormer-B2. The SPADE generator received these encoded maps as conditioning inputs. The spatially adaptive scaling (γ) and bias (β) parameters were produced by processing the semantic maps through convolutional layers within each SPADE layer. The generator was thus able to maintain motif boundaries, structural layouts, and semantic information during artistic synthesis by applying these parameters to the normalized feature activations. Semantic guiding was able to affect both high-level structural reconstruction and low-level texture synthesis since the conditioning process was carried out at several layers of the SPADE generator. As a result, created artistic patterns incorporated stylistic traits gained from the WikiArt dataset while maintaining the cultural motif structures found during the segmentation stage.

The WikiArt dataset, which includes works from 27 different artistic categories—such as Renaissance, Impressionism, Cubism, Expressionism, Surrealism, Realism, and Abstract Art—was used as the main resource for understanding artistic styles. In order to expose the model to a variety of creative traits and enhance style generalization, style images were randomly selected from the various categories during training. The dataset was divided into training, validation, and testing subsets with a balanced representation of artistic categories in order to lessen style bias. To ensure balanced representation of all 27 artistic categories, stratified sampling was used. Samples from each category were proportionally allocated to the training, validation, and testing subsets while preserving the original class distribution. The CAFFM module was used to merge the style aspects that were derived from WikiArt images with the reconstruction features produced by CycleGAN. In order to allow the synthesis network to transfer artistic qualities while maintaining the semantic structure and cultural motif boundaries derived from the segmentation maps, the resulting fused representations were supplied as conditioning information to the SPADE generator. The suggested framework was able to produce culturally authentic artistic patterns with consistent structural and stylistic representations thanks to this style-conditioning technique.

SPADE also introduces spatially adaptive parameters that depend on the segmentation map. The parameters can be defined as shown in Equation 19¹:

Here, γ(S) is the spatially varying scale parameter and β(S) is the spatially varying bias parameter. The parameters can help the generator create different textures and styles depending on the semantic region in the images. The final cultural artwork can be defined as shown in Equation 20:

 

Here, G_E is the SPADE generator, X_r^P is the reconstructed pattern, and SM is the segmentation map. The final artwork maintains the structural integrity of cultural motifs while enhancing artistic appearance. A composite loss function that balances semantic segmentation accuracy, cultural motif preservation, pattern reconstruction quality, and artistic style consistency was used to optimize the suggested SegCycle-SPADE architecture. A weighted mixture of segmentation loss (L_seg), adversarial loss (L_adv), cycle-consistency loss (L_cycle), reconstruction loss (L_recon), motif-preservation loss (L_motif), and style-consistency loss (L_style) was used to establish the overall training objective. The total loss function is defined in Equation 21:

  (21)

In order to guarantee structural consistency between the input and reconstructed cultural motifs, the cycle-consistency loss coefficient was set to 10. To maintain fine-grained motif features and enhance visual accuracy, the reconstruction loss coefficient was set at 5. To highlight the preservation of culturally essential structural patterns during artistic synthesis, a coefficient of 2 was used for the motif-preservation loss. In order to promote artistic style transfer without unduly distorting semantic motif structures, the style-consistency loss coefficient was adjusted to 1. To maintain a balanced contribution between realistic image production and precise motif segmentation, equal weights were given to the segmentation and adversarial losses. The WikiArt dataset's feature-based style representations were used to calculate the style-consistency loss. In particular, artistic style traits were captured using Gram matrix correlations that were taken from deep feature maps. The Frobenius norm difference between the Gram matrices of the target style image and the generated image was used to calculate the style loss, as shown in Equation 22:

Here, G_l is the Gram matrix computed from the l^th feature layer, I_gen denotes the generated artistic image, I_style denotes the target style image from the WikiArt dataset, and F denotes the Frobenius norm. This formulation enables the proposed framework to preserve artistic characteristics while maintaining the structural and semantic integrity of cultural motifs.

The fused feature representations produced by the CAFFM module are used as conditioning inputs to the SPADE module, which serves as the artistic synthesis component of the proposed framework. The SPADE generator comprises seven SPADE residual blocks with a latent feature dimension of 256 channels and generates output images at a resolution of 256 × 256 pixels. Semantic segmentation maps obtained from the SegFormer–CAFFM module are used as conditioning inputs throughout the SPADE residual layers. SPADE layers are applied prior to the activation functions within each residual block, enabling segmentation-guided semantic conditioning. Through successive upsampling and convolutional operations, the latent feature representation is progressively refined to generate high-resolution artistic patterns while preserving semantic consistency and motif structure. LeakyReLU activation functions are used throughout the generator, and a Tanh activation function is applied at the output layer to produce normalized images.

Algorithm and Workflow
The SegCycle-SPADE framework integrates semantic segmentation, feature fusion, pattern reconstruction, and artistic style synthesis within a unified training pipeline. The workflow begins with dataset acquisition and preprocessing, including image resizing, normalization, denoising, and segmentation-mask preparation. Preprocessed images are subsequently processed using the SegFormer-B2 segmentation network to extract semantic motif representations. The resulting segmentation features are fused with reconstruction features through the CAFFM, enabling interaction between structural motif information and artistic feature representations. The fused feature maps are then supplied to the CycleGAN reconstruction module, which restores incomplete cultural motif structures while preserving semantic consistency. The reconstructed patterns are subsequently provided to the SPADE generator for segmentation-guided artistic synthesis, allowing stylistic enhancement while maintaining motif boundaries and structural authenticity. During training, model parameters are optimized using the composite loss function described in Equations 21 and 22, which balances segmentation accuracy, motif preservation, reconstruction quality, and artistic style consistency. A detailed step-by-step implementation workflow, including dataset loading, preprocessing, model initialization, training iterations, validation procedures, checkpoint selection, and final evaluation, is provided in Supplementary File 1 (Algorithm 1). The complete algorithmic workflow was implemented using a batch size of 16, a learning rate of 0.0002, and 100 training epochs. A fixed random seed of 42 was used for dataset partitioning, model initialization, and all training experiments. The seed was applied consistently across Python, NumPy, and PyTorch random number generators to ensure reproducibility. The Adam optimizer was configured with β₁ = 0.5, β₂ = 0.999, and no weight decay (weight decay = 0). Model checkpoints were selected based on the highest validation IoU score achieved on the validation dataset.

Loss Function Optimization
To preserve cultural motif structures during reconstruction and artistic synthesis, the proposed framework employs a composite optimization objective that combines segmentation, adversarial, cycle-consistency, reconstruction, motif-preservation, and style-consistency losses. The complete mathematical formulation, weighting coefficients, and style-loss definition are provided in Equations 21 and 22 within the Artistic Style Generation using SPADE subsection. The motif-preservation component penalizes structural deviations between predicted motif regions and the corresponding ground-truth segmentation masks, thereby encouraging preservation of culturally significant pattern structures throughout the reconstruction process.

Results

The proposed SegCycle-SPADE framework was evaluated using two datasets: the Miao Batik cultural motif dataset and the WikiArt dataset. The Miao Batik dataset contains traditional cultural heritage images and was primarily used for semantic segmentation and structural reconstruction tasks, whereas the WikiArt dataset contains diverse artistic styles and was used for artistic style learning and generation. Images from both datasets were processed through the complete SegCycle-SPADE pipeline, including semantic segmentation, CAFFM, pattern reconstruction, and SPADE-based artistic style generation. The integration of cultural motif information and artistic style representations enabled the framework to generate culturally meaningful and visually coherent outputs.

All experiments were conducted on a GPU-enabled workstation equipped with an Intel Core i7 processor and an NVIDIA GPU. Specifically, experiments were performed on a workstation equipped with an Intel Core i7 (11th Generation) CPU, an NVIDIA RTX 3090 GPU with 24 GB VRAM, and 32 GB of system memory. The models were implemented using Python 3.8 and PyTorch 1.10 with CUDA acceleration. Training was performed using a single-GPU setup without distributed training. Standard FP32 precision was used throughout all experiments, and mixed-precision training was not employed. The Adam optimizer was used with a batch size of 16 for 100 training epochs. Table 3 summarizes the implementation parameters and computational environment used in this study.

The proposed architecture consists of four major components: SegFormer-B2 for semantic segmentation, CAFFM for feature fusion, CycleGAN for motif reconstruction, and SPADE for artistic style synthesis. Images from both datasets were resized to 256 × 256 pixels prior to training. This standardized resolution ensured computational efficiency while preserving important motif details and contributed to stable adversarial training of both CycleGAN and SPADE modules.

The performance of the proposed framework was evaluated using segmentation and image-quality assessment metrics, including Segmentation Accuracy, IoU, Dice Similarity Coefficient (Dice), SSIM, PSNR, and FID. Visual authenticity was assessed qualitatively through visual inspection of the generated cultural patterns. The qualitative assessment focused on motif preservation, semantic consistency, cultural characteristics, and artistic appearance relative to the reference cultural motifs. Visual authenticity was not used as an independent quantitative evaluation metric.

Quantitative evaluation of the proposed framework was performed using the following metrics.

Segmentation Accuracy was computed using Equation 23:

Here, TP, TN, FP, and FN denote true positives, true negatives, false positives, and false negatives, respectively.

The IoU was calculated using Equation 24:

 

The Dice Similarity Coefficient (Dice) was computed using Equation 25:

The SSIM was used to assess perceptual image similarity and is defined using Equation 26³³:

  (26)

Here, μ denotes the mean intensity, σ denotes the variance, σ_xy denotes the covariance between images, and C₁ and C₂ are stabilizing constants.

PSNR is a metric commonly used to evaluate the quality of generated images and it is defined in Equation 27³³:

Here, Max_I denotes the maximum possible pixel value of the image and MSE is the mean square error between the original and reconstructed image.

FID can be calculated using means and covariance matrices as in Equation 28³³:

   (28)

Here, μ_r is the mean value of the real image, μ_g is the mean value of generated image, and Σ_r and Σ_g are the covariance matrix of real and generated images, respectively.

For performance comparison, the proposed framework was evaluated against representative methods commonly used for semantic segmentation, image reconstruction, and artistic image generation. U-Net and DeepLabV3+ were included as segmentation baselines, whereas Pix2Pix and CycleGAN were included as reconstruction baselines. StyleGAN and AttentionGAN were used as representative artistic image generation methods. These comparisons were conducted to assess the effectiveness of SegCycle-SPADE in preserving structural motif information while simultaneously enhancing artistic quality.

Each experiment was repeated five times to evaluate performance stability and reproducibility. A fixed random seed of 42 was used for dataset partitioning to ensure consistent training, validation, and testing subsets across all experiments. Results are reported as mean ± standard deviation. The low standard deviation values observed across Accuracy, IoU, Dice, SSIM, PSNR, and FID indicate that the proposed framework achieved stable performance across repeated experimental runs. Statistical significance was evaluated using paired t-tests, and differences were considered statistically significant when p < 0.05. Because all models were assessed on the same dataset partitions, paired performance measurements across repeated runs were obtained, justifying the use of paired t-tests. To control the family-wise error rate associated with multiple pairwise comparisons, Bonferroni correction was applied, and statistical significance was determined using an adjusted threshold of α/n, where n denotes the number of pairwise comparisons.

The experimental findings strongly support the effectiveness of the proposed SegCycle-SPADE framework. The superior segmentation performance achieved by SegFormer-B2 demonstrates its ability to accurately identify and preserve culturally significant motif boundaries. Improvements in IoU and Dice scores further indicate effective preservation of semantic motif structures throughout the processing pipeline. The integration of CycleGAN and SPADE successfully reconstructed incomplete motif structures while maintaining fine-grained visual details, as reflected by the improved SSIM and PSNR values. Furthermore, lower FID scores indicate that the generated artistic patterns exhibit greater similarity to authentic cultural and artistic images, suggesting improved stylistic coherence and visual realism.

The CAFFM contributed significantly to these improvements by facilitating effective interaction between segmentation-derived structural information and generative style representations. Through cross-attention-based feature fusion, CAFFM enhanced both structural fidelity and artistic authenticity while preserving long-range relationships among cultural motifs.

Figure 6 presents the segmentation accuracy comparison between the proposed SegCycle-SPADE framework and existing methods on the Miao Batik and WikiArt datasets. Traditional segmentation approaches such as U-Net and DeepLabV3+ achieved competitive performance because of their ability to learn spatial representations. However, Pix2Pix and CycleGAN demonstrated lower segmentation accuracy because they were not specifically designed for segmentation tasks. The proposed SegCycle-SPADE framework achieved the highest segmentation accuracy on both datasets owing to the integration of SegFormer-B2 and CAFFM-based feature enhancement.

Figure 6. Comparison of Segmentation Accuracy Across Different Methods. Comparison of segmentation accuracy obtained using U-Net, DeepLabV3+, Pix2Pix, CycleGAN, and the proposed SegCycle-SPADE framework on the Miao Batik and WikiArt datasets. Results are presented as mean ± standard deviation (SD) from five independent runs (n = 5). Error bars represent one standard deviation. Higher values indicate improved segmentation performance. The proposed SegCycle-SPADE framework achieved the highest segmentation accuracy on both datasets. Please click here to view a larger version of this figure.

Figure 7 presents the IoU comparison among the evaluated methods. U-Net and DeepLabV3+ achieved strong overlap performance because of their segmentation-oriented architectures. In contrast, Pix2Pix and CycleGAN produced lower IoU values because their primary objective is image translation rather than semantic segmentation. The proposed SegCycle-SPADE framework achieved the highest IoU values on both datasets, indicating improved localization and preservation of cultural motif regions.

Figure 7. Comparison of Intersection over Union (IoU) Scores for Cultural Motif Segmentation. Comparison of IoU performance among segmentation methods on the Miao Batik and WikiArt datasets. Results are presented as mean ± standard deviation (SD) from five independent runs (n = 5). Error bars indicate one standard deviation. Higher IoU values indicate improved overlap between predicted and reference motif regions and better preservation of motif boundaries. The proposed SegCycle-SPADE framework achieved the highest IoU scores on both datasets. Please click here to view a larger version of this figure.

Figure 8 presents the Dice Similarity Coefficient comparison. Dice measures the overlap between predicted segmentation masks and ground-truth motif regions. Conventional segmentation approaches achieved relatively high Dice scores because of their effectiveness in learning spatial structures. However, the proposed SegCycle-SPADE framework achieved the highest Dice scores on both datasets, demonstrating improved segmentation consistency and motif preservation.

Figure 8. Comparison of Dice Coefficient for Cultural Motif Segmentation. Comparison of Dice coefficient values obtained using different segmentation approaches on the Miao Batik and WikiArt datasets. The Dice coefficient evaluates the overlap between predicted segmentation masks and reference masks, with higher values indicating improved segmentation performance and motif-region identification. The proposed SegCycle-SPADE framework achieved the highest Dice coefficient values across both datasets. Please click here to view a larger version of this figure.

Figure 9 presents the SSIM comparison among reconstructed cultural patterns. GAN-based methods such as Pix2Pix, CycleGAN, and StyleGAN achieved higher SSIM values than traditional segmentation methods because they were designed for image generation. Nevertheless, the proposed SegCycle-SPADE framework achieved the highest SSIM values on both datasets, indicating superior preservation of structural details and artistic coherence.

Figure 9. Comparison of Structural Similarity Index (SSIM). Comparison of Structural Similarity Index (SSIM) values obtained using different reconstruction methods on the Miao Batik and WikiArt datasets. Results are presented as mean ± standard deviation (SD) from five independent runs (n = 5). Error bars indicate one standard deviation. Higher SSIM values indicate greater structural similarity between reconstructed and reference patterns. The proposed SegCycle-SPADE framework achieved the highest SSIM scores across both datasets. Please click here to view a larger version of this figure.

FID was used to evaluate the realism and distributional similarity of generated artistic patterns. FID computation was performed using a pretrained Inception-v3 network, with feature vectors extracted from the pool3 layer, resulting in 2048-dimensional feature representations. Prior to feature extraction, generated and reference images were resized to 299 × 299 pixels using bilinear interpolation and normalized according to the standard Inception-v3 preprocessing protocol. FID was computed using feature distributions extracted from the independent test dataset. The mean and covariance statistics were estimated from the feature embeddings and used within the standard FID formulation.

As shown in Table 4, conventional image-generation approaches such as Pix2Pix and CycleGAN produced relatively high FID scores because they lacked explicit structural guidance. StyleGAN and AttentionGAN achieved lower FID scores through improved feature-learning capabilities. However, the proposed SegCycle-SPADE framework achieved the lowest FID scores on both datasets, demonstrating superior image realism, stylistic consistency, and cultural motif preservation.

Method	Miao Batik Dataset (FID)	WikiArt Dataset (FID)
Pix2Pix	48.6	52.3
CycleGAN	42.8	46.5
StyleGAN	39.7	43.1
AttentionGAN	36.9	40.4
SegCycle-SPADE (Proposed)	28.5	31.2

Table 4: Frechet Inception Distance (FID) Results for Cultural Pattern Reconstruction. Comparison of Frechet Inception Distance (FID) scores obtained by the proposed SegCycle-SPADE framework and baseline generative models on the Miao Batik and WikiArt datasets. Lower FID values indicate greater similarity between generated and real-image feature distributions and therefore reflect improved visual realism of reconstructed cultural patterns.

Figure 10 compares the reconstruction quality achieved by different methods. Reconstruction Quality (%) was calculated by converting the SSIM between the reconstructed image and the corresponding reference image into a percentage scale (SSIM × 100). Higher percentages indicate greater structural fidelity and visual similarity to the original cultural motif. Conventional generative models such as Pix2Pix and CycleGAN achieved moderate reconstruction performance. More advanced architectures, including StyleGAN and AttentionGAN, achieved improved reconstruction quality due to their enhanced feature-learning capabilities. However, the proposed SegCycle-SPADE framework achieved the highest reconstruction quality because of its integration of semantic segmentation guidance, cross-attention feature fusion, reconstruction learning, and artistic synthesis.

Figure 10. Comparison of Pattern Reconstruction Quality. Comparison of reconstruction quality achieved using Pix2Pix, CycleGAN, StyleGAN, AttentionGAN, and the proposed SegCycle-SPADE framework on the Miao Batik and WikiArt datasets. Results are presented as mean ± standard deviation (SD) from five independent runs (n = 5). Error bars indicate one standard deviation. Higher values indicate improved preservation of structural details, semantic consistency, and visual fidelity. The proposed SegCycle-SPADE framework achieved the highest reconstruction-quality scores on both datasets. Please click here to view a larger version of this figure.

PSNR was used to evaluate reconstruction fidelity by measuring pixel-level similarity between reconstructed images and corresponding reference images. PSNR was calculated between each reconstructed image and its corresponding original Miao Batik image used as the ground-truth reference. Higher PSNR values indicate lower reconstruction error. As shown in Table 5, Pix2Pix and CycleGAN achieved moderate PSNR values because they reconstructed patterns without explicit structural guidance. StyleGAN and AttentionGAN achieved higher PSNR values through deeper feature learning. The proposed SegCycle-SPADE framework achieved the highest PSNR values on both datasets, demonstrating superior reconstruction fidelity and preservation of cultural motif structures.

Method	Miao Batik Dataset (PSNR, dB)	WikiArt Dataset (PSNR, dB)
Pix2Pix	25.8	24.6
CycleGAN	27.3	26.1
StyleGAN	28.7	27.5
AttentionGAN	29.9	28.6
SegCycle-SPADE (Proposed)	32.4	31.2

Table 5: Peak Signal-to-Noise Ratio (PSNR) Results for Cultural Pattern Reconstruction. Comparison of Peak Signal-to-Noise Ratio (PSNR) values obtained by the proposed SegCycle-SPADE framework and baseline methods on the Miao Batik and WikiArt datasets. Higher PSNR values indicate improved reconstruction fidelity and reduced distortion relative to the corresponding reference images.

Figure 11 illustrates the convergence behavior of the proposed SegCycle-SPADE framework during training. The loss curves represent the mean training losses averaged across five independent training runs. To reduce stochastic variability and provide a more robust representation of training convergence, the loss values were averaged at each epoch across all runs. During the initial training epochs, loss values were relatively high because the model was still learning feature representations from the input data. As training progressed, all loss components gradually decreased and stabilized, indicating successful optimization of segmentation, reconstruction, and artistic synthesis objectives. The observed convergence behavior demonstrates the effectiveness, stability, and reproducibility of the proposed framework during training.

Figure 11. Training Convergence of the Proposed SegCycle-SPADE Framework. Training-loss curves of the proposed SegCycle-SPADE framework over 100 training epochs, averaged across five independent training runs (n = 5). The figure illustrates the evolution of the total loss (LTotal), segmentation loss (LSeg), generator loss (LG), and discriminator loss (LD) during training. The gradual reduction and subsequent stabilization of all loss components indicate successful convergence and stable optimization of the proposed framework. Please click here to view a larger version of this figure.

Ablation Study
To evaluate the contribution of each component within the proposed SegCycle-SPADE framework, an ablation study was conducted using multiple controlled model configurations. The results are summarized in Tables 6 and 7.

Model Configuration	Segmentation Accuracy (%)	IoU	SSIM
SegFormer Only	87.3	0.76	0.82
SegFormer + CAFFM	89.6	0.79	0.86
SegFormer + CAFFM + CycleGAN	91.4	0.82	0.89
SegFormer + CAFFM + CycleGAN + SPADE (Proposed)	93.8	0.86	0.93

Table 6: Ablation Study of SegCycle-SPADE Components. Performance comparison of progressively enhanced model configurations to evaluate the contribution of individual framework components. The study examines the effects of semantic segmentation, Cross-Attention Cultural Feature Fusion Module (CAFFM), CycleGAN-based reconstruction, and SPADE-based artistic synthesis on segmentation accuracy, Intersection over Union (IoU), and Structural Similarity Index (SSIM). Higher values indicate improved motif segmentation, reconstruction quality, and structural preservation.

Variant	IoU (%)	Dice (%)	SSIM	PSNR (dB)	FID ↓
SegFormer Only	84.2 ± 0.4	88.5 ± 0.3	0.82 ± 0.01	24.8 ± 0.2	31.8 ± 0.6
SegFormer + CycleGAN	86.7 ± 0.3	90.2 ± 0.2	0.85 ± 0.01	26.3 ± 0.2	27.5 ± 0.5
SegFormer + SPADE	87.0 ± 0.3	90.5 ± 0.2	0.88 ± 0.01	27.8 ± 0.2	23.4 ± 0.4
SegFormer + CAFFM	90.0 ± 0.2	93.1 ± 0.2	0.90 ± 0.01	29.6 ± 0.1	20.3 ± 0.3
SegCycle-SPADE (Full Model)	93.0 ± 0.2	95.4 ± 0.1	0.92 ± 0.01	31.2 ± 0.1	17.6 ± 0.2

Table 7: Component Contribution Analysis of the Proposed SegCycle-SPADE Framework. Performance comparison of individual framework variants used to assess the contribution of CAFFM, CycleGAN, SPADE, and the complete SegCycle-SPADE architecture. Results are reported using Intersection over Union (IoU), Dice coefficient, Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Frechet Inception Distance (FID). Higher IoU, Dice, SSIM, and PSNR values indicate improved segmentation and reconstruction quality, whereas lower FID values indicate greater visual realism of generated cultural patterns.

Table 6 evaluates the cumulative contribution of the major framework components using four configurations: (i) SegFormer only, (ii) SegFormer + CAFFM, (iii) SegFormer + CAFFM + CycleGAN, and (iv) SegFormer + CAFFM + CycleGAN + SPADE (proposed framework). The baseline SegFormer model achieved a segmentation accuracy of 87.3%, an IoU score of 0.76, and an SSIM value of 0.82. Incorporating the CAFFM module improved performance across all evaluation metrics, increasing segmentation accuracy to 89.6%, IoU to 0.79, and SSIM to 0.86. The addition of CycleGAN further enhanced structural reconstruction capability, resulting in an IoU of 0.82 and an SSIM value of 0.89. The complete SegCycle-SPADE framework achieved the best overall performance, with a segmentation accuracy of 93.8%, an IoU of 0.86, and an SSIM value of 0.93. These results demonstrate that each component contributes incrementally to improved segmentation accuracy, structural preservation, and image reconstruction quality.

Table 7 further investigates the contribution of individual framework components using five model variants: (i) SegFormer only, (ii) SegFormer + CycleGAN, (iii) SegFormer + SPADE, (iv) SegFormer + CAFFM, and (v) the full model incorporating SegFormer, CAFFM, CycleGAN, SPADE, and motif-preservation loss. Performance was evaluated using IoU, Dice coefficient, SSIM, PSNR, and FID metrics.

The baseline SegFormer-only configuration achieved an IoU of 84.2%, a Dice coefficient of 88.5%, an SSIM value of 0.82, a PSNR of 24.8 dB, and an FID score of 31.8. Adding CycleGAN improved reconstruction quality and reduced the FID score to 27.5, indicating enhanced image realism. Incorporating SPADE further improved structural similarity and image quality, resulting in an SSIM value of 0.88 and an FID score of 23.4. The inclusion of the proposed CAFFM module produced substantial gains across all metrics, increasing IoU to 90.0%, Dice coefficient to 93.1%, SSIM to 0.90, and PSNR to 29.6 dB while reducing the FID score to 20.3. The complete model, which additionally incorporates the motif-preservation loss, achieved the best overall performance with an IoU of 93.0%, a Dice coefficient of 95.4%, an SSIM value of 0.92, a PSNR of 31.2 dB, and an FID score of 17.6.

Table 8 presents a comparative overview of representative DL approaches used for cultural heritage pattern reconstruction and preservation. Conventional CNN-based methods provide effective local feature learning and segmentation performance but often struggle to preserve complex motif structures due to limited long-range dependency modeling. GAN-based approaches improve image synthesis and style transfer capabilities; however, they may suffer from semantic inconsistencies and loss of fine structural details. Transformer-based methods enhance global contextual understanding but typically lack generative reconstruction capabilities, whereas diffusion-based methods offer high visual realism at the cost of increased computational complexity. Hybrid Transformer-GAN approaches partially address these limitations by combining feature extraction and image generation mechanisms. In contrast, the proposed SegCycle-SPADE framework integrates transformer-based segmentation, cross-attention feature fusion, generative reconstruction, and segmentation-guided artistic synthesis within a unified architecture, thereby improving structural fidelity, semantic consistency, and cultural motif preservation. The comparison is summarized in Table 8.

Approach	Core Methodology	Strengths	Limitations	Relevance to Cultural Heritage
CNN-Based Methods (e.g., U-Net, DeepLabV3+)	Convolutional feature extraction and semantic segmentation	Effective local feature learning and accurate segmentation	Limited long-range dependency modeling; difficulty preserving complex motif structures	Suitable for motif segmentation but less effective for artistic reconstruction
GAN-Based Methods (e.g., Pix2Pix, CycleGAN)	Adversarial image generation and image-to-image translation	High-quality image synthesis and style transfer	Training instability; semantic inconsistency; potential loss of fine structural details	Useful for pattern restoration but may not preserve cultural motifs accurately
Transformer-Based Methods	Self-attention and global context modeling	Captures long-range dependencies and complex visual relationships	High computational cost; requires large training datasets	Effective for complex pattern analysis but limited generative capability when used alone
Diffusion-Based Methods	Iterative denoising-based image generation	High visual realism and image diversity	Slow inference speed; high computational requirements; limited structural control	Promising for heritage image generation but computationally intensive
Hybrid Transformer-GAN Methods	Combination of transformer-based feature extraction and GAN-based generation	Improved global feature representation and image quality	Often lack explicit semantic guidance for motif preservation	Enhances generation quality but is not specifically designed for cultural heritage motifs
Proposed SegCycle-SPADE	SegFormer + CAFFM + CycleGAN + SPADE	Transformer-based segmentation, attention-guided feature fusion, semantic consistency, motif preservation, and controllable artistic reconstruction	Higher computational complexity than standalone CNN-based approaches	Specifically designed for reconstruction and preservation of cultural heritage patterns with improved structural fidelity and semantic consistency

Table 8: Comparison of Representative Deep Learning Approaches for Cultural Heritage Pattern Reconstruction and Preservation. Qualitative comparison of representative deep-learning approaches used for image segmentation, pattern reconstruction, style generation, and cultural heritage preservation. The table summarizes the core methodology, strengths, limitations, and applicability of each approach and highlights how the proposed SegCycle-SPADE framework combines semantic segmentation, feature fusion, reconstruction, and style synthesis to address challenges related to structural preservation and semantic consistency in cultural heritage patterns.

The observed improvements demonstrate that the CAFFM module effectively enhances the interaction between segmentation-derived structural features and artistic style representations through cross-attention-based feature fusion. Furthermore, the motif-preservation loss contributes to maintaining culturally significant motif structures during reconstruction and artistic synthesis, resulting in improved segmentation consistency, structural fidelity, and visual realism. Collectively, the ablation results confirm that the integration of SegFormer-B2, CAFFM, CycleGAN, SPADE, and motif-preservation loss is responsible for the superior performance of the proposed SegCycle-SPADE framework.

Data Availability:
The WikiArt dataset used for artistic style learning is publicly available through the Kaggle WikiArt repository (https://www.kaggle.com/datasets/steubk/wikiart). The Miao Batik dataset used in this study is publicly available via Zenodo (https://doi.org/10.5281/zenodo.20807658). The processed datasets, segmentation masks, pretrained model weights, generated outputs, and Python implementation files associated with the proposed SegCycle-SPADE framework are also available through the same Zenodo repository.

SUPPLEMENTARY FILE:
Supplementary File 1. Algorithm 1: Implementation Workflow of the Proposed SegCycle-SPADE Framework. Step-by-step pseudocode describing the complete implementation pipeline of the proposed SegCycle-SPADE framework, including dataset loading, preprocessing, semantic segmentation using SegFormer-B2, feature fusion using the Cross-Attention Cultural Feature Fusion Module (CAFFM), cultural motif reconstruction using CycleGAN, artistic style synthesis using SPADE, model training, validation, checkpoint selection, and final performance evaluation. Please click here to download this file.

Discussion

The preservation and digital reconstruction of ICH patterns have gained increasing attention in recent years because of the gradual loss of traditional crafts. Previous studies have attempted to address cultural heritage loss through DL-based image processing techniques. For example, Quan et al.³² proposed a cultural heritage preservation framework based on knowledge graphs and DL for Guizhou Miao batik culture. Although the study focused on the digital reconstruction of cultural patterns, its methodology was primarily dependent on knowledge-graph representations. Similarly, Fu and Razmjooy¹⁶ proposed a framework based on the feature pyramid network (FPN) for the enhancement and restoration of cultural heritage images. Although this method provided effective feature extraction for image enhancement, it did not incorporate image segmentation guidance or generative reconstruction mechanisms for incomplete cultural patterns. In contrast, the proposed SegCycle-SPADE framework combines multiple DL components to overcome these challenges. First, semantic segmentation using the SegFormer model is employed for the accurate identification of motif regions within cultural heritage patterns. Subsequently, a CAFFM-based feature enhancement module improves feature representations for fine-grained motif structures. A CycleGAN reconstruction module is then used to reconstruct missing or incomplete regions within cultural patterns through adversarial learning. Finally, a SPADE module performs stylistic enhancement while maintaining structural alignment with segmentation maps. Existing CNN-, GAN-, transformer-, and diffusion-based methods^{17,19,20,21,22,23,24,25,30,34} each offer unique advantages; however, they also present limitations in preserving semantic consistency, maintaining structural features, or achieving computational efficiency, as summarized in Table 8. The proposed SegCycle-SPADE architecture combines the strengths of SegFormer-based segmentation, cross-attention feature fusion, CycleGAN reconstruction, and SPADE-guided synthesis while addressing these limitations. Consequently, the framework achieves improved reconstruction quality and enhanced preservation of culturally significant motifs.

Despite its encouraging performance, the proposed SegCycle-SPADE framework still presents several limitations. First, the assessment was conducted using only the Miao Batik and WikiArt datasets, which may limit the generalizability of the framework to other cultural heritage domains. Second, deployment on resource-constrained platforms may be challenging because the integration of SegFormer, CAFFM, CycleGAN, and SPADE increases computational complexity and memory requirements. Third, segmentation performance depends heavily on the quality and consistency of motif annotations, and annotation bias may affect the preservation of culturally significant structures. Finally, because the framework was evaluated using only two datasets, additional training data and domain-specific adaptations may be necessary to ensure applicability across diverse cultural heritage collections. Future studies should therefore explore more robust training strategies, lightweight architectures, improved annotation procedures, and cross-domain evaluations using additional cultural heritage datasets.

The scalability of the SegCycle-SPADE framework to larger and more diverse cultural heritage datasets will be a major focus of future research. Cross-cultural evaluation of heritage motifs from different regions and artistic traditions will be investigated to improve model generalization and cultural adaptability. Furthermore, multimodal extensions incorporating textual descriptions, historical records, and cultural metadata may provide stronger semantic guidance during reconstruction. The framework may also be extended to support three-dimensional cultural heritage reconstruction by integrating image-based reconstruction with 3D modeling techniques. In addition, deployment in digital archives, museums, virtual exhibitions, and cultural heritage preservation platforms will be explored to facilitate practical applications of AI-assisted heritage conservation and documentation. To further improve interpretability and cultural authenticity, future work will investigate the integration of cultural knowledge graphs, ethnographic documentation, historical metadata, and expert-curated knowledge bases to enable culturally informed motif analysis and reconstruction³².

Several practical considerations should be taken into account when implementing the proposed SegCycle-SPADE framework. During CycleGAN training, unstable adversarial convergence may occur if generator and discriminator losses become highly imbalanced. In such situations, reducing the learning rate, increasing the cycle-consistency loss weight, or monitoring discriminator dominance may improve training stability. Mode collapse may occur when the generator repeatedly produces visually similar outputs; this issue can be mitigated through balanced adversarial training, replay-buffer utilization, and careful monitoring of generator and discriminator loss trends. Segmentation failures may also arise when cultural motifs exhibit complex textures, low contrast, or ambiguous boundaries. To address this issue, image quality should be verified before training, and segmentation masks should be visually inspected to ensure annotation consistency. Maintaining the recommended input resolution and normalization settings is also important for achieving reliable SegFormer performance. Training instability may further result from inappropriate learning-rate settings, insufficient training data, or hardware-related numerical variations. To improve reproducibility, fixed random seeds should be employed for NumPy, PyTorch, CUDA, and dataset-shuffling operations. Moreover, the same preprocessing pipeline, dataset partitions, model hyperparameters, and software environment should be maintained across all experimental runs. Periodic checkpoint saving and validation-loss monitoring are also recommended to prevent performance degradation and facilitate recovery from interrupted training sessions.

The proposed work contributes to the theoretical advancement of AI-based cultural heritage reconstruction frameworks. Conventional image restoration approaches generally treat image segmentation, reconstruction, and style generation as independent processes^17,19,20,30. In contrast, this study proposes a framework that integrates these processes through a segmentation-guided generative reconstruction strategy. Consequently, the study contributes to the theoretical development of AI-assisted cultural heritage reconstruction by demonstrating how segmentation, reconstruction, and style generation can be unified within a single framework. Such integration is particularly important in cultural heritage applications, where preserving motif structure and semantic meaning is critical. From a practical perspective, the proposed framework offers an effective solution for the digital preservation, restoration, and artistic enhancement of traditional cultural patterns. Cultural heritage institutions, museums, and designers may employ SegCycle-SPADE to automatically identify motif regions, reconstruct damaged or incomplete patterns, and generate visually enhanced artistic representations of traditional designs. This capability is especially valuable for endangered craft traditions, including Miao batik textile patterns, where historical artifacts may be partially damaged or incomplete. Furthermore, through the incorporation of generative style synthesis, the framework may support modern cultural design applications such as textile design, digital artistic creation, and heritage education. In this manner, the proposed framework bridges the gap between cultural heritage preservation and contemporary cultural design applications.

Nevertheless, despite the promising results achieved by the proposed SegCycle-SPADE framework, several limitations remain. First, the evaluation was conducted using only the Miao Batik and WikiArt datasets, which may affect the generalization capability of the framework for reconstructing other forms of cultural heritage patterns. Second, because the reconstruction process relies on GAN-based models, generated outputs may occasionally contain artifacts or unnatural textures when dealing with highly complex motif structures. Third, the framework currently focuses on the reconstruction of two-dimensional patterns, whereas some cultural heritage artifacts involve inherently three-dimensional structures. Overall, the experimental findings demonstrate the effectiveness of the proposed SegCycle-SPADE framework for the artistic reconstruction of ICH patterns. The integration of SegFormer-based semantic segmentation, CAFFM, CycleGAN-based motif reconstruction, and SPADE-based artistic synthesis consistently improved segmentation, reconstruction, and image-quality performance metrics. The ablation studies further validated the contribution of each framework component, showing that CAFFM and motif-preservation loss substantially enhanced structural fidelity and visual authenticity. These findings support the central hypothesis that semantic segmentation-guided reconstruction combined with cross-attention feature fusion can effectively preserve culturally significant motifs while generating artistically rich outputs. Therefore, the proposed framework provides a reliable and reproducible computational approach for the digital preservation, restoration, and creative revitalization of ICH patterns.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Adam Optimizer	PyTorch Optimizer Library	Adam	Optimization algorithm used during model training.
CUDA Toolkit	NVIDIA Corporation	CUDA 11.3	GPU acceleration for deep learning computations.
cuDNN	NVIDIA Corporation	cuDNN 8.2	Deep neural network acceleration library used to accelerate training and inference.
CycleGAN	University of California, Berkeley / Open Source	Official PyTorch Implementation (https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix)	Generative adversarial network used for cultural motif reconstruction.
ImageNet Pretrained Weights	ImageNet / Open Source	mit_b2.pth (ImageNet-1K Pretrained Checkpoint)	Used to initialize the SegFormer-B2 backbone before fine-tuning on the Miao Batik dataset.
Intel Processor	Intel Corporation	Intel Core i7 (11th Generation)	CPU used for image preprocessing, model training support, and inference.
Matplotlib	Matplotlib Development Team	Matplotlib 3.5.1	Visualization of training curves and evaluation results.
Miao Batik Dataset	Zenodo Repository	DOI: 10.5281/zenodo.20807658	Cultural motif dataset containing 15,148 images used for semantic segmentation and pattern reconstruction.
NumPy	NumPy Developers	NumPy 1.21.5	Numerical computation and dataset processing.
NVIDIA GPU	NVIDIA Corporation	NVIDIA RTX 3090 (24 GB VRAM)	Hardware platform used for model training and inference.
OpenCV	OpenCV Foundation	OpenCV 4.5.5	Image preprocessing, resizing, interpolation, and Gaussian denoising.
Operating System	Canonical Ltd.	Ubuntu 20.04 LTS	Experimental execution environment.
Pillow (PIL)	Python Imaging Library	Pillow 8.4.0	Image loading and manipulation.
Python	Python Software Foundation	Python 3.8.10	Programming language used for implementation and experimentation.
PyTorch	Meta AI	PyTorch 1.10.0	Deep learning framework used for model training and inference.
Random Seed	Experimental Setting	42	Fixed seed used for dataset partitioning, model initialization, and experimental reproducibility.
Scikit-learn	Scikit-learn Developers	Scikit-learn 1.0.2	Dataset partitioning, statistical analysis, and evaluation metrics.
Seaborn	Open Source Community	Seaborn 0.11.2	Statistical data visualization.
SegFormer-B2	NVIDIA Research / Open Source	SegFormer-B2 (MIT-B2 Backbone)	Transformer-based semantic segmentation model used for cultural motif segmentation.
Segmentation Masks / Annotations	Miao Batik Dataset	Included with Dataset	Pixel-level semantic segmentation labels used for motif classification and training.
SPADE	NVIDIA Research / Open Source	Official PyTorch Implementation (https://github.com/NVlabs/SPADE)	Segmentation-guided image synthesis model used for artistic pattern generation.
TorchVision	Meta AI	TorchVision 0.11.1	Image-processing utilities and dataset transformations used with PyTorch.
WikiArt Dataset	WikiArt	https://www.wikiart.org/	Artistic style dataset containing more than 80,000 artworks across 27 artistic styles, used for style learning and synthesis.