산업 5.0의 지속 가능한 제조를 위한 설명 가능한 AI 지원 사이버-물리 협업

요약

본 연구는 MIMII 데이터셋을 기반으로 산업용 음향 이상 탐지를 위해 CNN-transformer와 SHAP을 이용한 설명 가능한 AI(Explainable AI) 기반의 Industry 5.0 사이버-물리 협업 프레임워크를 제시합니다. 엣지 컴퓨팅 통합을 통해 해석 가능하며 인간 중심적인 예측 보전을 지원합니다.

초록

In this study, we present an innovative approach to the sustainable manufacturing of industrial parts using an explainable artificial intelligence (XAI)- based cyber-physical collaboration system for Industry 5.0. Current cyber-physical human systems (CPHSs) have been found to integrate AI only to a limited extent and often lack explainability. Consequently, there is a need to improve their scalability and flexibility, in keeping with the tenets of Industry 5.0: resilience, long-term viability, and human-centricity. To address these shortcomings, we introduce an architecture that integrates deep learning and explainable AI into the Cyber-Physical System (CPS) ecosystem to enable smart, explainable decision-making. Specifically, we use the Malfunctioning Industrial Machine Investigation and Inspection (MIMII) dataset for machine fault detection from acoustic signals. Audio signals undergo spectral transformation to generate spectrogram images, which are then processed by a convolutional neural network (CNN) for spatial feature extraction and a transformer encoder for temporal features. To improve transparency and facilitate human-in-the-loop cooperation, the Shapley additive explanations (SHAP)-based approach helps operators understand the system's decision-making process by highlighting input features that influence model predictions. In addition, the framework is applied in an edge-computing-based cyber-physical system to achieve low latency and low energy consumption while providing timely responses, thereby helping ensure a sustainable production environment. Experimental findings show that the proposed CNN-Transformer architecture achieves 98.5% accuracy and outperforms existing CPHS systems while maintaining low latency. The use of explainable AI in such a cyber-physical cooperation system greatly increases operator confidence and enables seamless human-machine collaboration.

서론

The transition from Industry 4.0, which emphasized technology, mechanization, and communication, to the new Industry 5.0 model, which combines technological advancement with adaptability, sustainable development, and human-centeredness, has influenced the dynamic evolution of industrial systems^1,2. The European Commission defines Industry 5.0 as a paradigm focused not only on productivity and efficiency but also on the three pillars of human-centricity, sustainability, and resilience. It is important to note that recent research has pointed out the role of AI systems, human-machine cooperation, and sustainable manufacturing as the main drivers of Industry 5.0 environments^3,4. Traditional maintenance methods are no longer sufficient in this evolving industrial environment. To enable resilience in complex, unstable contexts, maintenance must evolve into a more adaptable and systemic form⁵. Since 2011, Industry 4.0 has been the dominant model for industrial transformation in Europe, encouraging the digital transformation of businesses to enhance operations and maximize available resources⁴. Increasing efficiency without eliminating human labor in production, while protecting workers’ health and safety, is therefore imperative. Because of this, businesses and academics have begun defining the tenets of the Industry 5.0 paradigm^5,6,7, partly motivated by the Japanese notion of Society 5.0^8,9,10,11. Despite the increased emphasis on sustainable maintenance in recent literature, most existing systematic evaluations remain fragmented or domain-specific, lacking a comprehensive framework that integrates resilience, sustainability, and AI. For example, the authors examine sustainable maintenance practices for specific systems or strategies, including sustainable total productive maintenance (STPM), whereas the researchers talk about incorporating sustainability themes into maintenance techniques^12,13,14. Author conducted a comprehensive assessment of sustainability adoption criteria and highlighted technological factors that enable manufacturing to have a positive sustainable impact¹⁵. Nevertheless, within the Industry 5.0 paradigm, sustainability, resilience, and AI-based decision-making are not fully integrated in any of these works. As a result, separate maintenance methods frequently address sustainability. SHAP stands out among other explainable artificial intelligence (XAI) techniques due to its theoretically sound feature attribution method^16,17,18.

Data-driven decision-making is enabled by AI, but accurate models often require large amounts of data. Conventional machine learning (ML) techniques, such as decision trees, logistic regression, and linear regression, frequently perform poorly in highly nonlinear situations because they assume relatively simple data distributions¹⁹. By learning intricate feature representations from massive datasets, deep neural networks (DNNs) get around these restrictions²⁰. Deeper networks often perform better than shallow structures on complex decision-making tasks, according to studies²¹, but their increasing depth and parameter complexity make the model less interpretable²².

A Cyber-Physical System (CPS) is a crucial enabling technology for Industry 5.0 manufacturing because it combines distributed intelligence, embedded computers, physical processes, and communication. To enable flexible, intelligent, and real-time industrial operations, modern CPSs are rapidly integrating edge computing, edge AI, and green Internet of Things (G-IoT)^23,24. Sustainability and resilience have been highlighted in recent studies on Maintenance 4.0 and smart maintenance. While earlier works have examined the significance of Maintenance 4.0 technology in sustainable manufacturing and suggested modeling frameworks for sustainable maintenance, bibliometric and systematic studies have assessed smart maintenance technologies^25,26,27,28. Further research has investigated the integration of sustainable maintenance into Industry 4.0 environments and sustainability-oriented production systems^29,30,31,32. The Operator 4.0 concept³³, human-centric Industry 5.0 frameworks³⁴, and socioeconomic perspectives³⁵ have all garnered significant attention to human-centric maintenance. Additionally, advancements in digital maintenance³⁶, predictive and prescriptive maintenance^37,38, and sustainable maintenance strategies^39,40 have been emphasized in recent studies. Despite these advancements, research remains dispersed, and few studies offer a cohesive paradigm that integrates explainable AI, sustainability, resilience, and human-centered cyber-physical cooperation. Cyber-Physical Systems (CPSs) have not attained full autonomy despite their growing deployment, and the Industry 5.0 paradigm does not favor it. Industry 5.0, on the other hand, places greater emphasis on human supervision, cooperation, and decision-making. By incorporating human thinking and cooperation into cyber-physical processes, cyber-human systems such as human-in-the-loop CPSs (HiLCPSs)⁴¹, Human-Centered CPSs (HCPSs)^42,43, and Cyber-physical human systems (CPHSs)⁴⁴extend traditional CPSs. This integration improves adaptability, transparency, safety, and resilience by enabling real-time collaboration between intelligent systems and human operators^45,46.

Industrial audio anomaly detection has been greatly enhanced by new architectures, including Vision Transformer (ViT), Audio Spectrogram Transformer (AST), Conformer, and self-supervised learning; however, their integration with explainable cyber-physical systems for Industry 5.0 remains limited. Without integrating explainability, edge deployment, and human-in-the-loop decision support into a single framework, current research primarily focuses on classification accuracy or on post hoc XAI-based interpretability. This study proposes a novel Explainable AI-enabled CNN-Transformer cyber-physical collaboration architecture that combines edge computing, temporal dependency modeling, SHAP-based explainability, spatial feature learning, and human-centric decision support to enable sustainable Industry 5.0 manufacturing and overcome these constraints.

Thus, there remains a substantial research deficit. Current CNN/Transformer-based acoustic anomaly detection techniques favor classification performance but lack cyber-physical cooperation and explainability. While Industry 5.0 frameworks offer conceptual human-centric models that do not integrate explainable deep learning, edge intelligence, and operator-assisted decision-making into a single system, XAI-based predictive maintenance techniques increase transparency but operate independently of CPHS architectures. Therefore, a single Industry 5.0 Cyber-physical human system that integrates precise auditory anomaly detection, SHAP-based explainability, edge-enabled deployment, and human-in-the-loop cooperation is still lacking.

This paper proposes a novel Explainable AI-enabled CNN-Transformer cyber-physical collaboration paradigm for Industry 5.0 to overcome these constraints. The suggested architecture combines edge computing deployment, CNN-based spatial feature extraction, Transformer-based temporal dependency learning, SHAP-based explainability, and human-in-the-loop decision support into a single CPHS framework. The suggested framework offers an integrated solution that concurrently enhances anomaly detection performance, model transparency, operational efficiency, and human-centric collaboration for sustainable manufacturing, in contrast to current methods that handle these elements independently.

This study's main goal is to create an Explainable AI-enabled Cyber-physical human system (CPHS) for sustainable Industry 5.0 manufacturing by combining SHAP-based explainability, edge computing, and human-in-the-loop decision support with a hybrid CNN-Transformer-based acoustic anomaly detection approach. In particular, the suggested framework aims to improve anomaly detection accuracy while providing clear, understandable forecasts, facilitating effective edge deployment, and encouraging cooperative decision-making between intelligent systems and human operators. By concurrently achieving high detection performance, explainability, low-latency operation, and human-centric cyber-physical collaboration, this integrated architecture addresses the shortcomings of current acoustic anomaly detection, XAI-based predictive maintenance, and CPHS frameworks.

The suggested architecture presents a unified Explainable AI-enabled Cyber-physical human system (CPHS) for Industry 5.0, in contrast to current acoustic anomaly detection techniques, which primarily focus on classification accuracy. Within a single architecture, it combines CNN-based spatial feature extraction, Transformer-based temporal modeling, SHAP-based explainability, edge computing, and human-in-the-loop decision support. The suggested architecture addresses key issues of transparency, operator assistance, and cyber-physical cooperation by fusing precise anomaly detection with comprehensible decision-making and real-time edge deployment. The main scientific contribution of this work is the system-level integration, which sets the suggested approach apart from previous research that focuses on anomaly detection, explainable AI, or Industry 5.0 frameworks alone.

프로토콜

The suggested methodology uses a system-based Industry 5.0 Cyber-physical human system (CPHS) approach that combines explainable deep learning with a cyber-physical collaboration framework enabled by edge computing to support sustainable manufacturing processes. First, the problem is addressed by recognizing certain limitations of current CPHS solutions, such as a lack of AI-powered intelligence, explainability, and human-centric collaboration. To address the challenges, the proposed solution leverages the MIMII dataset for industrial machine condition monitoring using acoustic signal processing.

The proposed framework begins by feeding the generated spectrograms to a CNN model that extracts spatial features by recognizing significant frequency patterns associated with different machine conditions. The extracted spatial features will be converted into sequences and fed into a transformer encoder module to capture temporal dependencies via self-attention. The transformed feature representation is subsequently passed to a fully connected classifier module to classify machine states as normal or abnormal. In addition, to ensure transparency and facilitate human-centric decision-making, the SHAP explainability approach is employed to explain predictions by selecting significant time-frequency regions in the model output. The designed model will be implemented in an edge-enabled CPS setting, enabling real-time inference with low computational latency and energy expenditure. In addition, a human-in-the-loop scheme will be integrated to enable human operators to validate using explainability results. Performance is validated using accuracy, precision, recall, and F1-score, as well as other system-level metrics such as latency and energy efficiency.

The proposed framework is an XAI-driven CPHS mechanism for sustainable manufacturing in the context of Industry 5.0, and its workflow is presented as a pipeline from data acquisition to intelligent decision-making and execution, as shown in Figure 1. In the initial stage of the data acquisition procedure, sound signals are collected from machines in industries such as motors, pumps, and bearings using datasets including MIMII. Data signals thus obtained are forwarded to the next stage, referred to as data preprocessing, where they undergo filtering, normalization, and segmentation. The segmented data signals are finally converted to a spectrogram using the short-time Fourier transform (STFT) at the feature representation layer.

Then, the AI model layer uses the spectrogram images to extract features via the CNN module, identifying spatial patterns and anomalies. Then, the extracted features are reshaped and fed into the transformer encoder, which learns temporal dependencies and long-range relationships via self-attention mechanisms. The resulting feature vectors are classified as normal or abnormal using the classification layer. To ensure transparency in the prediction process, the explainability layer uses SHAP to highlight significant time-frequency regions in the spectrogram.

Additionally, there is a human-in-the-loop process in which operators interpret and provide feedback on the model's outcomes. In the deployment layer, the model runs on an edge-based CPHS infrastructure, ensuring fast inference and low power consumption, while optionally interfacing with cloud systems for storage requirements or advanced analytics. Altogether, the framework ensures accurate fault detection, explainability, and efficient real-time operation in line with the requirements of Industry 5.0.

Problem definition and requirement analysis

Cyber-physical human systems (CPHS) in Industry 5.0 settings promise intelligence, sustainability, and human-centeredness. Still, contemporary frameworks for building CPHS are hindered by a lack of advanced AI-enabled capabilities, explainability, and adequate support for human-machine collaboration. Indeed, conventional CPHS are based on rule-based or uninterpretable models, which impede adaptation to changing industry conditions and limit the operators' confidence in automation. Moreover, conventional CPHS cannot effectively analyze highly sophisticated data, such as industrial acoustic signals containing both spatial (i.e., frequency) and time-dependent components, which are important for predictive maintenance. Hence, it is necessary to create a framework that can detect machine anomalies with high precision, provide explanations, and operate at the edge, thereby enabling CPS.

Here x(t) is the acoustic input signal, S(f,t) is the spectrogram representation of the signal,  is the label and θ is the parameter set of the CNN-Transformer architecture. Symbols used in subsequent equations are introduced right after their first appearance in order to preserve mathematical correctness.

From a mathematical point of view, the task can be formalized as a supervised learning problem. Let the industrial acoustic signal be denoted by x(t). The STFT of x(t) is computed according to the formula:

   (1)

where ω(.) is the window function and f is the frequency. The generated spectrogram  serves as the input to the deep learning algorithm. The goal is to train the mapping function

    (2)

where  is the class label (either normal or abnormal), and where S is the spectrogram input, y is the predicted output label and θ is the set of learnable parameters of the CNN-Transformer architecture. The optimization problem is formulated as the minimization of the classification loss:

  (3)

subject to the constraints of minimum latency , energy efficiency , and interpretability, wherein the explanation ∅(SHAP) captures the importance of features in the prediction process,

   (4)

where  is the contribution of each individual feature. In line with the foregoing, the design requirements include the following: (i) reliable anomaly detection based on hybrid deep learning, (ii) interpretable outputs for human intervention, and (iii) implementation in an edge-enhanced CPS framework.

Dataset acquisition and preprocessing

In this regard, to assess the proposed Explainable AI-based CPHS framework, the MIMII Dataset¹³ has been selected. This dataset is a widely used benchmark for detecting anomalies in industrial machinery using acoustic signals. This database contains sound recordings of various machines, such as slide rails, fans, pumps, and valves, under both normal and faulty operating conditions. For each machine, recordings vary with different settings and noise environments. This makes it suitable for training and evaluating models under real-time industrial conditions.

Whereas the MIMII dataset is widely used in unsupervised anomaly detection studies that rely solely on normal samples during training, it includes explicit annotations of both normal and anomalous machine states. The problem is posed as a supervised binary classification task to test the ability of the proposed Explainable AI-based CNN-Transformer architecture to separate healthy from faulty machine states. Specifically, the official MIMII labels are used, assigning class label 0 to normal samples and class label 1 to anomalous samples.

Table 1 shows the Dataset Description Details. Approximately 12,000 recordings of acoustic signals are considered in the current research, including both normal and faulty machinery operations. As per the requirements, all recordings in this study have been captured at 16 kHz and are 10 s long. Although the dataset contains approximately 12,000 recordings, a class-balanced subset of 2,000 test recordings was selected solely to visualize the confusion matrix. All reported performance metrics were computed using the complete test set (approximately 2,400 recordings). Furthermore, training and test samples are separated at an 80:20 ratio for model training. This database is well-suited for Industry 5.0 applications because it includes real-time noise conditions. The recording distribution used in this study is given in Table 2. From this dataset, approximately 12,000 recordings covering all machine types and signal-to-noise ratio (SNR) conditions were selected for the supervised binary classification experiments using an 80:20 train/test split, shows the industrial machine types, corresponding machine IDs, total numbers of normal and anomalous acoustic recordings, SNR conditions, and the experimental train/test partition employed for supervised binary classification.

The experiment was carried out using the MIMII (Malfunctioning Industrial Machine Investigation and Inspection) dataset version 1.0, which comprises four types of industrial machines: fans, pumps, valves, and slide rails. Data were collected from different machine IDs (ID00–ID06, depending on the machine type) and conditions. The MIMII database comprises data acquired at signal-to-noise ratios (SNRs) of −6 dB, 0 dB, and +6 dB. Normal data corresponds to the healthy condition of the industrial machine, while anomalous data corresponds to an anomalous condition. The classes were labeled based on the database annotation, with normal data labeled class 0 and anomalous data labeled class 1.

A fair evaluation procedure required that the split between train and test sets occur at the original audio-recording level prior to any splitting, augmentation, or spectrogram creation steps. As a result, audio segments from the same recording could not appear in both the train and test sets. For the evaluation experiment, recordings with different signal-to-noise ratios (−6 dB, 0 dB, and +6 dB) were used, which provided an opportunity to evaluate the suggested framework under various industrial conditions.

To prevent data leakage, the train/test split was performed using the original audio file as the basis, prior to any segmentation or spectrogram generation. Segments from the same 10 s audio recording were assigned exclusively to either the training or the testing set, with no segments present in both sets. Afterward, spectrograms were generated and augmented separately for each dataset. This way, the problem of data leakage is avoided, thereby preventing an overly optimistic assessment of performance.

Dataset preprocessing

The unprocessed acoustic signals from the MIMII dataset are then processed to improve signal quality and enable effective feature extraction. The input signal is denoted by x(t), where t is time. The first process is noise filtering, whereby background noises are filtered using a filtering function such that:

   (5)

where ∗ represents the convolution process and h(t) is the filter's impulse response. This process enhances the clarity of the acoustic signal and minimizes the effects of noise often prevalent in industrial environments. Signal normalization is used to ensure that the amplitudes of all audio signals are comparable before feeding them into the deep learning model. The normalized signal, denoted as x_n(t), is calculated as:

   (6)

where μ and σ are the mean value and standard deviation of the input signal, respectively. All input signals are guaranteed to have zero means and unit variances thanks to this procedure. Finally, for processing, the normalized input signal is divided into multiple tiny pieces. Given that the length of the entire signal is T, it is broken down into N overlapping frames, each having a length L using a sliding window function:

   (7)

where H is the hop size. The spectrogram can be defined by the formula below:

   (8)

The spectrograms obtained in this way are fed into the CNN-transformer-based framework to extract spatial and temporal features for successful anomaly detection. Figure 2 shows the preprocessing pipeline.

Figure 2 illustrates a vertical preprocessing pipeline that may be employed for acoustic data acquired in industry. Firstly, the raw audio signals are extracted from the MIMII dataset and fed into the noise-removal stage to reduce noise. This is followed by normalizing the signal to remove amplitude inconsistencies, ensuring the signal is consistent. The signal is later split into smaller overlapping sections. Following segmentation, STFT transforms will be applied to the data to convert time-domain signals into time-frequency signals. Then, the power spectrum is calculated as part of generating the signal's spectrogram. Preprocessed spectrograms will thus be the output generated in the process and will be used in the proposed CNN-transformer model.

Spectrogram generation

After data normalization and segmentation, acoustic signals are converted into a time-frequency representation using the Short-Time Fourier Transform (STFT). The resulting spectrogram provides a pictorial representation of changes in frequency composition over the period of analysis. In industrial settings, machine faults usually manifest as frequency patterns, making spectrograms more useful for detecting machine anomalies.

The resulting processed data is therefore converted into spectrogram images to distinguish between normal and abnormal instances within the machines. The audio data will be converted into multiple spectrograms to enable the learning model to analyze local frequency components and temporal changes within the machine process. These images are used as input data for the CNN-transformer model, where the CNN handles spatial (frequency) patterns while Transformers handle temporal relationships.

Algorithm 1: Spectrogram Generation from Acoustic Signals

Input: Raw audio signal x(t)

Output: Spectrogram S(t, f)

Step 1: Load audio signal  x(t)

Step 2: Apply noise filtering

Step 3: Normalize signal x_n(t)

Step 4: Segment signal into frames:

for k = 1 to N do

end for

Step 5: Apply a window function w(t) to each segment

Step 6: Compute STFT:

  (9)

Step 7: Compute magnitude:

   (10)

Step 8: Store spectrogram S(t, f)

Step 9: Return spectrogram dataset 

The process of generating spectrograms starts with acquiring the original audio signal, followed by preprocessing steps such as noise reduction and signal normalization to ensure signal integrity and quality. After that, the normalized audio is sliced into multiple overlapping windows to capture the temporal characteristics of both long- and short-term periods. To prevent spectral leakage, each of the slices is multiplied by the window function before being passed through the Short-Time Fourier Transform. Afterward, the transformed signal is represented as a complex-valued function in the frequency domain, and its magnitudes are squared. Thus, the energy of the frequencies over the given time is represented by the spectrograms. Finally, they are stored as image-like data and fed to the neural network for training.

Spatial feature extraction using CNN

After the process of creating spectrograms, the output time-frequency images are taken as inputs and are given to a Convolutional Neural Network (CNN). Assuming the input spectrogram as , where H is the frequency domain and W is the time domain. The purpose of using the CNN here is to learn discriminative spatial features from the input spectrogram. These discriminative spatial features will help differentiate between normal and abnormal machine behavior in an industrial environment.

A basic operation performed by the Convolutional Neural Network (CNN) is convolution. Convolution with a given filter  is given as:

   (11)

The operation described above enables the neural network to identify local spatial features, such as edges, frequencies, and harmonic content, in the spectrogram. This is done through multiple filter operations that generate various feature representations, producing several feature maps. Convolution is followed by a non-linear activation function, as shown below:

  (12)

This non-linearity enables the network to capture and represent complex relationships among variables. The next stage in a convolutional neural network involves the application of pooling operations. The common technique of max pooling is expressed mathematically as follows:

   (13)

where Ω refers to the pooling window. Pooling helps to achieve translation invariance and retain only the relevant information while discarding redundancy. From low-level variables such as frequencies to complex machine behavior, higher-level features can be extracted hierarchically using a convolutional neural network. Consequently, feature maps can be used to depict the network's ultimate output:

   (14)

where represents the learnable parameters of the CNN. The feature maps contain rich spatial information and are transformed into sequences to serve as inputs to the transformer encoder in the next phase. The above-described spatial feature extraction process is crucial for enabling the model to detect minute anomalies in industrial acoustic signals.

Figure 3 demonstrates how spatial features are extracted horizontally from the spectrogram input using a CNN model. It starts with the input spectrogram, which encodes time-frequency properties of the audio signal in a pictorial form. The input spectrogram is then passed through a convolutional layer, in which several types of filters extract spatial features such as frequency bands, edges, and textures, thereby producing feature maps. Thereafter, the extracted feature maps undergo a nonlinear activation function, rectified linear unit (ReLU), allowing the neural network to detect dependencies in the input data. The feature maps are then passed through a pooling layer, where the map size is reduced while retaining its most important characteristics. This process of convolutional, activation, and pooling layers repeats itself multiple times, creating a hierarchy of feature representation that begins with simple features and evolves toward more sophisticated ones related to machine operations.

Temporal modeling using a transformer encoder

Once the spatial features have been extracted using the CNN, the next step is to derive high-level feature maps that capture frequency-based patterns learned from the spectrogram. The resulting feature maps are then flattened and reshaped into sequences to capture temporal information in the acoustic signal. Assuming the feature maps are represented by , where H', W', and C are height, width, and channels, respectively, the following sequence is derived:

    (15)

where T represents the sequence length, and d is the feature dimension. The Transformer then uses the above sequence to model long-term dependencies using a self-attention mechanism. The transformer model, in contrast to conventional recurrent models, processes each element of the sequence in parallel and uses attention weights to determine each piece's relative relevance. The way the attention mechanism works is by using Query Q, Key K, and Value V vectors generated for each input x_t according to:

   (16)

whereas W_Q, W_K, W_V are trainable weights. The similarities among the elements are determined through:

   (17)

d_k is important vectors' dimension. This procedure makes it easier for the model to learn dependencies from distant time steps in the signal and focus just on the pertinent time segments. To increase the network's capacity for learning, multihead attention is used, which refers to the process whereby several heads of attention learn from the input in parallel:

  (18)

Each head attends to distinct features of the temporal input sequence. This is followed by feed-forward neural networks and residual connections:

   (19)

The resulting encoded sequence captures both the spatial and temporal features of the input. This sequence is sent to the classifier layer to detect anomalies.

Figure 4 shows the overall process of Temporal modeling using a transformer Encoder on features extracted from the CNN. First, the feature maps extracted from the CNN are used; they contain spatial information from the spectrogram. The feature maps are flattened into a sequence, with each item corresponding to a timestamp. After that, the sequence passes through a linear transformation that maps the data into vectors of a specific dimension. As transformer models lack the ability to capture temporal dependencies, positional encoding is also added to the embedding vectors to preserve the sequence's temporal structure. Finally, the sequence passes through a transformer encoder, consisting of a stack of multiple layers, in which a multi-head self-attention mechanism is employed to enable the model to learn connections between timestamps in the sequence. The feed-forward neural network further improves these embeddings, while residual connections ensure that gradients propagate efficiently. The process is repeated across the stacked transformer encoder layers, which capture progressively more complex temporal interactions. In the end, the sequence embeddings are combined into a fixed-length representation via pooling (e.g., average pooling or the classification token (CLS)), yielding a global temporal feature representation.

Classification layer

Following temporal encoding with the transformer encoder, the outputs will be a sequence of encoded vectors, each with features related to both the spatial and temporal aspects of the input signal. The first step toward classification from the encoded outputs is to aggregate all those features into a single vector. This is achieved either by applying mean pooling to the sequence of feature vectors or by employing the so-called "classification token". Suppose the output of the Transformer is , where each . The aggregated feature h is computed as:

   (20)

Then, the aggregated feature vector h represents the input in a compact form and serves as the classifier's input. The classification part comprises one or more dense layers, which transform the learned representation into the output space. A dense layer implements a transformation that is defined as follows:

   (21)

where o is the output (logits), b is the bias vector, and W is the weight matrix. The latter are the class scores in an unnormalized form. To incorporate nonlinearity into the model, activation functions such as ReLU are often used. Finally, a softmax activation function is applied to the logits to obtain a probability for each class:

   (22)

where C represents the number of classes, (C = 2; normal and abnormal). The class prediction will be based on the maximum probability value. During training, the optimal values of the model’s parameters are obtained by minimizing the cross-entropy loss between the actual and predicted labels, thereby achieving accurate classification of machine states. In summary, the classification layer serves an essential function in transforming the extracted temporal features using the Transformer architecture into practical decisions. Through proper discrimination between healthy and faulty machine states, anomaly detection and intelligent decision-making become possible within the Industry 5.0 paradigm.

Figure 5 shows how the classification layer works, which comes after the transformer Encoder. The process starts with the encoded sequence output, in which each vector contains temporal information learned from the input. These vectors are combined into a single vector, known as the global feature vector. The next step is to pass the global feature vector through several dense neural network layers. In each dense layer, matrix multiplication operations are performed to create linear combinations of the inputs. After that, a non-linear activation function, such as ReLU, can be applied to increase the flexibility in learning decision boundaries. At the end of the last dense layer, there are logit scores for each class. These logit scores can be used on the Softmax function to calculate probability scores for each class. These probability scores are used to determine whether the input belongs to Class 0 (Normal) or Class 1 (Abnormal).

Explainability using SHAP

After the model classifies input signals, predictions are generated based on whether the machine operates normally or abnormally. Nonetheless, within the Industry 5.0 environment, it is not enough to produce predictions; the system must also provide transparency and explainability to support human decision-making. The following framework uses SHAP, a game-theory-based model that quantifies the importance of each feature in determining the prediction. In doing so, SHAP assigns weights to features, allowing one to evaluate the relative importance of different sections of the spectrogram input to the overall prediction.

The mathematical formulation of SHAP is based on a decomposition of the model f(x):

   (23)

where ∅₀ is the baseline value (mean model output), ∅_i is the effect of the i-th feature, and M is the number of input features. For the purpose of this study, the features are the time-frequency bins from the spectrogram. SHAP values are positive if they contribute to the probability of the abnormal state and negative otherwise. This formulation guarantees a unique, locally consistent explanation for each prediction. The SHAP method can be applied to a trained CNN-transformer model to provide an explanation based on feature importance. The visual representation will show the most important time-frequency bins in the spectrogram that affect the decision-making process. For instance, abnormal behavior of the equipment may result in certain high-energy frequency bands that the SHAP framework identifies as important contributors to the decision process.

In addition, SHAP can foster cooperation between people and machines in a cyber-physical setting. This is because the predictions can be explained using intuitive visuals, allowing the operators and decision-making systems to verify predictions, analyze the problem, and take appropriate steps. This not only increases trust but also makes AI-based manufacturing processes reliable and accountable. By integrating deep learning algorithms that meet current performance standards with SHAP, which provides explainability, the proposed architecture fulfills the requirements of Industry 5.0.

Algorithm 2: SHAP-based Explainability for Model Prediction

Input: Trained model f(x), input spectrogram S

Output: SHAP values  and explanation map

Step 1: Input spectrogram S into a trained model

Step 2: Compute prediction: y = f(S)

Step 3: Initialize a deep learning-oriented SHAP explainer

Step 4: Compute SHAP values:

Step 5: For each feature i in S:

Calculate contribution 

Step 6: Generate explanation map:

Highlight regions with high 

Step 7: Visualize feature importance (heatmap)

Step 8: Return SHAP values and explanation map

The SHAP-based explainability algorithm starts by obtaining a preprocessed spectrogram image as input and feeding it into the trained CNN-transformer architecture to make predictions (e.g., normal or abnormal machine condition). After making predictions using the trained architecture, the SHAP method is used to interpret the model. In particular, the SHAP values (ϕ) are computed for each input feature to find out which time-frequency components of the input contribute most to the prediction. To calculate these values, the model outputs for inputs with and without certain features must be compared.

After calculating the SHAP values for each feature, its influence on the prediction can be evaluated by the sign and magnitude of the SHAP value. For example, features with higher SHAP values are more likely to influence the outcome, and a positive or negative sign indicates that the feature pushes the prediction toward the normal/abnormal class, respectively. Based on the results, an explanation map can be made in the form of a heatmap.

Lastly, the algorithm outputs both the SHAP values and the visualizations, providing insight into how humans can analyze the model's decision. The technique makes the process more transparent and allows humans to engage in decision-making, thus ensuring that both the algorithm's predictions and its reliability are achieved in the AI-driven industry. Representative test samples were selected from accurately diagnosed anomalous instances across various machine types to improve the reliability of the explainability analysis. While global SHAP feature-importance analysis was performed by aggregating SHAP values across the test dataset to identify consistently influential time-frequency locations, local SHAP explanations were generated for these representative cases. Without requiring expert annotation, this integrated local and global analysis offers a reliable interpretation of the model predictions.

Edge-enabled CPS Integration

This proposed solution is used within the edge-enabled CPS, which aims to achieve efficiency, sustainability, and real-time industrial operations. Once the model has been trained, the CNN-transformer will be deployed via an edge computing approach. The edge computing layer in question is near the industrial Internet of Things (IIoT) devices, including the sensors and machinery. Rather than transmitting all collected data to the cloud, the edge device performs inference on the incoming acoustic signal without first transferring it there. Therefore, the system can detect machinery abnormalities almost instantly and take the necessary steps before the situation deteriorates further. Another advantage of this edge-enabled approach is improved latency and efficiency since data is analyzed near the point of generation. In other words, unlike in the cloud approach, where inference time T_inf

   (24)

The latency, in general, will be considerably reduced through a decrease in T_{communication}. Another benefit is reduced bandwidth consumption and power expended when transmitting data continuously. This system is therefore scalable and sustainable for large industrial plants with numerous connected devices. In addition, the CPS enables interaction with the physical layer (machines), the AI (computational layer), and the human layer.

Human-in-the-loop collaboration

A key component of the suggested Industry 5.0 CPS, which includes human operators actively involved in the decision-making process alongside AI models, is human-in-the-loop collaboration. Namely, according to the designed framework, the system does not become an autonomous black box but instead allows operators to interact with it and observe explainable prediction results in the form of SHAP explanations. The predictions in question refer to the outcomes (normal/abnormal), accompanied by visualizations of parts of the spectrograms used to make them.

Such an interaction helps operators better understand exactly how the prediction is formed and which factors contribute to it. Human involvement implies verifying the system's results and allowing operators to override them when necessary, based on their professional judgment. Collaboration with human operators makes the system more trustworthy, efficient, safe, and reliable, helping prevent errors.

결과

제시된 설명 가능한 AI(Explainable AI) 통합 사이버-물리 협업 시스템은 정상 및 비정상 기계 상태가 모두 포함된 MIMII 데이터셋의 약 12,000개 오디오 녹음 파일을 통해 실험적으로 테스트되었습니다. 훈련 데이터셋은 약 9,600개의 데이터 레코드로 구성되었으며, 테스트 데이터셋에는 약 2,400개의 오디오 샘플이 포함되었습니다. 연구 결과에 따르면, 제안된 CNN-transformer 모델은 테스트 데이터셋에서 98.5%의 분류 정확도를 달성하였으며(0.97–0.98), 이는 Table 3의 결과, 혼동 행렬 및 통계적 안정성 분석과 연관됩니다. transformer 인코더를 통합함으로써 아키텍처의 시계열 모델링 능력이 향상되었으며, CNN은 입력 스펙트로그램에서 판별 가능한 공간적 특징을 추출합니다.

분류 성능 외에도 다른 시스템 관련 특성들을 평가했습니다. 첫째, 엣지 기반 구현의 추론 지연 시간은 20 ms를 초과하지 않아...

토론

The experimental results demonstrate that the proposed CNN-transformer architecture outperforms individual algorithms by effectively integrating spatial and temporal feature learning. According to the ablation study, although the CNN extracts frequency-based discriminative features from spectrograms, the transformer encoder improves model performance by capturing long-range temporal dependencies in acoustic signals. Moreover, the confusion matrix illustrates very few false negatives, implying that the proposed model can ...

재료

이 논문에 사용된 재료 목록

이름	회사	카탈로그 번호	댓글
사회적 지지 척도	출판된 학술적 척도; Park이 최초로 개발하고 Kim이 수정 및 보완함	25문항 버전; 5점 리커트 척도 응답 형식; 상업적 카탈로그 번호 없음	예비 유아교사의 정서적 지지, 정보적 지지, 물질적 지지 및 평가적 지지에 걸친 인지된 사회적 지지를 측정합니다. 전체 Cronbach’s alpha = 0.96.
교사 자기효능감 척도	출판된 학술적 척도; Enochs와 Riggs에 의해 최초 개발됨; 중국어 적응 연구 버전	25문항 버전; 5점 리커트 척도 응답 형식; 상용 카탈로그 번호 없음	중국 예비 유아교육 교사 환경에서 일반 교사 자기효능감과 개인 교사 자기효능감 전반에 걸친 교사 효능감을 평가합니다. 전체 Cronbach’s alpha = 0.96.
교사-아동 상호작용 척도	출판된 학술적 척도; Lee가 개발하고 전문가 검토 패널이 중국 내 사용을 위해 검토함	30문항 버전; 5점 리커트 응답 형식; 상업용 카탈로그 번호 없음	관찰된 교실 상호작용의 질보다는 교사가 스스로 보고한 교사-아동 상호작용 역량을 평가합니다. 정서적 상호작용, 언어적 상호작용 및 행동적 상호작용을 포함하며, 전체 Cronbach’$\alpha = 0.94$.
인구통계학적 정보 양식	연구팀, 사천예술과학대학교 / 대구대학교 협력	맞춤형 연구 설문지 섹션; 상용 카탈로그 번호 없음	성별, 교육 배경, 전문직 범주, 인턴십 장소, 인턴십 기간, 인턴십 유형 및 유치원 유형을 포함한 참여자 특성을 수집하였다.
참가자 정보 및 고지된 동의 스크립트	연구팀; 대구대학교 연구윤리심의위원회 승인 완료	윤리 승인 번호: XXX; 확인 가능한 현지 승인 기록 보유	설문지 배포 전, 예비 유아교사 참여자들에게 연구 목적과 설문 내용을 설명하고 고지된 동의를 얻기 위해 사용되었습니다.
파일럿 설문조사 설문지 패키지	연구팀 및 전문가 패널	6월 1일에 사용된 파일럿 버전–5, 2024; 상용 카탈로그 번호 없음	예비 유아교육 교사 50명을 대상으로 전문적 인식 척도, 사회적 지지 척도, 교사 자기효능감 척도, 교사-아동 상호작용 척도 및 인구통계학적 항목의 명확성과 적절성에 대해 사전 검사를 실시하였다.
전문가 패널 검토 절차	연구팀; 유아교육과 교수 2인, 유치원 원장 1인, 실습 지도교사 3인 및 연구팀으로 구성된 전문가 패널	합의 기반 콘텐츠 검토; 상업적 모델 번호 없음	정식 설문 조사 전, 척도 항목의 모호성, 적절성, 의미적 명확성, 발달적 적합성 및 문화적 관련성을 수정하는 데 사용됨.
현장 출력 설문지 양식	연구팀; 현지 기관 설문 조사 용품	종이 설문지; 상용 카탈로그 번호 없음	현장 설문조사 실시를 위해 사용되었습니다. 최종 표본에서, 온라인 설문조사와 결합하기 전 336부의 유효한 현장 설문지를 확보하였습니다.
위챗(WeChat)	텐센트	WeChat 모바일 애플리케이션; 버전은 6월 동안 사용된 기기/앱 기록을 통해 확인해야 함–2024년 7월	설문 조사를 위한 온라인 배포 채널로 사용되었으며, 온라인/WeChat 경로를 통해 총 27부의 완료된 온라인 설문지가 수집되었습니다.
전화 및 대면 모집 절차	연구팀	표준화된 모집 절차; 상용 카탈로그 번호 없음	기관 및 참여자에게 연락하여 연구 목적과 설문 내용을 설명하고, 설문지 작성 전 고지된 동의(informed consent)를 지원하는 데 사용됩니다.
SPSS 통계 분석 소프트웨어	IBM	버전 27.0	기술 통계, 피어슨 상관 계수, 크론바흐(Cronbach) 분석에 사용됨’Cronbach's alpha 신뢰도 분석, 왜도와 첨도를 이용한 정규성 검정, 그리고 Harman의 단일 요인 검정’공통 방법 편향에 대한 단일 요인 검사.
SPSS Amos	IBM	버전 26.0	확인적 요인 분석, 구조 방정식 모델링, 다집단 측정 동일성 검정, 간접 효과의 부트스트랩 검정 및 대안 모델 비교에 사용됩니다.
Microsoft Excel	Microsoft	Microsoft 365 Excel 또는 로컬 설치 버전 Excel; 제출 전 정확한 로컬 빌드 버전 확인 요망	코딩된 설문 데이터, 데이터 사전, 보충 표 및 JoVE 재료 표를 정리하기 위한 워크북 환경으로 권장/사용됩니다. 정확한 로컬 버전은 저자가 확인해야 합니다.
구조방정식 모델 적합도 지수 기준	출판된 방법론적 지침; 원고에 인용된 Schreiber et al. 및 Kline 문헌	상업적 카탈로그 번호 없음; 통계 분석 계획에 적용된 기준	카이제곱 통계량, 자유도 대비 카이제곱 값, Tucker-Lewis 지수, 비교 적합도 지수, 수정 적합도 지수, 증분 적합도 지수, 근사 오차 평균 제곱근 및 표준화 잔차 평균 제곱근을 포함한 모델 적합도 지수를 정의하고 보고합니다.
부트스트랩 매개 효과 검증 절차	SPSS Amos / 연구팀 통계 분석 계획	5,000개의 부트스트랩 샘플; 95% 편향 수정 신뢰 구간	교사 효능감을 통한 간접 연관성을 테스트하는 데 사용되었다. 95% 편향 수정 신뢰구간에 0이 포함되지 않을 때 간접 효과가 유의미한 것으로 간주하였다.
측정 동일성 검증 절차	SPSS Amos / 연구팀 통계 분석 계획	다집단 확인적 요인 분석; ΔCFI &< .010 및 Δ근사 오차 평균 제곱근 (RMSEA) &< .015 기준	통합 구조방정식 모델링(SEM) 추정치를 해석하기 전, 성별, 프로그램 유형, 인턴십 기간 및 기관에 따른 형태, 측정 및 스칼라 불변성을 평가하는 데 사용되었다.