Real-time Production Line Safety Monitoring Using Deep Learning-based Object Detection and Feature Enhancement

Summary

This paper proposes a production line safety monitoring method integrating You Only Look Once version 11 (YOLOv11) with convolutional neural networks. It achieved mean average precision at 0.5 intersection-over-union threshold (mAP@0.5) of 0.91, mAP@0.5:0.95 of 0.82, and 120 frames per second on a graphics processing unit, outperforming evaluated baseline models.

Abstract

With the deepening of Industry 4.0, the automation and intelligence levels of production lines have significantly improved, placing higher demands on the real-time performance, accuracy, and safety of monitoring. Traditional monitoring systems, relying on manual inspections or simple threshold-based decisions, generally suffer from slow response times, high false alarm rates, and limited intelligence. Therefore, this paper proposes a production-line safety monitoring system that integrates You Only Look Once version 11 (YOLOv11) with a convolutional neural network (CNN). First, the acquired images were preprocessed. Then, YOLOv11 was used to identify workers, equipment, and potential hazards in real time. Next, an enhanced CNN network with multi-scale feature fusion and attention mechanisms was introduced to improve the feature extraction capabilities for small and occluded targets. Finally, the detection results were fused with the CNN-enhanced features to assess safety status. Experiments were conducted using a self-built production-line safety dataset, employing the stochastic gradient descent (SGD) optimizer with momentum 0.9, an initial learning rate of 0.01, weight decay of 0.0005, cosine-annealed learning rate adjustment, a batch size of 32, and training for 200 epochs. mAP@0.5 and detection speed in frames per second (FPS) were used as evaluation metrics for comparison with the evaluated baseline algorithms. The results show that the proposed system achieved an mAP@0.5 of 0.91 and a detection speed of 120 FPS. It also demonstrated robust performance under complex conditions such as shading and varying lighting, supporting its effectiveness and practical application potential in production line safety monitoring.

Introduction

In modern industrial production, safety is the cornerstone of ensuring production efficiency and personnel well-being. The production line environment typically involves high-speed mechanical equipment, complex technological processes, and collaborative operations that involve multiple tasks. Any small potential safety hazard can lead to serious production accidents, resulting in significant economic losses and even fatalities. Therefore, it is crucial to establish an efficient, intelligent, and real-time safety monitoring system to enhance safety in industrial production^1,2.

Traditional safety monitoring methods mainly rely on manual video surveillance and various sensors (such as infrared, smoke, and vibration sensors)³. Manual monitoring is not only inefficient but also prone to missed detections due to monitoring personnel fatigue, and it cannot provide continuous and comprehensive coverage. Although sensors can provide a certain early warning function, their detection range is limited, and they are sensitive to environmental interference, resulting in prominent problems of false alarms⁴. Intelligent monitoring systems have become a growing focus of research. Real-time analysis of a video stream using a deep learning algorithm can automatically identify abnormal events, unsafe behaviors, and potential hazards, thereby greatly improving the monitoring system's intelligence^4,5.

Object detection is a core technology in intelligent monitoring. As representatives of single-stage object detectors, the You Only Look Once (YOLO) series algorithms demonstrate significant advantages. From YOLOv1 to YOLOv8, this collection of algorithms has undergone ongoing development, with enhancements to the network architecture, loss function, and training methodology, among other aspects^6,7. YOLOv11, in particular, has achieved higher detection accuracy while preserving a high frame rate, particularly when confronted with complex backgrounds and densely packed targets⁸.

However, despite its excellent performance on general object detection tasks, YOLOv11 still faces challenges in specific production-line safety-monitoring scenarios⁹. For example, YOLOv11 detection accuracy may decrease when workers are partially obscured by equipment or when safety signs are small and closely resemble the background color. This requires the model to have stronger feature extraction and semantic understanding capabilities¹⁰.

Convolutional neural networks (CNNs) have powerful capabilities in image feature extraction¹¹. By designing deeper and more complex network architectures, CNNs can learn richer, more abstract image features. Combining a CNN with YOLOv11 leverages YOLOv11's fast detection capabilities and the CNN's deep feature extraction, thereby building a more robust and intelligent safety monitoring system.

Based on this, this paper proposes a production-line safety monitoring system using YOLOv11 and a CNN. The innovative aspects of this study include: (1) proposing a deep fusion architecture of YOLOv11 and an improved CNN; (2) introducing multi-scale feature fusion and a composite attention mechanism to enhance the recognition of key safety features; and (3) verifying the performance advantages of the model on a self-built complex scene dataset.

Development of YOLO series algorithms
Since its initial introduction by Redmon et al. in 2015¹², the You Only Look Once (YOLO) algorithm has garnered considerable interest. Unlike two-stage detectors that rely on region proposals, YOLO treats object detection as a regression task. By directly predicting the classes and positions of objects in an image, YOLO markedly boosts detection speed, making it suitable for real-time use¹³.

As the pioneering work in this series, YOLOv1 achieves end-to-end target detection for the first time¹⁴. YOLOv1 involves partitioning the input image into a grid structure, where each grid cell, typically arranged in an S × S format, is tasked with detecting objects that fall within its boundaries.

Specifically, YOLOv1's output is a tensor. Among the S × S × (B × 5 + C), the prediction of each bounding box contains 5 elements: center point coordinate (x,y), width w, height h, and confidence c. The calculation process is illustrated in Equation (1):
   (1)

Among them, Pr(Object) represents the probability that there is a target in the grid, and IOU represents the ratio of the intersection over the union between the predicted bounding box and the ground-truth box. Each grid also predicts a set of class probabilities, Pr representing the probability that the target belongs to the i-th class in the presence of a target. The loss function of YOLOv1 is composed of three main components: the loss for coordinate prediction, the loss for confidence estimation, and the loss for category prediction¹⁵.

YOLOv2 introduces Batch Normalization, a high-resolution classifier, and a multi-scale training strategy, which improve stability and accuracy¹⁶. YOLOv3 uses Darknet-53 as its backbone network and draws inspiration from the Feature Pyramid Network (FPN) concept to enhance target detection¹⁷.

YOLOv4 has made numerous optimizations based on YOLOv3, introducing technologies such as Cross Stage Partial Network (CSPNet)¹⁸, Path Aggregation Network (PANet)¹⁹, and Mosaic data enhancement to further improve the model's performance. YOLOv5 has made significant contributions to engineering, offering an easier-to-use and more efficient implementation²⁰.

In recent years, the YOLO series has continued to evolve, with versions such as YOLOv6, YOLOv7, and YOLOv8 being released one after another. By introducing the Extended Efficient Layer Aggregation Network (E-ELAN) structure and model re-parameterization techniques, YOLOv7 achieves new breakthroughs in both speed and accuracy. These enhancements not only improve the efficiency of feature learning but also optimize the network's inference performance, enabling YOLOv7 to surpass previous versions and many mainstream detectors across various real-time object detection tasks. YOLOv8 further optimizes the network structure, adopts an anchor-free design, simplifies the model, and improves its generalization ability²¹.

Building on the advantages of previous versions, YOLOv11, used in this study, has been optimized for complex industrial scenarios. Its core improvements include a feature-extraction network, a more efficient feature-fusion module, and a more robust loss-function design. These improvements enable YOLOv11 to perform better when handling occlusions, small targets, and complex backgrounds in production environments. YOLOv11 is a version of the YOLO series released by Ultralytics. Compared with YOLOv8, it improves the C3K2 module and the feature fusion path, and introduces an adaptive anchor box strategy, thereby providing stronger detection robustness in industrial scenarios.

Convolutional Neural Network (CNN) foundation and progress
The convolutional neural network (CNN) is a pivotal model that has profoundly influenced the success of deep learning within computer vision. It operates by extracting local image features via convolutional operations and subsequently reducing feature dimensionality via pooling operations, thereby achieving hierarchical image abstraction²².

A typical CNN is composed of multiple convolutional, pooling, and fully connected layers. The convolutional layer scans the input image with a convolution kernel, computes dot products over local regions, and produces a feature map. The calculation process is shown in Equation (2):

   (2)

Where x is the input image, w^k and b^k are the weight and bias of the convolution kernel, respectively, and are the values of the output feature map at the position (i,j). The pooling layer commonly uses Max Pooling or Average Pooling. The fully connected layer is responsible for extracting features and predicting classification probabilities.

Based on this, this paper designs a CNN feature enhancement module for YOLOv11, which consists of two parallel branches: a multi-scale convolution branch that uses 3 × 3 and 5 × 5 convolution kernels to extract multi-scale features; and an attention branch that sequentially connects the channel attention (Squeeze-and-Excitation) and spatial attention modules to enhance the discriminative features of key targets. The outputs of the two branches are fused by element-wise addition, and the corresponding feature enhancement loss function is shown in formula (3):

  (3)

L_coord is the bounding box coordinate regression loss; L_conf is the target confidence loss; L_cls is the category classification loss; L_feat is the feature enhancement loss, used to constrain the discriminative features of small targets and occluded targets extracted by the CNN enhancement module; λ_coord, λ_conf, λ_cls, λ_feat are the weight coefficients of the corresponding loss terms. For this task, λ_feat = 0.05 is set, and the remaining weights are balanced according to the detection task requirements.

Classical CNN structures, such as VGGNet²³ and ResNet²⁴, have achieved great success on image classification tasks. Among these architectures, ResNet effectively mitigates the vanishing gradient problem in deep network training, thereby facilitating the construction of deeper and more complex network structures.

In object detection tasks, CNN is usually used as a feature extractor (backbone). For example, Darknet, cross-stage partial Darknet (CSPDarknet), etc., in the YOLO series of algorithms are all built upon CNN²⁵. To enhance feature extraction capabilities, researchers have proposed numerous improved CNN structures. For example, the Inception module extracts features in parallel through multi-scale convolution kernels. DenseNet enhances feature reuse with dense connections. The Squeeze-and-Excitation Network adaptively adjusts the importance of feature channels through attention mechanisms²⁶.

In this study, we designed an improved CNN module to enhance the feature extraction capabilities of YOLOv11. This module combines multi-scale feature fusion with an attention mechanism to more effectively capture key safety information in production line scenarios.

Application status of deep learning in industrial safety monitoring
Deep learning has gained significant traction in industrial safety monitoring. CNN-based algorithms are employed to determine whether workers are properly wearing safety helmets, detect unauthorized intrusions into hazardous zones, and monitor the condition of malfunctioning equipment²⁷.

In terms of behavior recognition, 3D CNNs and recurrent neural networks are used to analyze workers' operational behavior and identify potentially unsafe actions. For example, by analyzing workers' posture sequences, it can be determined whether they are violating safety operating procedures²⁸.

YOLO and Faster R-CNN are widely used for personnel and equipment detection in real-time surveillance videos. For example, a real-time helmet detection system based on YOLOv5 has been proposed in the literature, effectively enhancing the intelligence of construction-site safety management²⁹.

Although some progress has been made in existing research, several challenges remain. First, industrial scenes typically feature complex illumination, occlusion, and background interference, which impose stringent requirements on the algorithm's robustness. Second, real-time performance is a key requirement, and the algorithm must achieve high-speed inference while maintaining accuracy. Finally, existing research primarily focuses on single-task detection and lacks exploration of multi-source information fusion and comprehensive analysis³⁰.

In this study, the proposed YOLOv11 and CNN fusion model aims to address the challenges above and achieve comprehensive, real-time monitoring of production line safety risks by enhancing feature extraction and fusion capabilities.

Protocol

YOLOv11 and CNN algorithm

System overall architecture
The production line safety monitoring system proposed in this paper adopts a hybrid architecture that combines YOLOv11 with an improved CNN to achieve high-precision, high-speed real-time safety monitoring. As shown in Figure 1, the system mainly consists of an input preprocessing module, a YOLOv11 object detection module, a CNN feature enhancement module, and a fusion decision module. The input image is first normalized and augmented by the preprocessing module to improve the model's generalization ability. Subsequently, features are extracted by the CSPDarknet backbone, and the receptive field is expanded by the spatial pyramid pooling-fast (SPPF) module. Multi-scale features are fused after upsampling, and channel attention and spatial attention are applied sequentially to the fused features to further enhance the discriminative representation of key targets. The CNN feature enhancement module is inserted after the outputs of the C3, C4, and C5 layers of the YOLOv11 backbone network, receiving multi-scale feature maps with sizes of 80 × 80 × 256, 40 × 40 × 512, and 20 × 20 × 1,024, respectively. The CNN feature enhancement module further enhances feature extraction for small and occluded targets. Finally, the fusion decision module combines the detection results of YOLOv11 with the CNN-enhanced features and jointly optimizes them through a designed loss function to output the precise location, category, and confidence score of the target.

YOLOv11 detection framework
YOLOv11 has introduced several critical enhancements that leverage the strengths of the YOLO series' single-stage detection framework. Its architecture is divided into three main components: a backbone network, a feature fusion network, and a detection head. The backbone network utilizes an enhanced CSPDarknet structure. The feature fusion network incorporates cross-stage local connections and depthwise separable convolutions, drawing inspiration from feature pyramid networks and path aggregation networks to effectively merge multi-scale features.

Feature enhancement module
The feature enhancement module aims to improve the model's sensitivity to key safety features. It processes the feature maps produced by each layer of the YOLOv11 backbone network, fusing information across different scales through upsampling and downsampling operations. Specifically, the multi-scale branch captures spatial scale diversity, while the attention branch dynamically enhances key channels and positional responses. These two branches do not act independently but are complementary and fused through residual connections and feature recalibration. The feature map processed by the enhancement module at each scale is adjusted to match the original feature map's number of channels via a 1 × 1 convolution, then fused with the original YOLOv11 feature map via element-wise addition. The fused feature map is then fed into the subsequent Feature Pyramid Network (FPN) for multi-scale fusion, thereby forming a hierarchical safety feature representation. To ensure seamless integration between modules, an end-to-end joint training strategy is adopted, with the loss function consisting of the YOLOv11 detection loss and the CNN module feature enhancement loss.

Figure 1. YOLOv11-CNN algorithm. The feature enhancement module aims to amplify the model's sensitivity to critical safety features. It processes feature maps from various layers of the YOLOv11 backbone, fusing features of different scales through upsampling and downsampling operations. Additionally, it incorporates channel- and spatial-attention mechanisms. To ensure seamless integration between the YOLOv11 and CNN modules, a joint training strategy is employed. During training, the parameters of both modules are optimized end-to-end. The loss function combines the detection loss of YOLOv11 with the feature enhancement loss of the CNN module. Please click here to view a larger version of this figure.

Results

To verify the effectiveness of the proposed YOLOv11 and CNN fusion model in production line safety monitoring, a dataset covering various safety hazard scenarios was constructed. This dataset contains 12,000 production line scene images, divided into training, validation, and test sets in a 7:1.5:1.5 ratio, with a fixed random seed of 42. It covers various working conditions, including standard lighting, low lighting, partial occlusion, heavy occlusion, motion blur, and complex backgrounds. The confidence threshold for inference was set to 0.5, and the non-maximum suppression (NMS) threshold was set to 0.45. All experiments were repeated three times, and results are reported as mean ± standard deviation. The annotation categories are workers, safety helmets, robotic arms, hazardous areas, tools, and vehicles, using PASCAL Visual Object Classes (VOC) format. The intersection-over-union (IoU) threshold is set to 0.5, and annotation consistency is verified through two-person cross-validation. Input images were uniformly scaled to 640 × 640 and augmented with Mosaic data. Training was performed using the SGD optimizer with an initial learning rate of 0.01, momentum of 0.9, weight decay of 0.0005, and a batch size of 32. The training ran for 200 epochs, and cosine annealing was used to adjust the learning rate.

To thoroughly assess the model's effectiveness, a variety of evaluation metrics were utilized: mean average precision (mAP), frames per second (FPS) to gauge the rate of image frame processing, precision to evaluate the accuracy of positive predictions, recall to measure the model's ability to detect true positives, and area under the curve (AUC) to indicate the area under the receiver operating characteristic (ROC) curve, thereby reflecting the model's overall classification capability. The calculation formulas are shown in Equations (4), (5), (6), (7), (8), (9), (10), (11):

  (4)
  (5)
  (6)
Recall  (7)
  (8)
  (9)
  (10)
  (11)

  Here, mAP@0.5 indicates the mean average precision at an IoU threshold of 0.5; N is the number of categories; AP_i is the average precision of the i-th category; and mAP@[0.5:0.95] is the average mAP calculated across IoU thresholds from 0.5 to 0.95. TP, FP, FN, and TN represent the numbers of true positives, false positives, false negatives, and true negatives, respectively. F is the total number of frames processed, T is the total processing time, TPR is the true-positive rate, and FPR is the false-positive rate.
For the YOLOv11 detection component, the loss function is shown in Equation (12):

     (12)

  Where L_coord denotes the deviation between the predicted frame and the real frame, Lconf measures the confidence error of the predicted frame, Lclass measures the category prediction error, and λcoord, λconf, and λclass are the weight coefficients used to balance the corresponding losses.

Analysis of the data in Table 1 shows that the proposed method achieves an mAP@0.5 of 0.91 and an mAP@0.5:0.95 of 0.82, representing improvements of 0.04 and 0.04, respectively, compared to YOLOv5. Its F1-score is 0.935, which is also higher than those of the comparison models. The detection speed is 120 FPS, four times that of Faster R-CNN, but slightly lower than YOLOv10 and YOLOv11. The number of parameters is 6.7M, only 1.5M more than YOLOv11, with a 0.03 improvement in mAP@0.5. Compared to EfficientDet, which has a similar computational cost, the accuracy is 0.06 higher, while the number of parameters is 56% lower. The data demonstrate that the introduced multi-scale fusion and attention mechanism effectively enhances small-target detection accuracy, achieving a good balance between speed and accuracy. In summary, our proposed method achieves a balance between detection accuracy and speed. The mAP@0.5 is 0.02 higher than that of the second-best model, the F1-score reaches 0.935, and the 6.7M parameters are sufficient for practical deployment. The multi-scale fusion and attention mechanism enhances the recognition of small and occluded targets in complex scenes. The model balances accuracy and efficiency, making it suitable for real-time scenarios such as industrial safety monitoring. Of note, all comparison models use official pretrained weights, with a uniform input resolution of 640 × 640, NMS threshold of 0.45, and confidence threshold of 0.5.

Method	mAP@0.5	mAP@0.5:0.95	F1-score	FPS	Params (M)
YOLOv5	0.87	0.78	0.89	130	7.1
EfficientDet	0.85	0.73	0.88	50	15.3
RetinaNet	0.82	0.68	0.85	45	37.6
Faster R-CNN	0.84	0.7	0.87	30	45.2
YOLOv10	0.89	0.77	0.88	135	5.3
YOLOv11	0.88	0.75	0.895	140	5.2
OURS	0.91	0.82	0.935	120	6.7

Table 1: Comprehensive comparison table of model performance. Analysis of the data shows that the proposed method achieves mAP@0.5 of 0.91 and mAP@0.5:0.95 of 0.82, representing improvements of 0.04 and 0.04, respectively, over YOLOv5. Its F1-score is 0.935, which is also higher than those of the comparison models. The detection speed is 120 FPS, four times that of Faster R-CNN, but slightly lower than YOLOv10 and YOLOv11. The number of parameters is 6.7M, only 1.5M more than YOLOv11, with a 0.03 improvement in mAP@0.5. Compared to EfficientDet, which has a similar computational cost, the accuracy is 0.06 higher, while the number of parameters is 56% lower. The data demonstrate that the introduced multi-scale fusion and attention mechanism effectively enhances the accuracy of small-target detection, achieving a good balance between speed and accuracy. In summary, our proposed method achieves a balance between detection accuracy and speed. The mAP@0.5 is 0.02 higher than that of the second-best model, the F1-score reaches 0.935, and the 6.7M parameters are sufficient for practical deployment. The multi-scale fusion and attention mechanism enhances the recognition of small and occluded targets in complex scenes. The model balances accuracy and efficiency, making it suitable for real-time scenarios such as industrial safety monitoring. Of note, all comparison models use official pretrained weights, with a uniform input resolution of 640 × 640, a uniform NMS threshold of 0.45, and a uniform confidence threshold of 0.5.

Figure 2 provides an in-depth analysis of the model's false detection types. YOLOv11 + CNN has the lowest background false-detection rate (85 times/10,000 frames), with most false detections occurring in similar objects (62 times) and partially occluded scenes (48 times). ROC curve analysis indicates that the model's TPR reached 0.9 (AUC = 0.98) at an FPR of 0.1, and the narrowest spacing between curves supports the high reliability of detection confidence. These analysis results not only support the model's performance but also provide a clear technical roadmap for subsequent false-detection optimization, particularly by strengthening its ability to distinguish similar objects.

Figure 2. Error analysis. Analysis of the model's false detection types. YOLOv11 + CNN has the lowest background false detection rate (85 times/10,000 frames), with the majority of false detections concentrated in similar objects (62 times) and partially occluded scenes (48 times). ROC curve analysis indicates that the model's TPR reached 0.9 (AUC = 0.98) at an FPR of 0.1, and the narrowest spacing between curves supports the reliability of the detection confidence. These analysis results not only support the model's performance but also provide a clear technical roadmap for subsequent false-detection optimization, particularly by strengthening its ability to distinguish similar objects. Please click here to view a larger version of this figure.

Figure 3 compares the model's performance on various hardware platforms. YOLOv11 + CNN achieves an ultra-low latency of 18 ms on the Edge tensor processing unit (TPU), with a fluctuation of only ±2 ms, and power consumption is controlled at 15 W. Throughput tests demonstrate that the model achieves a processing speed of 70 FPS on Jetson Xavier edge computing devices, with the 99th percentile latency controlled within 50 ms. These deployment performance indicators enable it to adapt to the deployment needs of various computing platforms in industrial fields. The error analysis further shows that the model can maintain stable performance in resource-constrained environments, demonstrating its engineering applicability.

Figure 3. Deployment performance. Comparison of the model's performance on various hardware platforms. YOLOv11 + CNN achieves an ultra-low latency of 18 ms on the Edge TPU (with a fluctuation of only ±2 ms), and power consumption is controlled at 15 W. Throughput tests demonstrate that the model achieves a processing speed of 70 FPS on Jetson Xavier edge computing devices, with the 99th percentile latency controlled within 50 ms. These deployment performance indicators indicate that it can adapt to the deployment needs of various computing platforms in industrial fields. The error analysis further shows that the model can maintain stable performance in resource-constrained environments, demonstrating its engineering applicability. Please click here to view a larger version of this figure.

Figure 4 compares the detection performance differences of each model across six target categories. YOLOv11 + CNN achieved an accuracy of 0.96 on the safety helmet recognition task, surpassing the benchmark model by 4 percentage points. The dotted plot indicates that the model exhibits minimal performance fluctuations across categories (±0.05), reflecting its generalization capabilities. The confusion matrix, presented as a heat map, reveals a 15% mutual false detection between the hazardous area and the robotic arm. This finding provides a clear direction for subsequent model optimization.

Figure 4. Category detection analysis. Comparison of the detection performance differences of each model across six target categories. YOLOv11 + CNN achieved an accuracy of 0.96 on the safety helmet recognition task, surpassing the benchmark model by 4 percentage points. The dotted plot indicates that the model exhibits minimal performance fluctuations across categories (±0.05), reflecting its generalization capabilities. The confusion matrix, presented as a heat map, reveals a 15% mutual false detection between the hazardous area and the robotic arm. This finding provides a clear direction for subsequent model optimization. Please click here to view a larger version of this figure.

Figure 5 provides a comprehensive record of the training process for each model. YOLOv11 + CNN exhibits the fastest convergence rate, with the loss dropping to 0.1 within 30 epochs, and the gap between its validation-set mAP curve and the training set is consistently less than 1%. The error band plot analysis shows that the model's final validation mAP@0.5 is stable at 0.94 ± 0.01, and its performance fluctuation range is only one-third of that of RetinaNet. These results support the effectiveness of the joint training protocol. The model shows performance advantages in the early stages of training, indicating that its architectural design enables it to learn features quickly.

Figure 5. Analysis of the training process. A record of the training process for each model. YOLOv11 + CNN exhibits the fastest convergence rate (the loss drops to 0.1 within 30 epochs), and the gap between its validation set mAP curve and the training set is always less than 1%. The error band plot analysis shows that the final validation mAP@0.5 of the model is stable at 0.94 ± 0.01, and the performance fluctuation range is only one-third of that of RetinaNet. These results support the effectiveness of the joint training protocol. The model shows performance advantages in the early stages of training, indicating that its architectural design enables it to learn features quickly. Please click here to view a larger version of this figure.

Table 2 presents quantitative performance results for different detection models in complex environments. Test data across standard lighting conditions and various extreme scenarios show that YOLOv11 + CNN maintains optimal performance under all conditions. Under standard lighting conditions, the mAP@0.5 of this model reached 0.91, significantly better than YOLOv5 (0.87) and Faster R-CNN (0.84). With the deterioration of environmental conditions, the performance of each model decreases to varying degrees, but YOLOv11 + CNN shows the strongest environmental robustness: under low light conditions (< 50 lux), the performance only decreases by 3% (to 0.88), which is better than the 5% reduction of the comparative model; in heavily occluded scenarios (> 50%), its mAP can still be maintained at 0.78, and the performance degradation (13%) is significantly smaller than that of YOLOv5 (15%) and Faster R-CNN (16%). These results demonstrate that YOLOv11 + CNN enhances the model's adaptability to complex factors, such as illumination changes and occlusions, through improved feature fusion and attention mechanisms, thereby supporting practical applications in industrial scenarios.

Test Condition	YOLOv11 + CNN	YOLOv5	Faster R-CNN	Performance Fluctuation
Standard Lighting	0.91	0.87	0.84	0.01
Low Light (< 50 lux)	0.88 (-3%)	0.82 (-5%)	0.79 (-5%)	0.02
Partial Occlusion (30%–50%)	0.85 (-6%)	0.80 (-7%)	0.76 (-8%)	0.03
Heavy Occlusion (> 50%)	0.78 (-13%)	0.72 (-15%)	0.68 (-16%)	0.04
Motion Blur	0.83 (-8%)	0.77 (-10%)	0.73 (-11%)	0.05

Table 2: Quantitative analysis table of environmental adaptability. Performance of different detection models in complex environments through quantitative analysis. Test data from standard lighting conditions to various extreme scenarios show that YOLOv11 + CNN maintains the highest performance under all test conditions. Under standard lighting conditions, the mAP@0.5 of this model reached 0.91, significantly better than YOLOv5 (0.87) and Faster R-CNN (0.84). With the deterioration of environmental conditions, the performance of each model decreases to varying degrees, but YOLOv11 + CNN shows the strongest environmental robustness among the evaluated models: under low light conditions (< 50 lux), the performance only decreases by 3% (to 0.88), which is better than the 5% reduction of the comparative model; in heavily occluded scenarios (> 50%), its mAP can still be maintained at 0.78, and the performance degradation (13%) is significantly smaller than that of YOLOv5 (15%) and Faster R-CNN (16%). These results demonstrate that YOLOv11 + CNN enhances the model's adaptability to complex factors, such as illumination changes and occlusion interference, through an improved feature fusion mechanism and attention design, supporting practical applications in industrial scenarios.

Table 3 shows that the SGD optimizer was used in this experiment, with a momentum of 0.9 to accelerate convergence and suppress oscillations. An initial learning rate of 0.01 was used with cosine annealing, which gradually decayed during training to refine optimization. A weight decay of 0.0005 was introduced to apply L2 regularization and reduce overfitting. A batch size of 32 balanced computational efficiency with gradient estimation stability. The 200 training epochs ensured sufficient convergence. The parameters were adapted to the single-stage detection task, balancing training speed and accuracy.

Parameter	Value
Optimizer	SGD
Initial learning rate	0.01
Momentum	0.9
Weight decay	0.0005
Batch size	32
Training epochs	200
Learning rate scheduling	Cosine annealing

Table 3: Training hyperparameter table. The SGD optimizer was used in this experiment, with a momentum of 0.9 to accelerate convergence and suppress oscillations. An initial learning rate of 0.01 was used with cosine annealing, which gradually decayed during training to refine optimization. A weight decay of 0.0005 was introduced to apply L2 regularization and reduce overfitting. A batch size of 32 balanced computational efficiency with gradient estimation stability. The 200 training epochs ensured sufficient convergence. The parameters were adapted to the single-stage detection task, balancing training speed and accuracy.

Table 4 shows that, using YOLOv11 as the baseline, the mAP@0.5 is 0.89. Introducing multi-scale fusion alone reduces the accuracy to 0.88. Further layering channel attention and spatial attention sequentially improves the accuracy to 0.90 and 0.89, respectively. The combined accuracy of all modules reaches 0.91, an improvement of 0.02 over the baseline. The data show that channel attention directly improves accuracy, and the combined performance of multi-scale and attention is optimal.

Configuration	mAP@0.5
YOLOv11 (baseline)	0.89
+ Multi-scale fusion	0.88
+ Multi-scale + channel attention	0.9
+ Multi-scale + spatial attention	0.89
Full module (YOLOv11 + CNN)	0.91

Table 4: Ablation test table. Using YOLOv11 as the baseline, the mAP@0.5 is 0.89. Introducing multi-scale fusion alone reduces the accuracy to 0.88. Adding channel attention or spatial attention after multi-scale fusion improves the accuracy to 0.90 and 0.89, respectively. The combined accuracy of all modules reaches 0.91, an improvement of 0.02 over the baseline. The data show that channel attention yields greater gains than spatial attention in this setting, and that the combined performance of multi-scale fusion and attention is optimal.

Figure 6 systematically evaluates the model's performance in extreme environments. The mAP of YOLOv11 + CNN dropped by only 6% (to 0.88) in low light conditions and still maintained an mAP of 0.78 (8% ahead of the benchmark) in heavily occluded scenes. These results demonstrate that the model exhibits environmental adaptability, effectively addressing common illumination changes and local occlusion issues in industrial production, thereby supporting the stable operation of the detection system.

Figure 6. Environmental adaptability. A systematic evaluation of the model's performance in extreme environments. The mAP of YOLOv11 + CNN dropped by only 6% (to 0.88) in low light conditions, and still maintained an mAP of 0.78 (8% ahead of the benchmark) in heavily occluded scenes. These results demonstrate that the model exhibits environmental adaptability, effectively addressing common illumination changes and local occlusion issues in industrial production, thereby supporting the stable operation of the detection system. Please click here to view a larger version of this figure.

Figure 7 compares the feature distribution differences between YOLOv11 and YOLOv11 + CNN on six types of industrial targets through t-SNE dimensionality reduction. The left figure shows that the features of the basic model YOLOv11 exhibit significant intraclass dispersion (intraclass distance 2.34 ± 0.15), particularly in the "Danger Zone" (red) and "Vehicle" (purple), where there is a significant overlap, which is consistent with the 15% false detection rate in experimental group 3. The figure on the right shows that the improved YOLOv11 + CNN significantly enhances intraclass compactness via the feature enhancement module, increasing the average distance between cluster centers within each category by 1.8 times.

Figure 7. t-SNE Feature Distribution Comparison. A comparison of the feature distribution differences between YOLOv11 and YOLOv11 + CNN on six types of industrial targets through t-SNE dimensionality reduction. The left figure shows that the features of the basic model YOLOv11 exhibit significant intraclass dispersion (intraclass distance 2.34 ± 0.15), particularly in the "Danger Zone" (red) and "Vehicle" (purple) where there is substantial overlap, which is consistent with the 15% false detection rate in experimental group 3. The figure on the right shows that the improved YOLOv11 + CNN significantly improves the intraclass compactness through the feature enhancement module, and the average distance between cluster centers in each category is increased by 1.8 times. Please click here to view a larger version of this figure.

Figure 8 verifies the contribution of each module through systematic ablation experiments. The ablation results show that adding channel attention after multi-scale fusion increases mAP@0.5 to 0.90, while adding spatial attention increases mAP@0.5 to 0.89. The full module achieves the highest mAP@0.5 of 0.91. The heat map visualization shows that the compound attention mechanism can simultaneously enhance the detection response to the center of the hazardous area (activation value: +0.15) and small targets at the edge (+0.08). The experimental data demonstrate that multi-scale feature fusion and attention mechanisms have a cumulative effect, enabling the model to meet the target-detection requirements at different scales.

Figure 8. Modular ablation experiment. Verification of the contribution of each module through systematic ablation experiments. The ablation results show that adding channel attention after multi-scale fusion increases mAP@0.5 to 0.90, while adding spatial attention increases mAP@0.5 to 0.89. The full module achieves the highest mAP@0.5 of 0.91. The heat map visualization shows that the compound attention mechanism can simultaneously enhance the detection response to the center of the hazardous area (activation value: +0.15) and small targets at the edge (+0.08). The experimental data demonstrate that multi-scale feature fusion and attention mechanisms have a cumulative effect, enabling the model to meet the target-detection requirements across different scales. Please click here to view a larger version of this figure.

DATA AVAILABILITY:
Full raw production-line images and video streams are subject to industrial confidentiality restrictions and are not publicly released. A Supplementary Dataset Description (Supplemental File 1) provides dataset acquisition details, category and scenario distributions, annotation specifications, quality-control procedures, data splits, preprocessing, and augmentation procedures. A Pseudocode and Reproducibility Framework (Supplemental File 2) provides a pseudocode-level workflow, configuration descriptions, and guidance for reproduction. An anonymized subset and additional supporting materials may be requested from the corresponding author for non-commercial academic research, subject to institutional and confidentiality restrictions.

Supplemental File 1. Supplementary dataset description. This file describes the dataset acquisition protocol, scenario coverage, class distribution, annotation specifications, quality-control procedures, train/validation/test partitioning, preprocessing, and augmentation procedures. Please click here to download this file.

Supplemental File 2. Pseudocode and reproducibility framework. This file provides pseudocode-level workflow and configuration details for preprocessing, training, evaluation, and deployment, and describes restrictions on raw industrial footage and access to supporting materials. Please click here to download this file.

Discussion

Experimental results show that the proposed YOLOv11–CNN fusion model improves detection accuracy and real-time performance for production line safety monitoring. Leveraging the efficient single-stage detection of YOLOv11 and the feature enhancement of the CNN module, the system identified workers, equipment, and hazardous areas under complex lighting and occlusion conditions. Multi-scale feature fusion and attention mechanisms enhanced the ability to focus on key safety elements, providing interpretable visual evidence for early warning.

The system has practical value for production lines in settings such as automobile manufacturing and electronic assembly, where real-time monitoring can reduce reliance on manual inspection and shorten response times to potential safety hazards. For small targets, such as screws and tools, and for worker targets partially occluded by robotic arms or other equipment, the multi-scale feature fusion and attention mechanisms improve the model's ability to extract and emphasize safety-related features. These improvements are relevant for production environments in which targets vary in size and visibility.

However, this study still has clear limitations. Under severe occlusion or very low illumination, the model's performance may decline because target feature information becomes incomplete, and existing convolutional features may be insufficient. The newly added CNN feature enhancement module introduces 1.5M additional parameters, which may increase the computational burden on low-cost edge devices and limit deployment in resource-constrained environments. This study uses visible-light image data and does not integrate multimodal sensor information, such as infrared thermal imaging or millimeter-wave radar; therefore, performance may be limited in extreme industrial scenes such as complete darkness or smoke.

Future research will focus on the following directions: model lightweighting based on structured pruning and knowledge distillation to reduce the parameter burden while maintaining detection performance; construction of a visible-infrared multimodal fusion detection framework to improve detection ability under extreme illumination and smoke environments; introduction of a lightweight Transformer module to replace part of the convolutional layers and enhance long-distance context feature extraction; and development of an end-to-end abnormal behavior recognition subsystem to support the full workflow from target detection to behavior early warning.

In conclusion, this study focuses on a production-line safety monitoring system that fuses YOLOv11 and CNN algorithms to enable real-time detection and early warning of potential safety hazards in complex industrial environments. Through experimental verification, the fusion model showed performance advantages in object detection accuracy and inference speed. With its efficient single-stage detection architecture and optimized feature extraction, YOLOv11 performed well in rapidly identifying workers, equipment, and hazardous areas, and accurately captured key safety elements in production line scenarios. At the same time, the introduction of the improved CNN module further enhanced feature capture and detection of occluded targets. In production line safety monitoring, the model automatically focuses on key safety areas in the image through visual analysis, providing an interpretable basis for safety warnings. The fusion model demonstrated strong adaptability and stability across various production-line scenarios, including machining, assembly, and warehousing. The main work of this paper is as follows:

(1) A deep fusion architecture of YOLOv11 and CNN was designed to balance detection speed and accuracy.
(2) A multi-scale feature fusion and attention optimization mechanism was built to enhance the ability to focus on key safety elements in complex scenarios.
(3) Experiments on the self-built production line safety dataset showed that the model reached an mAP@0.5 of 0.91 and a detection speed of 120 FPS. In cross-scenario testing, the model showed stable performance.

Typical case studies showed that under standard lighting conditions, a safety helmet with only 40% exposure was accurately detected by multi-scale fusion and channel attention (confidence 0.93), while the baseline model misidentified it as background. In low-light scenarios, when a worker entered a hazardous area, spatial attention successfully bounded the target with directional lines, whereas the comparison method failed to detect it. Failures included a wrench being misdetected as a tool due to its texture similarity to the background under severe occlusion, and a safety helmet being missed due to a low signal-to-noise ratio at a distance under low-light conditions, revealing limitations in feature representation under extreme conditions. These results indicate that the model is robust in normal and moderately complex scenarios but still needs improvement in extreme conditions.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
COCO API		N/A	Assessment tool used for target detection accuracy evaluation and statistical analysis.
CUDA 11.4	NVIDIA	N/A	Parallel computing platform used with the PyTorch 1.12 framework.
Edge TPU		N/A	Deployment platform used for latency and power consumption testing; reported latency was 18 ms and power consumption was 15 W.
Jetson Xavier edge computing device	NVIDIA		Edge computing device used for throughput testing; reported processing speed was 70 FPS with 99th percentile latency within 50 ms.
NVIDIA Tesla V100 GPU	NVIDIA		GPU used in the training/evaluation server.
PyTorch 1.12		N/A	Deep learning framework used for model training and evaluation.
scikit-learn		N/A	Assessment/statistical analysis tool used for model evaluation.
Self-built production line safety dataset	N/A	N/A	Dataset of 12,000 production line scene images in VOC format, covering standard lighting, low lighting, partial occlusion, heavy occlusion, motion blur, and complex backgrounds; six annotation categories: workers, safety helmets, robotic arms, hazardous areas, tools, and vehicles.
Training server			Server equipped with an NVIDIA Tesla V100 GPU and Ubuntu 20.04 operating system.
Ubuntu 20.04 operating system		N/A	Operating system used on the training/evaluation server.
VOC annotation format	N/A	N/A	Annotation format used for the self-built production line safety dataset.
YOLOv11 object detection model	Ultralytics	N/A	Object detection model used to detect workers, equipment, and potential hazards in real time.