$$\rightleftharpoonup{xx}$$
$$\longleftharp{xx}$$,
$$\longrightharp{xx}$$,
2. XAI framework
Most state-of-the-art AI models-particularly deep neural networks-are often treated as black boxes; there is little visibility into how they make decisions12. That lack of transparency is a barrier to trust and explainability in diverse applications. As a result, XAI has become an indispensable part of trustworthy AI. No single approach fits all scenarios: some methods assume full access to model internals, whereas others treat the model as an immutable black box; some prioritize local explanations for individual predictions, while others provide global insight into a model's behavior. As depicted in Figure 1, these considerations have led to two broad categories of XAI techniques based on when explainability is introduced: post-hoc explanation methods and intrinsic interpretability methods13. Post hoc methods are further categorized into model-agnostic and model-specific14.

Figure 1: XAI framework for visual inspection. Please click here to view a larger version of this figure.
- Post-hoc explanation methods
Post-hoc methods generate explanations after a model has been trained, without altering its architecture or parameters15. These approaches are popular because they allow practitioners to take advantage of high-performance models and then retrospectively explain their outputs16. However, because the underlying model remains unchanged, post-hoc explanations must approximate or infer the model's reasoning, which can introduce approximation error and instability17. In the following sections, both model-specific and agnostic methods are explained.
- Model-specific
These methods exploit knowledge of a model's structure or training mechanism to provide explanations tailored to that model. A prime example is gradient-based attribution for deep neural networks. Methods like GradCAM (Gradient-weighted Class Activation Mapping) use the internal gradients of trained CNNs to produce visual "heatmaps" highlighting the regions of an input image most important for a given prediction17. GradCAM computes the gradient of the target class score with respect to feature maps in the final convolutional layer and projects these gradients onto the image, yielding a class-discriminative localization map. Other gradient propagation methods generalize to different network architectures. Integrated Gradients (IG)18 accumulates gradients along the path from a baseline input (e.g., a blank image) to the actual input, and satisfies desirable axioms for feature attribution. DeepLIFT19 similarly propagates differences from a reference activation through each layer to estimate each input feature's contribution. Layer-wise Relevance Propagation (LRP) back-propagates a prediction score layer by layer, distributing "relevance" to each input pixel or feature20. In summary, model-specific methods take advantage of internal gradients, activations, or attention to produce explanations tightly coupled to the model's structure.
- Model-agnostic
In contrast, model-agnostic methods treat the model as a black-box that can be queried for outputs. These techniques make minimal assumptions, typically requiring only that one can obtain the model's prediction for perturbed inputs. Examples include LIME (Local Interpretable Model-Agnostic Explanations), which uses a simple, interpretable model (such as a sparse linear model or decision tree) to mimic the complex model's behavior in the vicinity of a given input21. For a specific prediction, LIME generates many perturbed versions of the input, gets the black-box model's predictions for each, and then learns a weighted linear regression that approximates those local input-output relationships22. The coefficients of this surrogate linear model serve as an explanation, indicating which features most strongly drive the prediction23. LIME's appeal lies in its flexibility (usable with any classifier) and its intuitive output (e.g., a small set of weighted features). SHapley Additive exPlanations (SHAP) is another popular model-agnostic method based on game theory24. SHAP explains an individual prediction by computing feature attributions that approximate Shapley values from cooperative game theory, ensuring properties like additivity and consistency25. Beyond feature attribution, model-agnostic approaches also include visual perturbation methods for image models. RISE (Randomized Input Sampling for Explanation) generates importance maps by randomly masking out parts of the input image and observing how the model's output changes26. By sampling many such masks, RISE builds a probabilistic saliency map indicating which pixels or regions were most influential to the prediction26. Notably, RISE requires no access to model internals and only requires the ability to evaluate the model on modified inputs, making it truly black-box. Finally, a growing area of model-agnostic XAI focuses on counterfactual explanations. A counterfactual explanation does not enumerate feature importances but instead answers the question: "What minimal change to the input would have altered the model's prediction?"27. Algorithms to generate counterfactuals typically perform a search in input space (or a learned latent space) for the nearest example that yields the desired output, while enforcing realism and sparsity in the changes. In summary, post-hoc XAI methods are widely adopted because they can be applied to high-performance models after training, combining predictive power with a degree of transparency.
- Intrinsic interpretability methods
Intrinsic methods take a fundamentally different approach: rather than explaining a black-box after the fact, these methods make the model interpretable by design. Self-Explaining Neural Networks (SENN) are a notable example. In the SENN framework proposed by Alvarez-Melis and Jaakola28, the neural network is structured to first compute a set of intermediate concept activations and then generate a prediction as a simple, explainable function of those concepts. ProtoPNet (Prototypical Part Network) is a deep network that classifies inputs by comparing them to a set of learned prototype representations of each class. For instance, in an image classification task, a ProtoPNet might learn prototype patches and make decisions by finding which prototypes are most activated by the input29. In the realm of sequence models and transformers, attention mechanisms have been leveraged for intrinsic interpretability as well. Some models are trained with an attention alignment objective, which encourages the model's attention weights to correspond to known important features or to human-labeled relevant regions. By guiding attention to be "right for the right reasons"30, these approaches produce models that not only perform well but also have attention distributions that can be directly interpreted as explanations. Moreover, such models provide explanations that are typically faithful by construction, since the explanation is generated from the model's own interpretable structure, not from an external post-hoc approximation31.
3. XAI taxonomy
To provide a structured synthesis of the state of research, a systematic literature review was conducted using the PRISMA guidelines (Figure 2). The process included multiple academic databases (Scopus, Web of Science, IEEE Xplore, SpringerLink, PubMed, arXiv, and MDPI) and was guided by a set of search terms combining "explainable AI," "interpretability," "visual inspection," "defect detection," "anomaly detection," "manufacturing," and "medical imaging." Inclusion criteria required that papers explicitly applied or evaluated XAI methods in visual inspection contexts (industrial or medical) and reported sufficient methodological detail. Exclusion criteria removed purely theoretical proposals without implementation, as well as works outside visual inspection. The search initially retrieved 483 records. After removing duplicates and screening titles and abstracts, 147 remained for full-text review. Applying inclusion/exclusion criteria yielded 75 primary studies, which form the basis of this review.

Figure 2: PRISMA flow diagram of the systematic review process following PRISMA guidelines. The figure illustrates records identified, screened, and excluded at each stage, yielding 75 primary studies included in the final review. Please click here to view a larger version of this figure.
- Applied taxonomy of reviewed studies
Building on the conceptual framework introduced in Section 2 (post-hoc and intrinsic approaches), we refined a domain-specific taxonomy derived from these 75 studies. This applied taxonomy distinguishes methods according to their implementation in visual inspection tasks as shown in Figure 3.
- Distribution of studies across taxonomy
The full distribution of the 75 studies is summarized in Table 1. Each study is mapped to its methodological category and application domain. This provides an empirical foundation for the comparative analysis presented in Section 4.
- Synthesis
Several clear trends emerge from this systematic review. First, gradient-based post-hoc methods remain the most widely adopted in industrial and medical inspection tasks due to their efficiency and ease of integration. Second, perturbation-based methods provide richer feature attributions but are less common in high-throughput industrial settings due to computational demands. Third, intrinsic methods are gaining traction but remain underrepresented, with fewer than 15% of studies implementing them.
By anchoring this taxonomy in a reproducible PRISMA-guided review and linking it to a structured database of 75 studies, this section establishes a validated evidence base. These findings set the stage for the comparative analysis in Section 4, where representative methods are benchmarked across standardized metrics (fidelity, robustness, latency, and localization accuracy).

Figure 3: Proposed taxonomy of XAI methods in visual inspection tasks. Please click here to view a larger version of this figure.
| XAI Method | XAI Approach | Study | Industry | Year |
| Intrinsic | Rule-based & Linear Models | 32,33 | General, Medical | 2015–2019 |
| Prototype-based (ProtoPNet, Case-based, Deformable ProtoPNet) | 29,34,35,36 | Medical, Industrial, General | 2018–2024 |
| Concept-driven (Concept Bottleneck, Right-for-Right-Reasons) | 30,31,37 | General, Scientific Applications | 2020–2021 |
| Self-Explaining Neural Networks (SENN) | 28 | General | 2018–2020 |
| Disentangled Representation (β-VAE),SOXAI | 38,39 | General | 2018 |
| Vision Transformers (Intrinsic interpretability via attention) | 40,41,42,43,44,45 | Computer Vision, Manufacturing | 2020–2024 |
| Post-hoc, Model-specific | CAM Family | 46,47,48,49,50,51,52, 53,54,55,56,57,58 | Medical Imaging, Manufacturing, Automotive, Agriculture, Electronics | 2018–2025 |
| LRP (Layer-wise Relevance Propagation) | 20,59,60,61,62 | General, Medical Imaging | 2015–2024 |
| Gradient & Saliency Methods (Integrated Gradients, Smooth IG DeepLIFT, DeconvNet, T-Explainer, Saliency Benchmarks) | 18,19,63,64,65,
66,67,68,69,70 | General, Medical Imaging | 2013–2025 |
| Attribution in Specialized Models (SNNs, Transformers attention attribution) | 71,72,73 | Neuroscience, Vision Transformers | 2023–2025 |
| Post-hoc, Model-agnostic | LIME & Variants (LIME, ELIME, MASALA), SHAP & Variants (Tree SHAP), | | Finance, Prognostics and Health Management, Cybersecurity, Autonomous Vehicles, Industrial Inspection. | 2016–2025 |
| Counterfactual | 21,22,23,24,25,27,74,
75,76,77,78,79,80,81,
82,83,84,85,86 |
| Explanations | |
| Hybrid methods | Integration of multiple XAI methods including (GradCAM,SHAP, LRP,LIME) | 58,87,88,89,90,91,92 | Industrial Inspection, Healthcare, Manufacturing. | 2021–2025 |
Table 1: Classification of 75 reviewed studies by XAI method, year, industry, and approach.
4. Evaluation metrics for XAI
The performance of a computer vision model is judged by well-established metrics such as accuracy, precision, and recall. However, evaluating the explainability of a model's decisions is far less straightforward. Unlike standard predictive performance, there is no universally accepted metric for the quality of AI explanations66. In other words, while a classifier's accuracy can be precisely measured, there is no agreed-upon scale to quantify how "good" an explanation is for a given image classification93. This disparity highlights a fundamental gap in the field of XAI: how to evaluate an explanation.
Developing standardized XAI evaluation criteria has proven exceptionally challenging. One reason is that interpretability and explanation quality are multi-faceted and context-dependent and therefore resist a single universal metric10. Assessing an explanation effectively relies on several factors, including the application domain, the nature of the explanation produced, and the end-user's background. For instance, an explanation technique suitable for a medical image diagnosis might be evaluated differently from one used in an industrial visual inspection task10.
In practice, the absence of standard metrics has led researchers to rely on a variety of evaluation approaches. Many studies still resort to qualitative, anecdotal evaluation of XAI methods94,95. It is common to see authors present a few saliency maps or heatmaps and argue that these appear reasonable (the so-called "face-validity" test)96. Such visual inspection can be persuasive in a narrative sense, but it remains subjective and insufficient for rigorous comparison97. This illustrates the difficulty of standardizing XAI metrics; in the absence of agreed-upon criteria, researchers propose metrics tailored to each dataset or use case.
As a result, the literature now abounds with diverse XAI evaluation metrics intended to fill this gap98. Numerous frameworks and measures have been put forward, each targeting a particular aspect of what makes an explanation useful99. Broadly, metrics can be classified based on their primary focus. Fidelity-based metrics judge an explanation by how well it reflects the decision process of the model100. In contrast, interpretability-centered metrics evaluate how comprehensible and convincing the explanation is to a human observer101. A third category is task-based metrics, where the assessment of the explanation is dependent on its impact on end-user performance102. A fourth category is robustness-oriented metrics that evaluate the consistency of the results103. Finally, hybrid metrics combine multiple dimensions104.
This diversity of metrics indicates the aspects of explainability that researchers consider important but also highlights the lack of a single standard: each metric captures only one angle of explanation quality105. In the following, a range of evaluation metrics for XAI are grouped into four main categories and summarized in Table 2.
| Metric Type | Metric | Formula / Definition |
| Fidelity | Deletion Area under the curve (AUC)26 | 
are top-k important features ; lower = more faithful |
| Insertion Area under the curve (AUC)26 | 
higher = more faithful |
| Infidelity score103 | ![figure-protocol-6 Mathematical equation, Eδ[(δᵀE(x)−(f(x)−f(x−δ)))²], statistical analysis formula.](/files/ftp_upload/69440/69440eq3.jpg)
lower = better |
| ROAR106 | Accuracy drops after retraining without 'important' features |
| Robustness | Stability index107 | 
higher = more consistent |
| Lipschitz score103 | 
lower = more robust |
| Interpretability | Sparsity79 | 
lower = simpler |
| Surrogate depth79 | Depth of interpretable surrogate (e.g., decision tree);
shallower = simpler |
| Task-specific | intersection-over-union (IoU)108 | 
overlap of mask M and ground-truth G |
| Pointing Game108 | Accuracy = #hits / #samples
higher = better |
| Hybrid | Composite score104 | Weighted combination of fidelity, sparsity, and robustness |
Table 2: Evaluation metrics for XAI in visual inspection grouped into four main categories.
5. Comparative analysis
In this section, methods are compared across public datasets such as MVTec AD109 (>5,000 defect images with pixel-level masks) and DeepPCB110 (1,500 PCB images with six defect classes), CheXpert111 and ImageNet112 and evaluated across the following metrics: Deletion/Insertion AUC (faithfulness), Pointing-Game accuracy (localization), stability scores (robustness under perturbations), and latency (runtime cost).
For CAM-based methods (GradCAM, GradCAM++) results on MVTec AD and DeepPCB show that GradCAM achieves fidelity of 0.83-0.87 (insertion AUC ~0.68; <15% accuracy drop under deletion), sensitivity (0.80 - 0.85), and runtime <100 ms (~39 Frames Per Second (FPS)). Localization remains moderate (mean IoU ≈0.28 @ δ=0.25; pointing accuracy 86-87%). GradCAM++ improves multi-instance localization (mean IoU = 0.38) while maintaining similar latency (<120 ms) and higher user trust (109.69/250 vs. 56.08/250 for GradCAM). These methods are efficient but spatially coarse47,113.
For perturbation-based methods (LIME, SHAP, RISE), performance varies across datasets.On MVTec AD, LIME reaches local fidelity R² = 0.70-0.80 but shows low robustness (stability index <0.5), latency of 3-5 s/image, and moderate localization (IoU = 0.25-0.35). SHAP applied to PCB defect classification achieves consistent attributions (Δ log-odds = 0.2-0.3) with strong human alignment (70-80%) but is slower (>1 s/image) and has weak localization (IoU ≈0.02-0.03). RISE, benchmarked on MS-COCO and replicated for MVTec AD, averages 4,000-8,000 masked forward passes, yielding high robustness (stability >0.7), fidelity (0.78-0.82 AUC), and superior localization (Pointing-Game = 62.9% vs. 48.3% for GradCAM), but at 1-2 s/explanation latency26,74,79,114.
For Relevance-propagation methods (LRP, DeepLIFT), evaluations on DeepPCB and CheXpert indicate that LRP achieves fidelity of 0.82-0.90, sensitivity ~0.85, and localization IoU of 0.40-0.50. Runtime is 100-200 ms per image, slower than CAMs but feasible for near-real-time inspection. In CheXpert radiology tasks, expert studies show >70% overlap between LRP heatmaps and clinically relevant regions, demonstrating interpretability across domains60.
For Attention-based methods (Transformers), results for ViT models trained on ImageNet and CheXpert, raw attention maps yield fidelity of 0.75-0.85 but are unstable under perturbations, with sensitivity of 0.70-0.75 and modest localization (IoU = 0.30-0.45). Human trust is ~60-70%. Adapted approaches (attention-flow, Transformer-LRP) improve causal metrics, outperforming raw attention in deletion/insertion and pointing-game scores40,115,116.
For Prototype- and counterfactual-based methods, evaluations n MVTec AD highlight that ProtoPNet achieves exemplar-based explanations where predictions are justified by similar defect cases, aiding operator decision-making. SOM-based approaches cluster defect features on 2D maps, preserving topological relationships. Counterfactual methods, tested on both PCB defect images and CheXpert, provide actionable reasoning ("if crack length is less than 1 mm, prediction shifts to normal") but require costly optimization and struggle with high-resolution inputs. These methods align strongly with human reasoning but scale poorly117,118,119.
Finally, when considering trade-offs and practical selection, CAMs dominate latency-critical workflows but remain coarse. LRP balances fidelity and runtime, excelling at pixel-level analysis. RISE provides the strongest causal faithfulness and localization but at seconds-level latency. SHAP and LIME enhance interpretability but are slow and unstable. Prototypes and counterfactuals maximize human alignment but scale poorly in high-resolution inspection. Transformer-based models require specialized adaptations beyond raw attention. Thus, no single method universally outperforms others; instead, method choice must reflect dataset, task, latency, budget, and interpretability needs. As shown in Figure 4, the selection workflow integrates three of these benchmarks (task, latency, interpretability) into a task-oriented guide for choosing XAI methods in visual inspection.

Figure 4: Selection workflow for XAI method adoption. This is a decision flowchart guiding the selection of appropriate XAI methods in visual inspection. The workflow considers task type (classification, segmentation, anomaly detection, regression), latency constraints, and explanation detail, and links them to representative methods. Please click here to view a larger version of this figure.
6. Discussion
- Comparative findings
Our analysis shows that gradient-based methods such as GradCAM and Layer-wise Relevance Propagation (LRP) excel at highlighting the specific image regions most influential in a convolutional network's decisions, making them intuitive tools for tasks such as defect localization or medical image classification. These approaches are efficient since they operate in a single backward pass, but their explanations are often coarse and may become unclear in complex regions. In contrast, model-agnostic feature attribution techniques, such as SHAP and LIME, provide more quantitative insights into feature contributions by probing the model with perturbed inputs or fitting local surrogates, but at a higher computational cost than GradCAM.
Attention-based methods leverage the model's internal weights to highlight the regions of focus within the model115. RISE, on the other hand, represents a randomization-based strategy that masks and samples inputs to produce flexible saliency maps, though it requires thousands of forward passes. Taken together, these techniques excel under specific circumstances, but none universally outperforms the others26. Instead, the choice of method depends on model architecture, the visual inspection task, latency requirements, and the type of information needed by decision-makers.
- Case studies
The following are real-world case studies implementing XAI in both medical and industrial sectors: in healthcare, pneumonia and COVID-19 detection models trained on chest X-rays and CT scans were explained with GradCAM and LIME, and radiologists in user studies consistently preferred GradCAM because its highlighted regions aligned with expected pathology such as lung infiltrates120. Similarly, in digital pathology for head and neck cancer, CNN predictions supported by GradCAM and high-resolution CAM highlighted malignant cell clusters that pathologists confirmed as clinically relevant, showing that explanations reinforced both clinical accuracy and trust121.
In the manufacturing sector line, XAI has made a transformative difference in accuracy and explainability. One of the main applications is foreign object detection on X-ray images of tires employing a convolutional neural network based on the Xception architecture. Through the inclusion of Gradient-weighted Class Activation Mapping (GradCAM), the system achieved greater than 99% classification accuracy and produced heatmaps indicating the precise regions within the image responsible for predictions. These visual explanations allowed QA engineers to verify that the model's attention was on legitimate anomalies within the structural support for the tire and not irrelevant artifacts, vastly improving trust and facilitating deployment in high-stakes production settings48.
In the electronics industry, the SolderNet approach shows the integration of XAI into pixel-level defect inspection. The CNN-based method was developed for the inspection of solder joints on printed circuit boards, a task where high accuracy and explainability are mandatory. With the application of GradCAM and Layer-wise Relevance Propagation (LRP), the method extracted saliency maps that highlighted the pixels responsible for specific defect classifications. Through such explainability, the operator was able to differentiate true and false positives and draw insights into failure modes that previously would have been black-box outputs39. These cases show that XAI is valued when its explanations align with expert reasoning, but underused when explanations are unstable, coarse, or computationally impractical.
- Limitations of current evidence
Despite these advances, several limitations persist. First, most empirical studies rely on public datasets such as MVTec AD for industrial inspection or VinDR-CXR122 for medical imaging, which may not fully represent real-world environments of factories or hospitals. Second, evaluation remains fragmented: fidelity, robustness, and interpretability are measured inconsistently across studies, often with ad hoc definitions and without standardized thresholds. Third, many works lack human validation. While metrics such as deletion AUC, pointing game accuracy, or localization IoU provide quantitative results, they do not confirm that these explanations actually improve user decision-making. Finally, only ~8% of AI-based quality assurance studies in manufacturing integrate XAI in their workflow123. This highlights the practical adoption gap, even though this statistic varies by sector and methodology and may not be accurate.
- Practical implications for practitioners
For practitioners, several implications emerge. Gradient-based methods remain the most practical choice for real-time inspection tasks, as they deliver explanations with a low latency (~100 ms per image) and are easy to deploy. Perturbation-based approaches, such as SHAP or RISE, have higher latency than Gradient-based methods but are better suited to domains where fine-grained attribution outweighs latency, such as healthcare, where accuracy takes precedence over speed. Attention-based approaches are useful with transformer architectures, but they require careful interpretation as raw attention does not always provide a clear explanation. Beyond technical trade-offs, successful adoption depends highly on integration, user training, and demonstrating the clinical value of interpretability. As the case studies show, explanations only gain value when they clearly align with domain expertise and help practitioners verify or challenge AI predictions.
- Future research directions
Future research should prioritize several actionable directions:
Standardization of evaluation protocols: Establish reproducible reporting guidelines for key metrics, including deletion/insertion AUC, pointing game accuracy, stability indices, and latency.
Domain-specific benchmark datasets: Develop curated datasets for industrial and medical visual inspection with ground-truth annotations to reduce over-reliance on narrow public datasets.
Human-centered validation: Expand studies that measure whether XAI explanations improve decision quality, user trust, and human-AI collaboration beyond proxy metrics
Cost - benefit analysis: Conduct rigorous deployment studies to quantify the business and clinical value of integrating XAI into visual inspection workflows.