Adaptive NLI-Driven Claim Verification with Statistical Decision Modeling for Low-Latency Hallucination Reduction in Large Language Models

Summary

This study presents a lightweight framework for reducing hallucinations in large language models through claim-level verification and adaptive statistical thresholding. By selectively correcting unsupported claims using Natural Language Inference, the approach improves factual accuracy while maintaining low response latency and practical deployment efficiency.

Abstract

Large Language Models (LLMs) exhibit a critical tendency to generate factually incorrect yet linguistically fluent outputs - a phenomenon termed hallucination - which poses serious risks in precision-critical applications. Existing mitigation strategies, including retrieval-augmented generation and self-consistency sampling, either introduce substantial inference latency or depend on external knowledge infrastructure, limiting their applicability in real-time deployments. This paper proposes a lightweight two-step claim verification framework that decomposes LLM responses into atomic factual claims and independently verifies each extracted claim against a separately generated reference produced through an isolated factual recall prompt. Although the generator and verifier share the same underlying language model, separating response generation from factual recall reduces direct response conditioning and mitigates confirmation bias during verification, using Natural Language Inference, and applies an adaptive statistical threshold - defined as τ = µ + kσ over the NLI confidence score distribution - to selectively correct only contradicted claims. Unlike prior NLI-based methods that rely on fixed decision boundaries, the proposed framework dynamically adapts its verification threshold to the confidence distribution of each response, showing consistent performance across the evaluated benchmarks without requiring model retraining. Evaluated on TruthfulQA and FEVER, the framework reduces the hallucination rate from 28% to 9% on TruthfulQA - a 67.9% relative reduction - while incurring only 160 ms of additional latency over the baseline LLM and outperforming SelfCheckGPT and FActScore in hallucination detection accuracy. These results indicate that the framework can provide a favorable balance between factual reliability and response latency on the evaluated benchmarks, while further validation across domains and deployment settings is needed.

Introduction

Large Language Models (LLMs) have emerged as foundational components of modern artificial intelligence systems, demonstrating remarkable capability across a broad spectrum of natural language tasks, including text generation, question answering, summarization, and complex reasoning¹. Their ability to produce fluent, contextually coherent responses has accelerated adoption across high-impact domains such as clinical decision support, legal analysis, software engineering, and educational technology². However, a critical and persistent limitation undermines their reliability in these settings: hallucination, in which models generate responses that are linguistically fluent yet factually incorrect, unsupported, or entirely fabricated 3. Empirical studies report hallucination rates ranging from 15% to over 40% depending on the task and model, with even state-of-the-art systems such as GPT-4 exhibiting measurable factual inconsistencies under domain-specific evaluation^2,3. This limitation poses serious risks in precision-critical applications where erroneous outputs can lead to misinformation, misdiagnosis, or flawed decision-making⁴. Hallucinations arise fundamentally from the probabilistic nature of language modeling - LLMs generate tokens by predicting statistically likely continuations of input sequences rather than by retrieving or reasoning over verified factual knowledge⁵. As a consequence, generated outputs may embed plausible-sounding but inaccurate claims, introduce unsupported assertions, or conflate semantically related but factually distinct concepts⁶.

Existing mitigation strategies address this problem through several paradigms, each with notable limitations. Retrieval-Augmented Generation (RAG) reduces hallucination by grounding responses in externally retrieved documents, but introduces significant latency overhead, requires access to maintained knowledge bases, and remains vulnerable to retrieval failures in specialized or low-resource domains^7,8,9.

Although the response generator and reference generator are instantiated using the same underlying GPT-3.5 Turbo model, they operate under different prompting objectives and execution contexts. The response generator produces an unconstrained answer to the user query, whereas the reference generator performs an isolated factual recall task without access to the previously generated response. This separation reduces direct response-conditioning effects and limits confirmation bias during claim verification. The framework, therefore, treats the generated reference as procedurally independent rather than statistically independent.

Recent research has also explored lightweight decoding strategies to reduce hallucinations in text generation without relying on external retrieval or repeated verification. One representative approach is Decoding by Contrasting Layers (DoLA), which improves factuality by contrasting representations from intermediate and final transformer layers during decoding. Unlike post-generation verification frameworks, such methods intervene directly during token generation and therefore optimize a different stage of the language generation process. These complementary strategies demonstrate that hallucination mitigation can be achieved either during decoding or through post-generation verification, with each approach offering distinct trade-offs in computational overhead, implementation complexity, and factual reliability.

Self-consistency methods improve factual reliability by sampling multiple responses and selecting the most frequent or coherent output, yet their computational cost scales linearly with the number of samples, making them unsuitable for latency-sensitive deployments⁸. Iterative refinement approaches, which repeatedly revise outputs through self-feedback loops, progressively reduce error rates but substantially extend inference time with each additional pass⁹. NLI-based verification methods offer a more targeted alternative by evaluating semantic entailment between generated claims and reference texts, but existing implementations rely on fixed decision thresholds that fail to adapt across varying confidence distributions, domains, or model behaviors^10,11. Collectively, these approaches either impose unacceptable computational overhead, depend on external infrastructure, or lack the adaptability required for generalized deployment. A comparison of the principal characteristics of these hallucination-verification approaches is provided in Supplementary Table 1.

This paper proposes a lightweight, two-step claim-verification framework designed to reduce hallucinations in LLM outputs while maintaining low response latency. In the first step, the LLM generates an initial response, which is then decomposed into atomic factual claims. In the second step, each claim is verified using Natural Language Inference against a separately generated factual reference produced through an isolated reasoning pass that does not access the original generated response, and a statistically derived confidence threshold governs the accept-or-correct decision at the claim level. The primary contributions of this work are as follows: (1) A two-step claim-level verification pipeline that decouples response generation from factual validation, enabling independent and targeted evaluation of each atomic claim without requiring external retrieval or full response regeneration; (2) An adaptive statistical thresholding mechanism based on the distribution of NLI confidence scores (τ = µ + kσ), which dynamically determines acceptance and correction boundaries rather than relying on fixed decision rules, improving robustness across varying confidence distributions; (3) A selective correction strategy that restricts regeneration to only those claims classified as contradictory, substantially reducing computational overhead compared to full-response resampling or iterative refinement approaches; (4) Empirical evaluation on a hallucination-focused benchmark demonstrating that the proposed framework reduces hallucination rates from 28% to 9% while maintaining response latency within practical deployment bounds.

Protocol

This study did not involve human participants, animals, biological specimens, or patient data. Therefore, institutional ethical approval and informed consent were not required.

Study design overview

A two-stage verification framework was implemented to improve the factual reliability of large language model (LLM) outputs. The procedure consisted of response generation, claim extraction, Isolated Reference Construction, Natural Language Inference (NLI)-based verification, adaptive statistical decision-making, selective correction, and final response assembly. The framework was evaluated using TruthfulQA and FEVER benchmark datasets.

Overall workflow

User Query → Initial Response Generation → Claim Extraction → Independent Reference Generation → NLI Verification → Statistical Threshold Computation → Claim Correction → Final Verified Response. The overall workflow of the adaptive claim-verification framework is illustrated in Figure 1.

Dataset preparation

The TruthfulQA dataset¹², containing 817 fact-oriented questions, was downloaded and prepared for evaluation. The FEVER dataset¹³ comprises 185,445 annotated claims labeled as Supported, Refuted, or Not Enough Information; the evaluation used the labeled development data with its claim, evidence, and ground-truth label fields. Each claim was evaluated against its provided evidence, while the original FEVER label was retained as the reference outcome for verification. The FEVER dataset, which contains fact-verification claim-evidence pairs, was obtained and preprocessed. Input questions and claims were converted into a standardized text format. Duplicate entries and incomplete samples were removed before experimentation. TruthfulQA was divided into validation and testing subsets. Approximately 20% of the TruthfulQA samples (164 questions) were used for threshold tuning, while the remaining 653 questions were reserved for performance evaluation. For FEVER, the benchmark-provided claims were used directly as verification units and did not undergo the response-generation and claim-extraction procedure applied to TruthfulQA. The characteristics of the benchmark datasets used for framework development and evaluation are summarized in Table 1.

Initial response generation

Each query was submitted to the base language model. Responses were generated using nucleus sampling with a temperature of 0.7 and a top-p of 0.9. The maximum output length was limited to 256 tokens. Generated responses were stored without any post-processing or manual modification. The generated text was forwarded directly to the claim extraction stage.

Claim extraction

Each generated response was decomposed into independently verifiable factual statements. A structured decomposition prompt was used to identify atomic claims. Compound statements were separated into minimal factual units. Subjective opinions, stylistic expressions, and conversational fillers were removed. Each extracted claim was stored as an independent verification unit.

Example:

Original response:

"Marie Curie was a Polish-born physicist and chemist who conducted pioneering research on radioactivity."

Extracted claims: (1) Marie Curie was Polish-born; (2) Marie Curie was a physicist and chemist; (3) Marie Curie conducted research on radioactivity.

Isolated reference construction:

For every extracted claim, an independent factual reference was generated. The verifier model received only the original user query. Access to the original generated response was restricted to avoid confirmation bias. A factual recall prompt was used to encourage concise, evidence-based responses. Generated references were stored for subsequent verification.

Natural language inference verification

Each claim-reference pair was submitted to a pre-trained NLI model. The NLI model classified the relationship as: Entailment, Neutral, or Contradiction. Confidence probabilities for all three classes were recorded. A signed verification score was assigned: Positive value for entailment, zero for neutral, or a negative value for contradiction. All verification scores were collected for threshold computation.

Adaptive statistical threshold computation

The verification confidence produced by the NLI model varies substantially across responses because different queries generate different numbers of claims, level of semantic complexity, and confidence distributions. A fixed global threshold assumes that all responses follow a similar confidence profile, which is rarely observed in practice. To accommodate this variability, the proposed framework estimates the decision boundary individually for each response using the mean and standard deviation of its verification scores. The mean reflects the overall confidence level of the response, while the standard deviation captures the dispersion of confidence values among the extracted claims. This adaptive formulation allows the acceptance criterion to adjust to response-specific uncertainty rather than relying on a single threshold for all inputs.

The mean (µ) and standard deviation (σ) of all signed verification scores were calculated.

The adaptive decision threshold was computed as:

where τ represents the adaptive threshold, µ denotes the mean verification score, σ denotes the standard deviation, and k represents the sensitivity parameter.

The sensitivity parameter was initialized at 0.7. Claims classified as Entailment with scores greater than or equal to +τ were accepted. Claims classified as Contradiction with scores less than or equal to −τ were marked for correction. Claims with scores within the interval (−τ, +τ) were retained as neutral.

Selective claim correction

A claim was forwarded for correction only when the NLI model classified the claim-reference pair as Contradiction, and its corresponding signed verification score satisfied the adaptive threshold criterion s_i ≤ -τ. Thus, the correction decision was jointly determined by the NLI class label and the response-specific statistical threshold. Claims classified as Entailment with s_i ≥ τ were accepted, whereas claims falling within the interval -τ < s_i < τ were treated as neutral and retained without correction. This combined criterion ensured that correction was applied only to claims exhibiting both semantic contradiction and sufficiently strong negative verification evidence.

Response reconstruction

Accepted and corrected claims were arranged in their original sequence. Sentence boundaries were restored to maintain readability. Minor grammatical adjustments were applied when necessary. The assembled output was stored as the final verified response.

Performance evaluation

The framework was evaluated using four metrics: (1) Accuracy: The proportion of responses that remained factually correct after verification; (2) F1 Score: The harmonic mean of hallucination detection precision and recall; (3) Response Latency: The total processing time from query submission to verified response generation; (4) Hallucination Rate

Latency was reported as the mean execution time across repeated measurements. Because individual per-run latency values were not retained, standard deviations and confidence intervals were not calculated.

Benchmark-specific correctness evaluation

For TruthfulQA, the final verified response was compared against the benchmark's human-validated answer references and lists of incorrect answers. A response was counted as correct when its factual claims were consistent with the accepted answer information and did not contain content corresponding to the identified incorrect response patterns. For FEVER, each claim in the final output was evaluated against its associated evidence and original FEVER annotation; Supported claims were treated as factually verified, while Refuted claims were counted as incorrect. Not Enough Information cases were retained as indeterminate rather than treated as positive factual evidence. The resulting claim-level decisions were aggregated across each benchmark to calculate the reported Accuracy and Hallucination Rate.

Experimental environment

The framework was implemented using Python 3.9. Data processing operations were performed using numerical and tabular analysis libraries. The NLI model operated in inference mode without additional fine-tuning. Experiments were executed on a workstation equipped with an Intel Core i5 processor, 16 GB RAM, and a 64-bit operating system. All evaluations were conducted using a single-pass inference setting to ensure low-latency operation. All experiments were conducted using identical hardware and software configurations to ensure consistent evaluation across datasets and baseline methods.

Expected outcome

The protocol is designed to generate claim-level verification decisions by identifying potentially unsupported claims and selectively forwarding strongly contradicted claims for correction. The expected output is a verified response with reduced factual inconsistencies and limited additional processing overhead, subject to the performance observed during experimental evaluation.

Results

Initial response generation

The baseline language model generated fluent and contextually relevant answers for questions from both benchmark datasets. However, a detailed examination revealed unsupported and factually inaccurate statements in several responses. On the TruthfulQA dataset, the baseline system exhibited a hallucination rate of 28%, while a hallucination rate of 26% was observed on the FEVER dataset. These findings confirmed that direct language generation alone was insufficient for applications requiring high factual reliability and established the baseline condition against which all subsequent verification procedures were compared.

Claim extraction

The claim extraction procedure successfully decomposed generated responses into independently verifiable factual units. Responses from TruthfulQA contained an average of 3.2 atomic claims, while FEVER samples generally consisted of a single claim. The decomposition process isolated factual assertions without introducing additional information, allowing each statement to be evaluated separately. This observation supported the hypothesis that claim-level verification provides greater precision than evaluating entire responses as a single unit.

Independent reference construction

Independent references were generated for each extracted claim using a separate reasoning process conditioned only on the original query. The generated references provided concise factual descriptions that were sufficiently distinct from the original responses to serve as independent verification sources. The reference-generation process was isolated from the initial response to avoid directly conditioning the factual reference on the model’s earlier output. This separation was intended to reduce potential confirmation bias and provide a controlled basis for subsequent claim-level verification; the study does not independently quantify the extent of this effect. The successful generation of independent references supported the proposed design principle of separating knowledge recall from response generation.

Natural language inference verification

The NLI verification stage effectively classified claim-reference pairs into entailment, contradiction, and neutral categories. Contradicted claims consistently received negative verification scores, whereas factually supported claims received positive scores. Neutral classifications were assigned to claims for which insufficient supporting evidence was available. This behavior demonstrated that semantic inference could reliably identify unsupported factual statements and provided the evidence required for targeted correction. Compared with a simple consistency-checking strategy, NLI verification reduced the hallucination rate from 20% to 15%, indicating improved detection capability.

Adaptive statistical thresholding

The application of adaptive statistical thresholding further improved verification performance by dynamically adjusting decision boundaries based on the confidence score distribution of each response. Threshold sensitivity analysis demonstrated that performance improved as the threshold parameter increased from 0.3 to 0.7. At a sensitivity value of 0.7, the framework achieved an accuracy of 0.87 while reducing hallucinations to 9% (Figure 2). The complete sensitivity analysis results for the evaluated values of k are provided in Supplementary Table 2. Increasing the threshold beyond this point yielded only minor gains in accuracy while increasing latency. These observations supported the hypothesis that adaptive thresholds are more effective than fixed decision boundaries for handling varying confidence distributions across responses.

The adaptive threshold also reflects the variability of confidence scores within each response. Responses with highly consistent verification scores produce a smaller standard deviation, resulting in a more selective decision boundary. In contrast, responses containing claims with heterogeneous confidence values yield a larger standard deviation, leading to a broader acceptance region and reducing unnecessary corrections for uncertain claims. Although this adaptive behavior cannot eliminate every false positive or false negative, it enables the decision boundary to respond to the uncertainty characteristics of each response instead of applying a uniform criterion across all inputs.

Selective claim correction and response assembly

Only claims identified as contradictory were submitted for correction, while supported and neutral claims were retained unchanged. This selective correction strategy minimized unnecessary regeneration and preserved the original structure of the response. Following correction and response reconstruction, the hallucination rate on TruthfulQA decreased from 28% to 9%, representing a relative reduction of 67.9%. Similar improvements were observed on FEVER, where hallucinations decreased from 26% to 10%. Accuracy and hallucination-rate comparisons across the evaluated configurations are presented in Figures 3 and 4, respectively.

Performance evaluation

Comparative evaluation showed that the proposed framework achieved the highest overall performance among all tested methods. On TruthfulQA, the framework achieved an accuracy of 0.87 and an F1 score of 0.85, outperforming both fixed-threshold verification and self-consistency-based approaches (Table 2, Figures 3 and 4). On FEVER, the framework achieved an accuracy of 0.89 and an F1 score of 0.87 (Table 3, Figures 3 and 4). The incremental contribution of the framework components is examined through the ablation analysis presented in Table 4. These improvements confirmed that combining claim-level verification with adaptive statistical decision-making enhanced factual reliability across multiple datasets. Hallucination-detection accuracy for SelfCheckGPT, FActScore, and the proposed framework is compared in Figure 5.

Latency analysis

The complete verification pipeline maintained low computational overhead. The average response time increased from 820 ms for the baseline model to 980 ms for the full framework, representing only a modest increase in processing time (Table 5). Response generation accounted for most of the total latency, while verification and correction stages contributed relatively little additional overhead. The stage-level processing-time breakdown is provided in Supplementary Table 3. Compared with retrieval-based verification systems, which required approximately 1600 ms per query, the proposed framework achieved substantially lower latency while maintaining comparable factual accuracy. These findings supported the hypothesis that hallucination reduction can be achieved without sacrificing real-time usability.

Because response generation relies on a cloud-based language model, individual latency measurements are subject to fluctuations due to network conditions and server-side scheduling, rather than to a deterministic execution time. Repeated measurements were therefore used to obtain representative average values, reducing the influence of transient execution variability while preserving relative comparisons among the evaluated methods. Latency values are reported as mean execution times across repeated experimental runs. Individual per-run latency values were not retained; therefore, post hoc calculation of variance, confidence intervals, or other measures of variability were not possible. Accordingly, the latency values and the speed–accuracy comparison in Figure 6 are presented as point estimates without error bars.

To conclude, the results demonstrated that combining claim extraction, independent reference generation, Natural Language Inference verification, adaptive statistical thresholding, and selective correction improved the factual reliability of large language model outputs. The framework reduced the hallucination rate from 28% to 9% on TruthfulQA and from 26% to 10% on FEVER. The results indicate that adaptive claim-level verification improved factual-reliability metrics while limiting additional response latency under the evaluated conditions

Data Availability

The datasets analyzed in this study are publicly available through their respective official sources. The manuscript provides the dataset information, model configurations, and methodological details necessary to support replication of the described analyses. Processed evaluation data and supplementary results generated during the study are provided in the Supplementary Files.

Figure 1: Workflow of the adaptive claim-verification framework. An initial LLM-generated response is decomposed into factual claims, followed by independent reference generation, NLI-based verification, confidence scoring, and statistical thresholding. Claims meeting the acceptance criterion are retained, whereas claims meeting the correction criterion undergo targeted correction before final response assembly. Please click here to view a larger version of this figure.

Figure 2: Effect of the sensitivity parameter (k) on verification accuracy. Accuracy on TruthfulQA and FEVER at k values of 0.3, 0.5, 0.7, and 1.0. Accuracy increased with k on both datasets, with k = 0.7 selected for the primary evaluation. Please click here to view a larger version of this figure.

Figure 3: Accuracy comparison across verification methods. Accuracy of the baseline LLM, NLI-based verification, and the proposed adaptive verification framework on TruthfulQA and FEVER. Please click here to view a larger version of this figure.

Figure 4: Hallucination-rate comparison across verification methods. Hallucination rates for the baseline LLM, NLI-based verification, and the proposed framework on TruthfulQA and FEVER. The proposed framework reduced the rate from 28% to 9% on TruthfulQA and from 26% to 10% on FEVER. Please click here to view a larger version of this figure.

Figure 5: Hallucination-detection accuracy across methods. Detection accuracy of SelfCheckGPT, FActScore, and the proposed framework on TruthfulQA and FEVER. Please click here to view a larger version of this figure.

Figure 6: Response-time and accuracy trade-off across verification methods. Relationship between response latency and accuracy for the evaluated methods on TruthfulQA and FEVER, illustrating the performance–latency trade-off associated with the proposed framework and comparator approaches. Please click here to view a larger version of this figure.

Dataset	Samples Used	Domains	Avg. Response Length (tokens)	Baseline LLM Hallucination Rate
TruthfulQA	817	38 (science, history, law, etc.)	42	28%
FEVER	1,000	General factual claims	18	26%

Table 1: Benchmark dataset characteristics. Summary of the TruthfulQA and FEVER datasets used for framework development and evaluation, including sample numbers, baseline accuracy, baseline hallucination rate, and average claims per sample.

Method	Accuracy	Precision	Recall	F1 Score	Hallucination %
Baseline LLM (Single Inference)	0.72	0.71	0.69	0.7	28%
NLI-Based Verification (Fixed Threshold)	0.82	0.815	0.785	0.8	15%
SelfCheckGPT	0.76	0.755	0.725	0.74	22%
FActScore	0.85	0.8425	0.8175	0.83	11%
Proposed Framework (Statistical Threshold)	0.87	0.86	0.84	0.85	9%

Table 2: Performance comparison on TruthfulQA. Accuracy, precision, recall, F1 score, and hallucination rate for the baseline LLM, SelfCheckGPT, FActScore, NLI verification, and the proposed framework.

Method	Accuracy	Precision	Recall	F1 Score	Hallucination %
SelfCheckGPT	0.78	—	—	—	—
FActScore	0.87	—	—	—	—
Baseline LLM†	0.74	0.73	0.71	0.72	26%
NLI-Based Verification (Fixed Threshold)	0.83	0.8225	0.7975	0.81	14%
Proposed Framework (Statistical Threshold)	0.89	0.8775	0.8625	0.87	10%

Table 3: Performance comparison on FEVER. Accuracy, precision, recall, F1 score, and hallucination rate for the baseline LLM, SelfCheckGPT, FActScore, NLI verification, and the proposed framework.

Configuration	Accuracy	Precision	Recall	F1 (added)	Hallucination Rate
Baseline Language Model	0.72	0.71	0.69	0.7	28%
Baseline + Secondary Check (Without NLI)	0.78	0.7725	0.7475	0.76*	20%
Baseline + NLI-Based Verification (Fixed Threshold)	0.82	0.815	0.785	0.8	15%
Baseline + Statistical Decision Module (Full)	0.87	0.86	0.84	0.85	9%

Table 4: Ablation analysis of framework components. Changes in accuracy, F1 score, and hallucination rate following sequential incorporation of secondary validation, NLI verification, and adaptive statistical thresholding.

Method	Mean Response Time (ms)
Baseline LLM (Single Generation)	820
Self-Validation Mechanism	910
Proposed Statistical Framework	980
Retrieval-Augmented Verification (RAG)	1600

Table 5: Response latency across verification methods. Mean response time and additional latency relative to the baseline LLM for Self-Validation, the proposed framework, and retrieval-augmented verification.

Supplementary Table 1: Comparison of hallucination-verification approaches. Comparison of the proposed framework with existing methods in terms of external retrieval, claim-level verification, adaptive thresholding, and latency-efficient processing.Please click here to download this file.

Supplementary Table 2: Sensitivity analysis of the threshold parameter (k). Accuracy, precision, recall, F1 score, and hallucination rate were obtained at threshold parameter values of 0.3, 0.5, 0.7, and 1.0. A value of k = 0.7 was used for the primary evaluation.Please click here to download this file.

Supplementary Table 3: Processing time across stages of the verification pipeline. Mean execution time and percentage contribution of initial response generation, claim decomposition, reference generation, NLI inference, and selective correction to the total response time.Please click here to download this file.

Supplementary Table 4: Distribution of errors identified during verification. Number and percentage of false negatives, false positives, and correction errors, together with the primary hallucination categories associated with each error type.Please click here to download this file.

Discussion

The proposed framework improved factual-verification performance while avoiding repeated sampling and external retrieval. On TruthfulQA, the hallucination rate decreased from 28% for the baseline LLM to 9% with the complete framework, while accuracy increased from 0.72 to 0.87. On FEVER, the hallucination rate decreased from 26% to 10%, with accuracy increasing from 0.74 to 0.89. These comparisons should be interpreted within the experimental settings and implementations used in this study rather than as evidence of universal superiority across deployment conditions. The framework performs post-generation claim-level verification using an isolated factual reference and selective correction, providing a trade-off between external knowledge grounding, repeated inference, and lightweight post-generation verification. The response and reference generators use the same underlying GPT-3.5 Turbo model but operate under different prompting objectives and isolated execution contexts. The reference generator receives only the original query and does not access the previously generated response. Thus, the two generation processes are procedurally rather than statistically independent and may retain correlated factual biases originating from their shared pretrained model.

Comparison with existing approaches

The fixed-threshold NLI baseline applies a confidence threshold of 0.7 based on prior work on NLI-based factuality checking¹⁴. SelfCheckGPT¹⁵ represents a zero-resource hallucination-detection approach that generates multiple stochastic samples and uses NLI to identify inconsistent claims. FActScore¹⁶ decomposes generated responses into atomic facts and verifies them against retrieved references from a Wikipedia-based knowledge source. In contrast, the proposed framework performs claim-level verification without repeated full-response generation or external retrieval.

DoLA is a complementary lightweight approach that enhances factual generation by contrasting token probabilities derived from different transformer layers during inference. A direct experimental comparison was not included because DoLA modifies the token-generation process, whereas the proposed framework performs post-generation claim-level verification and selective correction. Implementing DoLA would require access to internal transformer-layer representations, which are not exposed by the GPT-3.5 Turbo API used in this study. The absence of external retrieval reduces retrieval dependency and associated processing requirements but also limits verification to knowledge available to the underlying models. Consequently, complete fabrications or specialized claims outside the verifier's knowledge coverage remain difficult to correct.

Adaptive statistical thresholding and selective correction

The adaptive threshold is derived from the empirical confidence-score distribution and controls the trade-off between hallucination-detection recall and correction precision. Sensitivity analysis evaluated values of 0.3, 0.5, 0.7, and 1.0 on a held-out TruthfulQA validation split. A value of k = 0.7 provided the most balanced trade-off between hallucination reduction and computational efficiency for the evaluated benchmarks.

The optimal threshold may vary across application domains because the distribution of NLI confidence scores depends on the nature of the generated content. For application to a new domain, a validation set reflecting the target application can therefore be used to evaluate candidate values and select a value that balances factual-verification performance, correction rate, and processing latency. Because threshold optimization involves a single scalar parameter rather than retraining the models, adaptation does not require modifying the underlying models.

Selective correction restricts regeneration to claims classified as Contradiction and satisfying the statistical threshold criterion. This avoids repeated processing of claims that do not meet the correction criterion and distinguishes the framework from full-response resampling and iterative refinement approaches.

Latency and performance trade-off

The proposed framework achieved a mean response time of 980 ms compared with 820 ms for the baseline LLM and 1600 ms for retrieval-augmented verification. Stage-level analysis showed that initial response generation accounted for 83.7% of total latency, while NLI inference contributed 3.9% and selective correction contributed less than 1% on average. Latency values are reported as mean execution times across repeated experimental runs. Individual per-run latency values were not retained; therefore, post hoc calculation of variance, confidence intervals, or other measures of variability was not possible. Accordingly, latency values and the speed–accuracy comparison are presented as point estimates without error bars. The reported latency values reflect the experimental environment used in this study and may vary under different deployment conditions, particularly when cloud-based language model APIs are employed. Network latency, server workload, and service availability can independently influence absolute response times, regardless of the proposed verification framework.

Error analysis

Claims incorrectly handled in the TruthfulQA test set were categorized as false negatives, false positives, or correction errors. The distribution and principal categories of these errors are summarized in Supplementary Table 4. False negatives accounted for 52.6% of all errors and were primarily associated with temporal hallucinations and subtle factual substitutions. Temporal errors may be missed when the verifier shares the training cutoff of the base model, while subtle factual substitutions may receive NLI scores in the Neutral band rather than Contradiction.

False positives accounted for 35.9% of errors and occurred primarily for uncommon, highly specific, or recently established facts outside the verifier model's high-confidence knowledge coverage. These cases produced weak NLI entailment scores despite the claims being factually correct. Correction errors accounted for 11.5% of all errors and occurred when complete fabrications were identified, but the correction process generated another inaccurate statement due to insufficient verifier knowledge coverage.

These findings indicate that residual errors arise from both NLI discrimination and verifier knowledge coverage, particularly for temporal inaccuracies, subtle factual substitutions, uncommon facts, and complete fabrications.

Statistical analysis

The hallucination rate decreased from 15% with fixed-threshold NLI verification to 9% with the proposed statistical framework, while the complete framework reduced the rate from 28% for the baseline to 9% on TruthfulQA. These differences are reported as descriptive performance changes, and no formal statistical significance is claimed because the present evaluation does not include the statistical test results and effect-size estimates required to support such an inference^17,18.

Limitations

The verification performance of the framework is bounded by the capability of the underlying NLI model. DeBERTa-v3-large may struggle with fine-grained numerical comparisons, temporal reasoning, and claims requiring multi-hop inference across multiple facts. NLI model limitations were also associated with false negatives involving subtle factual substitutions, and systematic weaknesses in the NLI model may propagate into verification decisions.

The response and reference generators use the same underlying GPT-3.5 Turbo model. Although different prompting objectives and isolated execution contexts provide procedural independence, the two processes are not statistically independent and may retain correlated factual biases from their shared pretrained model.

Evaluator–verifier coupling represents an additional limitation. The final hallucination assessment uses the same NLI model that verifies claims within the proposed pipeline. This may introduce agreement between the verifier and evaluator and influence the measured improvement. The reported results should therefore be interpreted as performance under the defined NLI-based evaluation procedure rather than as fully independent evidence of hallucination reduction. An independent human assessment or a separately developed evaluation model would provide stronger validation.

The claim-decomposition stage was evaluated solely for its contribution to end-to-end verification and was not independently assessed against human-annotated claim boundaries. Consequently, the study does not provide a separate quantitative estimate of decomposition accuracy or its individual error propagation to downstream verification.

Evaluation was limited to TruthfulQA and FEVER and to English-language content. Therefore, the observed performance does not establish generalization to specialized domains, multilingual settings, or long-form generation. The optimal value of k may also vary across application domains because NLI confidence-score distributions depend on the nature of the generated content. Domain-specific calibration and broader evaluation are therefore required before extending the findings beyond the conditions evaluated in this study.

Finally, latency measurements are specific to the experimental configuration and may vary across execution environments. Because individual per-run latency measurements were not retained, variability and confidence intervals could not be estimated post hoc. Future evaluation should retain individual measurements and include broader statistical characterization of latency across execution environments.

Future work

Future studies should address the principal limitations identified in the present framework. Domain-aware threshold calibration is needed because the fixed value (k = 0.7) selected using TruthfulQA may not be optimal for specialized domains with different NLI confidence-score distributions. Domain-specific validation data could be used to calibrate the threshold and, where required, apply asymmetric penalties for false-negative and false-positive errors¹⁹.

Improved claim decomposition should include independent evaluation using human-annotated claim boundaries to quantify completeness, correctness, and downstream error propagation. Fine-tuned decomposition models and decomposition-confidence measures could further reduce errors arising from poorly segmented claims. Future comparative evaluations should also include lightweight decoding-based approaches such as DoLA under a unified experimental protocol.

Lightweight retrieval augmentation may help address complete fabrications, which remain difficult when the verifier lacks sufficient factual knowledge. A targeted retrieval fallback for low-confidence contradicted claims could provide external factual support without requiring retrieval for every claim²⁰.

A multilingual extension is also required because the current evaluation is limited to English-language benchmarks³. Future studies should evaluate multilingual NLI models and language-specific claim-decomposition and correction prompts on multilingual hallucination benchmarks.

Conclusion

This study presented a two-step claim-verification framework combining atomic claim decomposition, separately prompted factual reference generation, NLI verification, adaptive statistical thresholding, and selective correction. On TruthfulQA, the framework reduced the hallucination rate from 28% to 9% while increasing the mean latency by 160 ms relative to the baseline LLM. On FEVER, the hallucination rate decreased from 26% to 10%.

The proposed framework achieved competitive performance in factual verification without repeated full-response sampling or external retrieval. Adaptive thresholding and selective correction contributed to the observed performance while limiting additional processing overhead. However, the findings are restricted to the evaluated benchmarks and experimental conditions and remain subject to NLI-model dependence, evaluator–verifier coupling, claim-decomposition uncertainty, domain-specific threshold calibration, and latency variability. Broader, independent, domain-specific, and multilingual evaluation is therefore required before extending the findings to wider deployment settings.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
FEVER benchmark dataset	FEVER Shared Task (Thorne et al.)	https://fever.ai/dataset/fever.html	Publicly available fact-verification dataset of claim-evidence pairs used for evaluation
GPT-3.5 Turbo (large language B10+2:12:12	OpenAI	Model ID: gpt-3.5-turbo; https://developers.openai.com/api/docs/models/gpt-3.5-turbo	Used as both the response generator and the independent reference generator, accessed via the OpenAI API
NumPy	NumPy Developers (open source)	https://numpy.org/	Numerical computation library used for statistical calculations, including mean and standard deviation of NLI verification scores
Openai (Python API client library)	OpenAI	https://github.com/openai/openai-python	Python client library used to submit prompts to and retrieve completions from GPT-3.5 Turbo
pandas	pandas Development Team (open source)	https://pandas.pydata.org/	Tabular data-analysis library used for dataset preprocessing and results handling
Pretrained Natural Language Inference (NLI) classification model	Hugging Face model hub (model built on Microsoft's DeBERTa-v3-large architecture)	MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli	Used in inference mode, without fine-tuning, to classify each claim-reference pair as entailment, neutral, or contradiction
Python	Python Software Foundation	Version 3.9; https://www.python.org/downloads/release/python-390/	Core programming language used to implement the verification framework
SciPy	SciPy Developers (open source)	https://scipy.org/	Scientific computing library used to support statistical analysis of experimental results
Transformers (Hugging Face Transformers library)	Hugging Face	https://github.com/huggingface/transformers	Python library used to load and run the pretrained NLI model
TruthfulQA benchmark dataset	Originally released by Lin, Hilton, and Evans	https://github.com/sylinrl/TruthfulQA	Publicly available benchmark of 817 fact-oriented questions used for threshold tuning and evaluation
Workstation computer	Not specified in manuscript (please confirm manufacturer/model)	Intel Core i5 processor; 16 GB RAM; 64-bit operating system	Hardware used to execute all experiments under identical configurations
Note: All URLs listed above were checked and confirmed active as of the date of this table.