Fine-Tuning Large Language Models Using Entity Hallucination Index for Text Summarization

Summary

We propose a reinforcement-learning-based fine-tuning approach for large language models that uses the Enti Hallucination Index (EHI) as a reward signal to reduce entity-level hallucination in text summarization. Experiments on meeting transcripts show that this method improves entity faithfulness and accuracy.

Abstract

Recent advancements in large language models (LLMs) have led to notable improvements in abstractive summarization quality. However, hallucination - especially entity-level hallucination where non-existent or incorrect entities are introduced - remains a critical challenge. In this work, we propose a reward-driven fine-tuning framework for summarization models using the Entity Hallucination Index (EHI) as a guiding metric. The methodology here begins with generating initial summaries from pre-trained models such as Flan-T5, DistilBART, and Mistral (or other popular LLM) on structured transcript datasets, XSUM. We compute EHI by extracting named entities from both generated summaries and gold references, evaluating precision, and penalizing fabricated entities. The fine-tuning process is guided by reinforcement learning, where EHI serves as the reward signal. We adopt a REINFORCE-style update mechanism to optimize the summarization model towards maximizing entity faithfulness. Experiments demonstrate that models fine-tuned with EHI achieve lower hallucination rates without compromising informativeness. Furthermore, we show that EHI-guided models generalize better on out-of-domain summarization tasks, suggesting enhanced robustness. The approach here offers a practical direction for improving factuality in summarization, emphasizing the critical role of accurate entity representation.

Introduction

Abstractive summarization models, powered by large language models (LLMs), have achieved impressive results across various domains. However, a persistent challenge remains hallucination, where generated summaries include incorrect or fabricated information not grounded in the source input^1,2. In high-stakes applications such as meeting summarization, medical reporting, or financial documentation, hallucinations, particularly involving named entities, can significantly undermine trustworthiness and utility³.

Researchers conducted a large-scale human evaluation and found that summarizers frequently produce hallucinations -- content that is not supported by the source document -- and that these hallucinations correlate with repetition and incoherence¹. Another study showed that models with higher abstractiveness tend to be less faithful; their study reported that outputs with more abstraction had lower factual accuracy and more unfaithful sentences². The FRANK benchmark reported that roughly 30% of summaries contain factual inconsistencies, and another popular study noted similar error rates, arguing that high hallucination levels render summaries virtually useless^1,2. Hallucinations can be categorized as intrinsic (contradicting the source) or extrinsic (not verifiable from the source). A tutorial on hallucinations in abstractive summarization explains that intrinsic hallucinations occur when generated content contradicts the input (e.g., misreporting an approved vaccine as being rejected), while extrinsic hallucinations add unverifiable facts, such as claiming clinical trials for a vaccine that are not mentioned⁴. It also reported that state-of-the-art summarizers produce hallucinated content in roughly 25% of their outputs when evaluated with ROUGE, BLEU, and METEOR, and cautions that hallucinations can be harmful in domains such as health, law, or cybersecurity.

Traditional n-gram metrics (ROUGE⁵, BLEU, METEOR) correlate poorly with human judgments of factuality, prompting the development of reference-based and reference-free metrics⁶. QA-based metrics such as FEQA and QuestEval evaluate summaries by asking whether question-answer pairs derived from the summary can be answered using the source text. FEQA correlates more strongly with human judgments and shows that more abstractive summaries tend to be less faithful, whereas QuestEval substantially improves correlation with human judgments across consistency, coherence, fluency, and relevance. QAFactEval refines this approach by selecting noun-phrase answers and generating questions before answering them; this strategy improves correlation with human assessments on the SummaC benchmark. Information-extraction (IE) metrics represent knowledge as subject-relation-object tuples, but they are vulnerable to errors in the extraction process. Natural language inference (NLI) metrics -- such as FactCC and SummaC -- classify summary sentences as entailed or contradicted by the source⁴. FactCC uses weakly supervised data to train a classifier that detects factual consistency and highlights inconsistent spans². SummaC applies NLI models at the appropriate granularity and yields strong factuality estimates, but the FRANK benchmark notes that these metrics still treat factuality as binary and lack a unified framework^1,7,8. Human evaluation dimensions proposed by researchers fluency, coherence, relevance and consistency are widely adopted⁹. Another evaluation study emphasizes that consistency is the most objective dimension because it simply checks whether the summary is entailed by the source; it points out that up to 92% of XSum summaries contain faithfulness errors^9,10. However, human evaluation is time-consuming and expensive, so automatic metrics are essential for scalable evaluation⁸. Current metrics often aggregate intrinsic and extrinsic hallucinations into a single score, hindering error diagnosis⁴.

Since maximum-likelihood training does not directly optimize for summary quality, reinforcement learning (RL) has been applied to improve relevance and factuality. Pasunuru and Bansal introduced a multi-reward RL framework that combines saliency (ROUGE) and entailment rewards and demonstrated state-of-the-art performance¹¹. RL was applied for extractive summarization using the REINFORCE algorithm; their model maximizes ROUGE scores and yields more informative summaries than classifier-based approaches, as shown by human question-answering evaluations¹². Multi-task models further incorporate question generation and entailment generation to ensure coverage and coherence¹². A comprehensive 2024 survey on abstractive summarization highlights factual consistency as a fundamental challenge and advocates RL to encourage truthful summarization. The survey suggests rewarding factually consistent summaries and incorporating external knowledge sources and attribute control to enhance accuracy¹³. RL from human feedback (RLHF) extends this idea by training a reward model on human preferences and fine-tuning the summarizer to maximize this learned reward⁹. Reinforcement learning from human feedback (RLHF) has gained traction for aligning model outputs with desired qualities, such as factuality and helpfulness^4,14.^{While multi}-reward RL and RLHF align models with human notions of fidelity, they rely on coarse metrics like ROUGE or binary factuality. In contrast, the work detailed here proposes using a fine-grained entity-level hallucination index as the reward, which is empirical and computationally less expensive without human feedback.

Despite numerous metrics, several limitations persist. N-gram metrics ignore semantic correctness, QA-based and IE-based metrics propagate errors from question generation and extraction models, and NLI metrics reduce factuality to binary entailment decisions^1,4,6. In the research work, it was emphasized that existing model-based metrics struggle to identify hallucinations at a fine granularity because they operate at the sentence or document level and cannot isolate specific entities causing errors⁴. Human evaluations remain the gold standard but are costly and subjective¹⁵. Moreover, existing metrics do not distinguish between intrinsic and extrinsic hallucinations or capture the over and under-focus of relations¹. Large language models can hallucinate because they predict text based on statistical likelihood rather than factual verification. Models generate confident but incorrect outputs due to limitations in training data, probabilistic generation and lack of fact-checking, and overconfidence and biases can cause them to state false information with high certainty; the guide advocates retrieval-augmented generation and long-term memory to ground models in external data¹⁵. However, these techniques focus on retrieval-based tasks; summarization still requires metrics that identify and penalize hallucinated entities. Prior work has recognized the importance of entity-level evaluation for factuality. Metrics like FEQA and QuestEval attempt to assess factual consistency using question answering techniques^16,17. However, these methods often require reference summaries or external QA systems, limiting scalability. Our work builds on this line by employing the Entity Hallucination Index (EHI), a lightweight empirical metric focusing specifically on named entity faithfulness by categorizing the entities into five types of hallucination factors as depicted in Figure 1.

Evidence indicates that hallucination is pervasive, evaluation metrics are imperfect, and RL techniques lack fine-grained rewards. Human studies show that summarization models hallucinate in a significant fraction of cases, often swapping entities or fabricating details⁹. Existing metrics either emphasize n-gram overlap or treat factuality as binary, and current RL methods optimize ROUGE or coarse factuality signals. There is a gap for entity-level metrics that measure when a summary mentions entities that are not present in the source (Negative hallucination), omits key entities (Lost focus hallucination), and other scenarios. The proposed fine-tuning method over Entity Hallucination Index (EHI) fills this gap by quantifying hallucinations at the entity level and providing a structured reward for reinforcement learning. By integrating EHI into RLHF, we aim to fine-tune large language models to minimize entity-level hallucination and produce summaries that are both fluent and factually faithful.

In this work, we propose a novel fine-tuning approach that leverages the Entity Hallucination Index (EHI) as a reward signal to reduce hallucinations at the entity level. EHI quantifies the correctness and grounding of entities by comparing extracted entities from model outputs to those present in the input document, enabling a reduction in dependency on references¹⁸. The method described here biases the model towards producing more entity-faithful summaries without requiring human factuality annotations.

In summary, factuality-aware reward models have been used for fine-tuning^19,20. Unlike previous approaches that rely on coarse factuality rewards or human-labeled datasets, we propose fine-tuning using EHI as a direct, automatic reward signal, thereby promoting entity-grounded summarization without additional supervision. The novel contributions of this study are as follows: (i) the study introduces an EHI-guided fine-tuning framework that improves entity faithfulness in abstractive summarization, (ii) the study presents a scalable reinforcement learning pipeline that does not depend on human-labeled factuality datasets, (iii) the study demonstrates empirical improvements in EHI scores across meeting transcript datasets, XSUM news dataset and conduct qualitative analysis to highlight reductions in hallucination, (iv) the study shares a reproducible Colab-based implementation to facilitate further research on hallucination-aware summarization²¹.

This study assesses the hypothesis that optimizing language models with fine-grained entity-focused rewards like EHI provides an effective pathway for enhancing faithfulness and reliability in summarization tasks with minimal computation. The approach contributes to the growing field of parameter-efficient fine-tuning for faithfulness in summarization by: (i) demonstrating effectiveness across both encoder-decoder (DistilBART, FlanT5) and decoder-only (Mistral) architectures, (ii) providing a framework that can be adapted to different model types, domains, and scales, and (iii) offering computational efficiency compared to full retraining.

Protocol

The objective of this study is to fine-tune a language model (LM) to generate summaries with reduced entity hallucination. Hallucinations occur when a model "confidently produces incorrect or irrelevant output" because it generates text according to statistical likelihood rather than factual verification. This is achieved by designing a reward function based on the Entity Hallucination Index (EHI) and applying reinforcement learning to maximize it.

Problem formulation
Given an input document X_i, the goal is to generate a summary Y_i such that entities mentioned in Y_i are grounded in X_i. Traditional maximum likelihood training does not explicitly optimize factual consistency. These training methods do not penalize the inclusion of fabricated entities. Therefore, a reward-driven fine-tuning strategy is introduced where the reward is proportional to entity faithfulness, measured using EHI.

Data set and method
Two corpora are used: firstly, a dialogue transcription dataset of multi-turn meeting transcripts and abstractive gold summaries, and secondly, the XSUM news summarization benchmark^22,23. Each dataset was cleaned, flattened, and split into 80% train, 10% validation, and 10% test partitions. Dialogues are rich in informal language and pronouns, whereas XSUM contains concise news articles; the combination allows us to evaluate both in-domain and out-of-domain generalization.

Entity extraction is performed on both source documents and summaries for computing the Entity Hallucination Index (EHI). During fine-tuning, this study exclusively utilize the training split, while validation EHI scores are monitored to select the best checkpoint. Final evaluations are reported on the held-out test set, which remains zero-shot before and after the fine-tuning of the models.

In summary, firstly, transcripts are organized into a flattened format and split the data. The pre-trained model generates baseline summaries. Secondly, EHI is computed using transformer-based NER and the five hallucination factors. During fine-tuning, the model is updated with the EHI reward via LoRa adapters. Finally, summaries are generated using the fine-tuned model and evaluated using EHI, Entity F1, and other metrics (Figure 2). All experiments are reproducible through the provided code and configuration files. Figure 3 shows the stepwise process flow for dataset preparation, baseline summarisation, EHI computation, fine-tuning, and evaluation.

Baseline summarization
Three pre-trained LLMs are chosen, and their baseline summarizations are captured: Flan-T5, Mistral (Nous-Hermes-2-Mistral-7B-DPO), and DistilBART^24,25,26. Each model was run in zero-shot mode to produce an initial summary (Y_i) for every input document(x_i). Beam search with a beam width of 4 and a length penalty of 1.0 was used to encourage fluent yet concise summaries. These baseline summaries served to estimate the prevalence of hallucination and provided starting points for fine-tuning.

Entity extraction and EHI computation
Accurately computing the Entity Hallucination Index (EHI) requires robust NER. Initially, Spacy was used for NER operations. It is understood that transformer-based NER models are more accurate, but Spacy was the initial choice for the following reasons: (1) Transformer-based NER models themselves have proven to hallucinate than Spacy. (2) Spacy being statistical brings in computational efficiency in terms of implementation cost and execution time. (3) It also helps us prove that EHI is robust in reducing hallucination even with a weaker NER model. However, given the fact that the experiments are done on established datasets and spaCy's en_core_web_sm model is brittle on colloquial transcripts¹³. Spacy is replaced with a transformer-based NER model (dslim/bert-base-NER), which is a BERT model fine-tuned on the CoNLL-2003 dataset. BERT-based NER systems have been shown to outperform spaCy models on domain-specific tasks-for example, an Aviation-BERT-NER model reached an F1 of 94.78% while identifying 17 entities compared with 93.73% for spaCy NER that recognized seven entities²⁷. The BERT NER model was configured via Hugging Face Transformers²⁸ and performed case-insensitive matching. To capture pronouns and surface-form variations, this method additionally applied simple rules that map "Mr. Smith" and "Smith" to the same canonical form; however, full coreference resolution remains future work. The comparative analysis of the NER models on a sample of 200 records from XSUM dataset can be inferred from Table 1.

The Entity Hallucination Index (EHI) is calculated as:

where PH, EF, OF, NH, LF represent scores corresponding to Positive Hallucination (PH), Extractiveness Factor (EF), Negative Hallucination (NH), Overfocused Entities (OF), and Lost Focus (LF), respectively, for article i.

Details of hallucination factors:

Positive Hallucination (PH): Measures of newly introduced entities that are factually correct and beneficial.

Extractiveness Factor (EF): Measures entities accurately extracted from the input document into the summary.

Negative Hallucination (NH): Captures hallucinated entities that are incorrect or not grounded in the input.

Over Focused (OF): Penalizes summaries that overly focus on a narrow subset of entities, missing diversity.

Lost Focus (LF): Penalizes summaries that omit important entities present in the input.

Computation of above factors is detailed in Figure 4 with a sample computation for illustration purposes. Considering an example of Input Document Entities (I):" John"," AI"," conference" Reference Summary Entities (R):" John"," AI" Generated Summary Entities (G):" John"," AI"," technology". Higher values of EHI indicate better entity faithfulness, rewarding summaries that are both extractive and positively hallucinated (introducing useful but faithful entities), while penalizing ungrounded, excessive, or missing entity²⁰. These factors were normalized by the total number of detected entities to produce an EHI score between 0 and 1. EHI validation was monitored during training to select the best checkpoint. PH and EF reward beneficial new entities and correct extraction, whereas OF, NH, and LF penalize repetition, fabrication, and omission. A perfect score of 1.0 means that all entities in the summary are grounded or beneficially hallucinated, while a score of 0 indicates that no named entity in the summary appears in the source.

Fine - tuning procedure with RL
The detailed hyperparameter configuration for fine-tuning are provided separately in Table 2, Table 3, and Table 4, which depict optimizer types, learning rates, batch sizes, hardware specifications, and other configuration details for Mistral-7B, DistilBart, and Flan-T5, respectively. The objective here is to fine-tune the model parameters θ by maximizing the expected Entity Hallucination Index (EHI) reward across generated summaries. Formally, for an input document X_i, a generated summary Y_i, and model parameters θ the expected reward objective is defined as:

     (2)

where EHI (y, x) denotes the Entity Hallucination Index computed between the generated summary Y_i and the input X_i.

Following the REINFORCE algorithm²⁵, the gradient of this objective is estimated as:

      (3)

In practice, summaries were sampled from the models, their corresponding EHI scores were computed, and policy gradient updates were applied to encourage higher-reward outputs. To enable parameter-efficient fine-tuning, this study used Low-Rank Adaptation (LoRA). Only LoRA adapters were updated while the base model weights remained frozen. Fine-tuning was performed on an A100 GPU with a small learning rate (e.g., 5 × 10^-6), batch size of 4 and gradient accumulation over 8 steps with Adam optimizer²⁹. Summaries were regenerated every 500 updates to refresh reward estimates, and training continued until validation EHI converged. The REINFORCE baseline was set to the running average of recent rewards to reduce variance. In addition to EHI, Entity F1², ROUGE-1/2/L scores were logged to monitor general quality. Summaries are regenerated periodically to reflect model improvements during fine-tuning.

Evaluation metrics
The outputs were evaluated using Informativeness metrics like Rouge-1, Rouge-2⁵. Fluency Metric like Rouge LCS⁵. Entity overlap metric Entity F1, and hallucination metrics like EHI, FactCC, and QAGS^2,6,18,30. FactCC uses a classifier trained on synthetic contradiction data to label sentences as entailed or contradicted by the source. The factCC gives two scores; the scores were captured where the label is VALID. On the other hand, QAGS uses a question-and-answer method that runs on an LLM. Light model T5 small was used for generating questions on the text³¹. Roberta_base_Squad2 was used for generating answers over the output text³². These models were chosen for their convenient computation cost for execution.

Results

The experiments here focus on quantitative evidence for the impact that EHI-guided fine-tuning makes in reducing entity-level hallucinations across multiple model architectures and datasets. The results demonstrate consistent improvements in hallucination metrics while maintaining factual consistency and preserving question-answering capabilities.

Dataset performances
Dialogue dataset performance:
Table 5 presents the results for hallu...

Discussion

QA-based metrics (e.g., FEQA/QuestEval/QAFactEval) and NLI-based metrics (e.g., FactCC/SummaC) require auxiliary QA/entailment components and often provide sentence-level judgments. In contrast, EHI is a lightweight, reference-free, entity-level score that (i) directly targets the primary failure mode we observe -- entity hallucination; (ii) is model-agnostic and fast to compute; and (iii) serves as a dense reward for RL fine-tuning. Empirically, EHI complements QA/NLI metrics by pinpointing which entities drive factuali...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Colab NVIDIA A100 GPU	NVIDIA	N/A	Google Colaboratory Pro
DistilBART (Encoder–Decoder)	Hugging Face	https://huggingface.co/sshleifer/distilbart-cnn-12-6
dslim/bert-base-NER	Hugging Face (dslim)	https://huggingface.co/dslim/bert-base-NER	Fine-tuned BERT base for English NER
Evaluation Models  QAGS (QA-based)	Salesforce Research	N/A	Factuality classifier
Evaluation Models (Factuality) FactCC	Google Research	N/A	Factuality classifier
Flan-T5 (Encoder–Decoder)	Google	https://huggingface.co/docs/transformers/main/en/model_doc/flan-t5	open-source, text-to-text, large language model
Hugging Face Transformers	Hugging Face, Inc.	https://huggingface.co/docs/transformers/index	Model loading & fine-tuning
Nous-Hermes-2-Mistral-7B-DPO	Mistral	https://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO	7 billion parameter Language model
Programming Language Python 3.10 (Colab runtime)	Python Software Foundation	Version 3.10	Core implementation
Streamlit	Open Source	https://streamlit.io/	Factuality classifier
XLSUM dataset	BUET	https://github.com/csebuetnlp/xl-sum
XSUM dataset	University of Edinburgh	https://github.com/EdinburghNLP/XSum