Enhancing Fairness in Large Language Models for Clinical Artificial Intelligence Applications Through Fine-Tuning and Prompting

Summary

Healthcare-specific large language models face ethical and technical challenges, as they, like other large language models, reflect the biases inherent in their training data. The protocol presented here verifies the suppression of four types of bias (gender, race, occupation, religion) using prompt variants across three open-source models.

Abstract

Large Language Models (LLMs) are increasingly being applied to sensitive domains such as medical care, which pose multiple technical and ethical challenges. The most immediate issues include biases built into these models, which are trained on large datasets that may reflect societal prejudices. Such biases can yield outputs that are unfair, ungeneralizable, or even harmful, rendering them clinically inapplicable in the real world and noncompliant with ethical and regulatory constraints. We examine how well bias suppression works when fine-tuned models are prompted across four critical categories (gender, race, profession, and religion). We manually curate a diverse inference dataset and create twelve prompt variants -- six of which are debiased -- to test the outputs of three open-source LLMs: Llama2-7B, Mistral-7B, and Dolly-7B. The fairness and interpretability of the outputs are evaluated using a bias-scoring metric, where lower scores indicate better fairness and interpretability. We also note that debiased prompts reduce bias, and fine-tuning the model performs even better. The results emphasize the critical importance of timely action, model robustness, and ongoing ethical scrutiny for trustworthy and fair LLM deployment in real-world settings, such as medical imaging, and the maintenance of Electronic Health Records (EHR).

Introduction

Large Language Models have prodigious reverberation as tools to support diverse tasks, including decision-making^1,2, text generation³, text translation⁴, customer sentiment analysis⁵, question answering⁶, and generating clinical reports⁷. Several instances have occurred in recent times in which applications have used LLMs to generate content that harms socially disadvantaged individuals or groups^8,9,10

Protocol

All the experiments are conducted on the Google Colab platform (A100 with 40 GB RAM). For performing experiments, API keys (Model cards) of Llama2-7B, Mistral-7B, and Dolly-7B were collected from Hugging Face.

The protocol is divided into three parts. First, build a dataset that serves as a basis for evaluating social bias in LLMs. Then design twelve distinct prompt variants, aligned with the work done by Kamruzzaman and Kim³⁸ to investigate the presence of social bias in the inferences generated by LLMs. Then, fine-tune the LLMs on a dataset of unbiased sentences related to race, religion, gender, and profession, an....

Results

Using the bias score defined in Equation 1, we systematically answer our research questions and evaluate societal biases in LLMs across four social dimensions. The statistically significant reductions of bias scores observed provide empirical validation of the proposed protocol for bias reduction in LLMs for high-stakes tasks. To assess the effectiveness of NeutralSet, i.e., the curated inference dataset, we query all three vanilla LLMs on both StereoSet and NeutralSet through the baseline standard prompt and calculate t.......

Discussion

In this work, we present a scalable protocol for reducing social biases in LLMs that leverages Llama2-7B¹³, Mistral-7B¹⁴, and Dolly-7B¹⁵. This approach combines HP2 prompt engineering, debiasing prompt modifications, and targeted fine-tuning to develop a protocol for continued reduction of bias in LLM-generated outputs across the four major societal dimensions of gender, race, profession, and religion. While evaluating vanilla models with basic and d.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Google Colab	Google	https://colab.research.google.com/	used for running the experiments
Mendeley Desktop	Mendeley	https://www.mendeley.com/	used for managing citations and references.