Interoperable Web Platform Based on Large Language Models for Medical Data Analysis: A Protocol for Clinical Decision Support

Summary

This protocol describes the implementation of an interoperable web platform that integrates FHIR-based clinical data with Retrieval-Augmented Generation and multi-agent large language models for clinical decision support. The workflow enables reproducible deployment, standardized data integration, and evaluation of artificial intelligence (AI)-assisted medical data analysis.

Abstract

Clinical decision-making is frequently hindered by fragmented electronic health records and limited interoperability among heterogeneous healthcare information systems. This article presents a step-by-step protocol for implementing an interoperable web platform that integrates clinical data via the Fast Healthcare Interoperability Resources (FHIR) standard, along with Retrieval-Augmented Generation (RAG), Large Language Models (LLMs), and a multi-agent clinical reasoning framework to support medical data analysis and clinical decision support. The protocol describes the complete workflow, including computational environment configuration, clinical dataset preprocessing, FHIR-based data integration, vector database construction, retrieval configuration, prompt engineering, multi-agent orchestration, and system evaluation. Representative results demonstrate the platform's ability to generate clinically relevant and contextually consistent responses while improving semantic interoperability across heterogeneous data sources. System performance was evaluated using complementary quantitative and semantic metrics, including BLEU, ROUGE, BERTScore, and cosine similarity. A qualitative assessment was conducted using publicly available, fully anonymized benchmark healthcare datasets to evaluate the reproducibility of the proposed methodological workflow. The proposed architecture combines standardized healthcare interoperability with retrieval-enhanced language models to improve contextual reasoning, reduce hallucination, and support reproducible AI-assisted clinical workflows. This protocol provides a scalable and reproducible framework for researchers and developers seeking to implement interoperable, privacy-aware, and intelligent healthcare systems for clinical decision support, medical data analysis, and future translational research.

Introduction

Health information is routinely generated across hospitals, diagnostic centers, laboratories, and outpatient clinics, often remaining distributed among heterogeneous information systems. This fragmentation limits interoperability and restricts timely access to comprehensive patient records, creating persistent challenges for clinical care, data integration, and healthcare management^1,2,3,4.

The consequences of this fragmentation become particularly evident in clinical workflows that depend on timely access to patient information. When healthcare professionals cannot retrieve complete and integrated medical records, clinical assessment becomes more difficult, increasing the risk of incomplete decision-making and reducing the efficiency of care delivery, particularly in emergency settings^5,6.

Recent developments in Artificial Intelligence (AI), particularly Large Language Models (LLMs), have expanded the range of computational approaches available for healthcare applications. These models have been investigated in tasks such as diagnostic assistance, clinical triage, and decision support. For example, Jahan et al.⁵ evaluated the use of LLMs in biomedical tasks, while Taylor et al.⁷ investigated fine-tuned models for digital mental health screening, reporting encouraging results in specialized clinical settings.

The application of LLMs has also expanded to more specialized clinical domains. Song et al. ⁴⁹explored their use in pneumoconiosis diagnosis, whereas Chien et al.⁸ applied these models to analyze workload patterns among informal caregivers. In ophthalmology, Xue et al.⁹ proposed a question-answering system for glaucoma diagnosis, while Tan et al.³ and Wu et al.¹⁰ reported the use of LLMs in hybrid diagnostic systems and traditional Chinese medicine consultations.

The use of LLMs is not limited to direct clinical applications. Zhang et al.¹¹ investigated their application to emotion recognition in mental health scenarios, whereas Sarzaeim et al.¹² explored intelligent policing systems that may also contribute to public health-related analyses. These studies illustrate the broad applicability of LLM-based approaches across different healthcare-related domains.

Although LLMs have expanded the possibilities for healthcare applications, their integration into clinical environments remains associated with important technical, organizational, and regulatory challenges. Practical deployment requires appropriate computational infrastructure, effective data governance, and compliance with regulatory frameworks such as the General Data Protection Regulation (GDPR) and the Brazilian General Data Protection Law (LGPD). These frameworks establish requirements for the secure and ethical processing of sensitive health information while addressing issues related to privacy, reliability, and algorithmic bias in AI-assisted healthcare systems^13,14 .

Interoperability across heterogeneous healthcare data sources, including Electronic Health Records (EHRs), medical imaging systems, and Internet of Things (IoT) devices, continues to represent a major technical challenge for the deployment of AI-assisted healthcare solutions. In this context, standardized interoperability frameworks such as HL7 FHIR provide a structured mechanism for exchanging clinical information across different systems while preserving semantic consistency and supporting scalable integration.

This work presents an interoperable web platform that integrates Large Language Models with healthcare interoperability standards to support medical data analysis in heterogeneous clinical environments. The proposed architecture combines HL7 FHIR-based data integration with Retrieval-Augmented Generation (RAG) and a multi-agent processing framework to provide context-aware AI-assisted analysis while maintaining alignment with applicable data protection requirements, including the Brazilian General Data Protection Law (LGPD).

This protocol describes an interoperable architecture for integrating heterogeneous clinical data using established healthcare interoperability standards. It also details the implementation of a multi-agent processing pipeline based on large and small language models (LLMs and SLMs), together with an evaluation framework that combines quantitative metrics and qualitative analyses to assess the behavior of the proposed platform.

Protocol

This study did not involve the recruitment of human participants, access to identifiable patient records, or experiments involving animals. The protocol was developed and evaluated exclusively using publicly available and fully anonymized datasets for methodological validation. No personal health information was accessed or processed. Therefore, approval from an Institutional Review Board (IRB) or Research Ethics Committee was not required. The protocol was developed in accordance with applicable data protection principles, including the Brazilian General Data Protection Law (LGPD), to support future applications involving clinical data.

Dataset selection and preprocessing

The proposed protocol was evaluated using publicly available and fully anonymized clinical datasets comprising structured Electronic Health Records (EHRs), clinical question-answering data, and medical imaging datasets for methodological validation. Before integration into the platform, the datasets underwent standardized preprocessing procedures, including data normalization, removal of inconsistent or incomplete records, mapping to HL7 FHIR resources, text cleaning, segmentation into retrieval chunks, and embedding generation for vector indexing. These preprocessing steps ensured semantic consistency across heterogeneous data sources, facilitating interoperability and enabling the reproducibility of the proposed workflow while maintaining compliance with applicable data privacy principles.

The datasets were obtained from publicly available benchmark repositories commonly used in artificial intelligence and digital health research. They were selected to represent heterogeneous clinical information, including structured Electronic Health Records (EHRs), unstructured clinical narratives, clinical question-answering tasks, and medical imaging metadata. Rather than evaluating a specific clinical cohort, the protocol focuses on demonstrating a reproducible implementation workflow that can be adapted to different healthcare datasets. The diversity of these benchmark datasets enables validation of the interoperability pipeline, Retrieval-Augmented Generation (RAG), and multi-agent reasoning framework across multiple clinical data modalities.

Experimental environment configuration

The experimental environment was configured to evaluate the interoperable platform under controlled and reproducible conditions. The architecture comprises data ingestion modules, interoperability layers, large language models (LLMs), and evaluation components organized into a single processing pipeline for medical data analysis. Figure 1 illustrates the complete workflow, from clinical data ingestion to the generation of diagnostic outputs.

Figure 1: Overall system workflow illustrating the processing pipeline from raw clinical data to disease status output. The process begins with Electronic Health Records (EHR) ingestion, followed by data filtering and preprocessing to extract disease-sensitive information. A structured prompt design stage integrates expert knowledge, disease definitions, and hyperparameters, enabling effective interaction with the Large Language Model (LLM). The LLM performs text inference to generate context-aware responses, which are subsequently evaluated through clinical rules to determine the final disease status. The workflow highlights the integration of data preprocessing, knowledge-driven prompting, and AI-based inference to support clinical decision-making. Please click here to view a larger version of this figure.

Healthcare interoperability architecture

The backend infrastructure adopts a modular architecture based on RESTful APIs to support communication among the platform components (Figure 2). This architecture accommodates heterogeneous clinical information, including structured Electronic Health Records (EHRs), physician notes, and metadata derived from medical imaging systems. Because these data originate from multiple sources and formats, interoperability is achieved through standardized data models, particularly the Fast Healthcare Interoperability Resources (FHIR) framework^15,16,17.^.The adoption of FHIR supports structured information exchange while preserving scalability and flexibility in distributed healthcare environments. HL7-based communication mechanisms were also incorporated to facilitate integration with legacy clinical systems, which remain widely used in healthcare institutions^16,17.

Figure 2: System architecture of the proposed interoperable platform. The web interface communicates with the backend through a Flask API using HTTP POST/GET requests. The API manages routing, query processing, and interaction with both structured and unstructured data sources. A MySQL database stores structured clinical data, while a FAISS-based vector store supports similarity search for retrieval operations. The LLaMA-based pipeline processes textual inputs and generates responses using vector representations, enabling Retrieval-Augmented Generation (RAG). The architecture highlights the integration of web services, database management, vector retrieval, and large language model inference in a unified system. Please click here to view a larger version of this figure.

Multi-agent workflow configuration

The multi-agent architecture is organized into specialized functional agents responsible for distinct stages of the workflow. A preprocessing agent performs data normalization and FHIR mapping, followed by a retrieval agent responsible for semantic search within the vector database. A reasoning agent integrates the retrieved context with the LLM to generate responses, while a validation agent verifies output consistency and formatting before the final response is returned. Agent coordination follows a sequential orchestration strategy in which the output of each agent serves as the input for the subsequent stage, ensuring a reproducible and modular implementation.

Clinical data integration

The data integration layer aggregates information from multiple clinical sources and prepares it for downstream processing. Preprocessing includes data normalization, tokenization, and entity alignment to improve semantic consistency across heterogeneous datasets. Because clinical information varies in structure and quality, these operations help reduce noise and facilitate interaction with AI models. Structured mapping strategies were also applied to harmonize different data formats and maintain compatibility with the processing pipeline, as illustrated in Figure 3^15,16,17.

Figure 3: Detailed example of the multi-agent clinical reasoning process. The figure illustrates how a clinical query is analyzed through multiple stages, including complexity assessment, specialist recruitment, collaborative discussion, and final decision-making. This process demonstrates the system’s ability to dynamically adapt reasoning strategies based on query complexity, improving both efficiency and diagnostic accuracy in clinical decision support scenarios. Please click here to view a larger version of this figure.

Configure the Retrieval-Augmented Generation pipeline

Retrieval-Augmented Generation (RAG) is incorporated to provide context-aware analysis by combining information retrieval with the generative capabilities of large language models. User queries are converted into vector representations using embedding models and matched against the vector database using semantic similarity search to retrieve the most relevant contextual passages. The retrieved documents are then combined with the original query before inference by the LLM. This strategy helps provide contextual information during response generation and has been associated with improved factual consistency and reduced hallucination in knowledge-intensive applications, including healthcare^18,19. Figure 1 and Figure 3 illustrate the overall retrieval workflow and the corresponding reasoning process.

For each user request, the final prompt is dynamically constructed by combining the original query with the most relevant contextual passages retrieved from the vector database. The retrieved information is incorporated as contextual evidence before inference, allowing the language model to generate responses grounded in the retrieved healthcare knowledge while preserving semantic consistency and reducing unsupported generations.

Prompt engineering and multi-agent reasoning

The protocol incorporates structured prompting strategies to improve contextual interpretation during response generation. These prompting techniques, together with advanced inference mechanisms, help guide the reasoning process in complex clinical scenarios and are consistent with recent developments reported in the literature²⁰.

The architecture includes a multi-agent framework composed of specialized modules that perform distinct functions within the processing pipeline, including data validation, context filtering, clinical reasoning support, and output verification. The modular organization allows tasks to be executed sequentially or in parallel, providing flexibility for different processing requirements (Figure 4). Distributing these activities across multiple agents reduces reliance on a single language model and supports a more robust processing workflow. This architectural strategy is consistent with recent developments in distributed artificial intelligence and intelligent system design^21,22.

Figure 4: Multi-agent decision framework for clinical reasoning. The process begins with a user query, which is evaluated by an agent checker responsible for assessing query complexity. For complex cases, the system dynamically recruits a multi-disciplinary team (MDT) of specialized agents that engage in iterative discussion rounds to analyze the problem and synthesize knowledge before producing a final decision. For simpler cases, the query is handled by a primary care clinician (PCC) agent, enabling faster response generation. This adaptive architecture balances efficiency and analytical depth, improving decision quality and system scalability in healthcare applications. Please click here to view a larger version of this figure.

Performance evaluation

System performance was evaluated using complementary metrics that capture both linguistic quality and semantic consistency of the generated outputs. BLEU was applied to measure syntactic similarity based on n-gram overlap²³, whereas ROUGE assessed recall and content coverage, particularly in summarization and information extraction tasks²⁴. Semantic similarity was evaluated with BERTScore, which uses contextual embeddings derived from transformer-based models to compare generated and reference texts²⁵. Additional analyses included perplexity and cosine similarity to examine model confidence and semantic coherence, respectively^26,27.

The selected evaluation metrics provide complementary perspectives on system performance by combining lexical and semantic analyses. This combination is particularly relevant in healthcare applications, where contextual interpretation is as important as lexical similarity. The platform was evaluated in a controlled computational environment that supports both cloud-based and local deployment. Local execution of LLMs was included as an option to support data privacy requirements and reduce reliance on external services when processing sensitive clinical information. This deployment strategy is compatible with data protection frameworks and can be adapted to different operational environments^4.

Results

System performance was assessed using complementary quantitative metrics together with qualitative analysis to examine both linguistic accuracy and semantic consistency of the generated outputs. This evaluation strategy combines lexical and semantic measures to provide a broader characterization of model behavior in healthcare-related language generation tasks.

Table 1 presents the quantitative results obtained with BLEU, ROUGE, and BERTScore. These metrics were selected becau...

Discussion

The evaluation presented in this study represents a methodological validation performed using publicly available and fully anonymized benchmark healthcare datasets. No prospective clinical validation involving healthcare professionals or patient recruitment was conducted, as the objective of this work is to demonstrate a reproducible implementation protocol rather than clinical effectiveness.

The findings of this study suggest that combining healthcare interoperability standards with large lan...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
FAISS	Meta Platforms Inc.	Version 1.8.0	Vector database used for similarity search in the Retrieval-Augmented Generation (RAG) pipeline.
FHIR Standard	HL7 International	Release 4 (R4)	Healthcare interoperability standard adopted for structured clinical data exchange.
Flask	Pallets Projects	Version 3.0.3	Python web framework used to implement the RESTful backend API.
HL7 Standard	HL7 International	Version 2.x	Legacy messaging protocol used for interoperability with hospital information systems.
HTTP REST API	Custom Implementation	N/A	RESTful communication layer between frontend, backend, database and AI services.
LangChain	LangChain Inc.	Version 0.3.x	Framework used to integrate LLMs, prompt engineering and Retrieval-Augmented Generation workflows.
LangGraph	LangChain Inc.	Version 0.2.x	Framework used for orchestrating the multi-agent clinical reasoning workflow.
LLaMA 3.1 8B Instruct	Meta Platforms Inc.	Version 3.1	Large Language Model employed for clinical reasoning and natural language generation.
MySQL Community Server	Oracle Corporation	Version 8.0	Relational database used to store structured clinical data.
Ollama	Ollama Inc.	Version 0.9.x	Local inference engine used for executing Large Language Models while preserving patient privacy.
Python	Python Software Foundation	Version 3.11	Programming language used for implementing the complete platform.
PyTorch	Linux Foundation	Version 2.4	Deep learning framework used for Large Language Model inference.
Retrieval-Augmented Generation (RAG) Pipeline	Custom Implementation	N/A	AI framework combining semantic retrieval with LLM inference for context-aware response generation.
Sentence Transformers	Hugging Face Inc.	all-MiniLM-L6-v2	Embedding model used to generate dense vector representations for semantic document retrieval.
Ubuntu Linux	Canonical Ltd.	Version 24.04 LTS	Operating system used for development and experimental evaluation.
Visual Studio Code	Microsoft Corporation	Version 1.100	Integrated development environment used for software implementation and debugging.