Interoperable Web Platform Based on Large Language Models for Medical Data Analysis: A Protocol for Clinical Decision Support

Summary

This protocol describes the implementation of an interoperable web platform that integrates FHIR-based clinical data with Retrieval-Augmented Generation and multi-agent large language models for clinical decision support. The workflow enables reproducible deployment, standardized data integration, and evaluation of artificial intelligence (AI)-assisted medical data analysis.

Abstract

Clinical decision-making is frequently hindered by fragmented electronic health records and limited interoperability among heterogeneous healthcare information systems. This article presents a step-by-step protocol for implementing an interoperable web platform that integrates clinical data via the Fast Healthcare Interoperability Resources (FHIR) standard, along with Retrieval-Augmented Generation (RAG), Large Language Models (LLMs), and a multi-agent clinical reasoning framework to support medical data analysis and clinical decision support. The protocol describes the complete workflow, including computational environment configuration, clinical dataset preprocessing, FHIR-based data integration, vector database construction, retrieval configuration, prompt engineering, multi-agent orchestration, and system evaluation. Representative results demonstrate the platform's ability to generate clinically relevant and contextually consistent responses while improving semantic interoperability across heterogeneous data sources. System performance was evaluated using complementary quantitative and semantic metrics, including BLEU, ROUGE, BERTScore, and cosine similarity. A qualitative assessment was conducted using publicly available, fully anonymized benchmark healthcare datasets to evaluate the reproducibility of the proposed methodological workflow. The proposed architecture combines standardized healthcare interoperability with retrieval-enhanced language models to improve contextual reasoning, reduce hallucination, and support reproducible AI-assisted clinical workflows. This protocol provides a scalable and reproducible framework for researchers and developers seeking to implement interoperable, privacy-aware, and intelligent healthcare systems for clinical decision support, medical data analysis, and future translational research.

Introduction

Health information is routinely generated across hospitals, diagnostic centers, laboratories, and outpatient clinics, often remaining distributed among heterogeneous information systems. This fragmentation limits interoperability and restricts timely access to comprehensive patient records, creating persistent challenges for clinical care, data integration, and healthcare management^1,2,3,4.

The consequences of this fragmentation become particularly evident in clinical workflows that depend on timely access to patient information. When healthcare professionals cannot retrieve complete and integrated medical records, clinical assessment becomes more difficult, increasing the risk of incomplete decision-making and reducing the efficiency of care delivery, particularly in emergency settings^5,6.

Recent developments in Artificial Intelligence (AI), particularly Large Language Models (LLMs), have expanded the range of computational approaches available for healthcare applications. These models have been investigated in tasks such as diagnostic assistance, clinical triage, and decision support. For example, Jahan et al.⁵ evaluated the use of LLMs in biomedical tasks, while Taylor et al.⁷ investigated fine-tuned models for digital mental health screening, reporting encouraging results in specialized clinical settings.

The application of LLMs has also expanded to more specialized clinical domains. Song et al. ⁴⁹explored their use in pneumoconiosis diagnosis, whereas Chien et al.⁸ applied these models to analyze workload patterns among informal caregivers. In ophthalmology, Xue et al.⁹ proposed a question-answering system for glaucoma diagnosis, while Tan et al.³ and Wu et al.¹⁰ reported the use of LLMs in hybrid diagnostic systems and traditional Chinese medicine consultations.

The use of LLMs is not limited to direct clinical applications. Zhang et al.¹¹ investigated their application to emotion recognition in mental health scenarios, whereas Sarzaeim et al.¹² explored intelligent policing systems that may also contribute to public health-related analyses. These studies illustrate the broad applicability of LLM-based approaches across different healthcare-related domains.

Although LLMs have expanded the possibilities for healthcare applications, their integration into clinical environments remains associated with important technical, organizational, and regulatory challenges. Practical deployment requires appropriate computational infrastructure, effective data governance, and compliance with regulatory frameworks such as the General Data Protection Regulation (GDPR) and the Brazilian General Data Protection Law (LGPD). These frameworks establish requirements for the secure and ethical processing of sensitive health information while addressing issues related to privacy, reliability, and algorithmic bias in AI-assisted healthcare systems^13,14 .

Interoperability across heterogeneous healthcare data sources, including Electronic Health Records (EHRs), medical imaging systems, and Internet of Things (IoT) devices, continues to represent a major technical challenge for the deployment of AI-assisted healthcare solutions. In this context, standardized interoperability frameworks such as HL7 FHIR provide a structured mechanism for exchanging clinical information across different systems while preserving semantic consistency and supporting scalable integration.

This work presents an interoperable web platform that integrates Large Language Models with healthcare interoperability standards to support medical data analysis in heterogeneous clinical environments. The proposed architecture combines HL7 FHIR-based data integration with Retrieval-Augmented Generation (RAG) and a multi-agent processing framework to provide context-aware AI-assisted analysis while maintaining alignment with applicable data protection requirements, including the Brazilian General Data Protection Law (LGPD).

This protocol describes an interoperable architecture for integrating heterogeneous clinical data using established healthcare interoperability standards. It also details the implementation of a multi-agent processing pipeline based on large and small language models (LLMs and SLMs), together with an evaluation framework that combines quantitative metrics and qualitative analyses to assess the behavior of the proposed platform.

Protocol

This study did not involve the recruitment of human participants, access to identifiable patient records, or experiments involving animals. The protocol was developed and evaluated exclusively using publicly available and fully anonymized datasets for methodological validation. No personal health information was accessed or processed. Therefore, approval from an Institutional Review Board (IRB) or Research Ethics Committee was not required. The protocol was developed in accordance with applicable data protection principles, including the Brazilian General Data Protection Law (LGPD), to support future applications involving clinical data.

Dataset selection and preprocessing

The proposed protocol was evaluated using publicly available and fully anonymized clinical datasets comprising structured Electronic Health Records (EHRs), clinical question-answering data, and medical imaging datasets for methodological validation. Before integration into the platform, the datasets underwent standardized preprocessing procedures, including data normalization, removal of inconsistent or incomplete records, mapping to HL7 FHIR resources, text cleaning, segmentation into retrieval chunks, and embedding generation for vector indexing. These preprocessing steps ensured semantic consistency across heterogeneous data sources, facilitating interoperability and enabling the reproducibility of the proposed workflow while maintaining compliance with applicable data privacy principles.

The datasets were obtained from publicly available benchmark repositories commonly used in artificial intelligence and digital health research. They were selected to represent heterogeneous clinical information, including structured Electronic Health Records (EHRs), unstructured clinical narratives, clinical question-answering tasks, and medical imaging metadata. Rather than evaluating a specific clinical cohort, the protocol focuses on demonstrating a reproducible implementation workflow that can be adapted to different healthcare datasets. The diversity of these benchmark datasets enables validation of the interoperability pipeline, Retrieval-Augmented Generation (RAG), and multi-agent reasoning framework across multiple clinical data modalities.

Experimental environment configuration

The experimental environment was configured to evaluate the interoperable platform under controlled and reproducible conditions. The architecture comprises data ingestion modules, interoperability layers, large language models (LLMs), and evaluation components organized into a single processing pipeline for medical data analysis. Figure 1 illustrates the complete workflow, from clinical data ingestion to the generation of diagnostic outputs.

Figure 1: Overall system workflow illustrating the processing pipeline from raw clinical data to disease status output. The process begins with Electronic Health Records (EHR) ingestion, followed by data filtering and preprocessing to extract disease-sensitive information. A structured prompt design stage integrates expert knowledge, disease definitions, and hyperparameters, enabling effective interaction with the Large Language Model (LLM). The LLM performs text inference to generate context-aware responses, which are subsequently evaluated through clinical rules to determine the final disease status. The workflow highlights the integration of data preprocessing, knowledge-driven prompting, and AI-based inference to support clinical decision-making. Please click here to view a larger version of this figure.

Healthcare interoperability architecture

The backend infrastructure adopts a modular architecture based on RESTful APIs to support communication among the platform components (Figure 2). This architecture accommodates heterogeneous clinical information, including structured Electronic Health Records (EHRs), physician notes, and metadata derived from medical imaging systems. Because these data originate from multiple sources and formats, interoperability is achieved through standardized data models, particularly the Fast Healthcare Interoperability Resources (FHIR) framework^15,16,17.^.The adoption of FHIR supports structured information exchange while preserving scalability and flexibility in distributed healthcare environments. HL7-based communication mechanisms were also incorporated to facilitate integration with legacy clinical systems, which remain widely used in healthcare institutions^16,17.

Figure 2: System architecture of the proposed interoperable platform. The web interface communicates with the backend through a Flask API using HTTP POST/GET requests. The API manages routing, query processing, and interaction with both structured and unstructured data sources. A MySQL database stores structured clinical data, while a FAISS-based vector store supports similarity search for retrieval operations. The LLaMA-based pipeline processes textual inputs and generates responses using vector representations, enabling Retrieval-Augmented Generation (RAG). The architecture highlights the integration of web services, database management, vector retrieval, and large language model inference in a unified system. Please click here to view a larger version of this figure.

Multi-agent workflow configuration

The multi-agent architecture is organized into specialized functional agents responsible for distinct stages of the workflow. A preprocessing agent performs data normalization and FHIR mapping, followed by a retrieval agent responsible for semantic search within the vector database. A reasoning agent integrates the retrieved context with the LLM to generate responses, while a validation agent verifies output consistency and formatting before the final response is returned. Agent coordination follows a sequential orchestration strategy in which the output of each agent serves as the input for the subsequent stage, ensuring a reproducible and modular implementation.

Clinical data integration

The data integration layer aggregates information from multiple clinical sources and prepares it for downstream processing. Preprocessing includes data normalization, tokenization, and entity alignment to improve semantic consistency across heterogeneous datasets. Because clinical information varies in structure and quality, these operations help reduce noise and facilitate interaction with AI models. Structured mapping strategies were also applied to harmonize different data formats and maintain compatibility with the processing pipeline, as illustrated in Figure 3^15,16,17.

Figure 3: Detailed example of the multi-agent clinical reasoning process. The figure illustrates how a clinical query is analyzed through multiple stages, including complexity assessment, specialist recruitment, collaborative discussion, and final decision-making. This process demonstrates the system’s ability to dynamically adapt reasoning strategies based on query complexity, improving both efficiency and diagnostic accuracy in clinical decision support scenarios. Please click here to view a larger version of this figure.

Configure the Retrieval-Augmented Generation pipeline

Retrieval-Augmented Generation (RAG) is incorporated to provide context-aware analysis by combining information retrieval with the generative capabilities of large language models. User queries are converted into vector representations using embedding models and matched against the vector database using semantic similarity search to retrieve the most relevant contextual passages. The retrieved documents are then combined with the original query before inference by the LLM. This strategy helps provide contextual information during response generation and has been associated with improved factual consistency and reduced hallucination in knowledge-intensive applications, including healthcare^18,19. Figure 1 and Figure 3 illustrate the overall retrieval workflow and the corresponding reasoning process.

For each user request, the final prompt is dynamically constructed by combining the original query with the most relevant contextual passages retrieved from the vector database. The retrieved information is incorporated as contextual evidence before inference, allowing the language model to generate responses grounded in the retrieved healthcare knowledge while preserving semantic consistency and reducing unsupported generations.

Prompt engineering and multi-agent reasoning

The protocol incorporates structured prompting strategies to improve contextual interpretation during response generation. These prompting techniques, together with advanced inference mechanisms, help guide the reasoning process in complex clinical scenarios and are consistent with recent developments reported in the literature²⁰.

The architecture includes a multi-agent framework composed of specialized modules that perform distinct functions within the processing pipeline, including data validation, context filtering, clinical reasoning support, and output verification. The modular organization allows tasks to be executed sequentially or in parallel, providing flexibility for different processing requirements (Figure 4). Distributing these activities across multiple agents reduces reliance on a single language model and supports a more robust processing workflow. This architectural strategy is consistent with recent developments in distributed artificial intelligence and intelligent system design^21,22.

Figure 4: Multi-agent decision framework for clinical reasoning. The process begins with a user query, which is evaluated by an agent checker responsible for assessing query complexity. For complex cases, the system dynamically recruits a multi-disciplinary team (MDT) of specialized agents that engage in iterative discussion rounds to analyze the problem and synthesize knowledge before producing a final decision. For simpler cases, the query is handled by a primary care clinician (PCC) agent, enabling faster response generation. This adaptive architecture balances efficiency and analytical depth, improving decision quality and system scalability in healthcare applications. Please click here to view a larger version of this figure.

Performance evaluation

System performance was evaluated using complementary metrics that capture both linguistic quality and semantic consistency of the generated outputs. BLEU was applied to measure syntactic similarity based on n-gram overlap²³, whereas ROUGE assessed recall and content coverage, particularly in summarization and information extraction tasks²⁴. Semantic similarity was evaluated with BERTScore, which uses contextual embeddings derived from transformer-based models to compare generated and reference texts²⁵. Additional analyses included perplexity and cosine similarity to examine model confidence and semantic coherence, respectively^26,27.

The selected evaluation metrics provide complementary perspectives on system performance by combining lexical and semantic analyses. This combination is particularly relevant in healthcare applications, where contextual interpretation is as important as lexical similarity. The platform was evaluated in a controlled computational environment that supports both cloud-based and local deployment. Local execution of LLMs was included as an option to support data privacy requirements and reduce reliance on external services when processing sensitive clinical information. This deployment strategy is compatible with data protection frameworks and can be adapted to different operational environments^4.

Results

System performance was assessed using complementary quantitative metrics together with qualitative analysis to examine both linguistic accuracy and semantic consistency of the generated outputs. This evaluation strategy combines lexical and semantic measures to provide a broader characterization of model behavior in healthcare-related language generation tasks.

Table 1 presents the quantitative results obtained with BLEU, ROUGE, and BERTScore. These metrics were selected because they evaluate complementary aspects of generated text, including lexical similarity, information coverage, and semantic alignment, and are widely used in natural language processing research.

Judge	Prompt	Pearson	RMSE	MAE	Hit Rate
Mixtral	short	0.098	1.779	1.346	6/28
	zero-shot	0.174	1.618	1.293	15/28
	few-shot	0.475	1.673	1.367	9/28
LLaMA 3	short	0.485	1.617	1.314	11/28
	zero-shot	0.479	1.614	1.314	10/28
	few-shot	0.425	1.652	1.339	13/28
LLaMA 3.1	short	0.349	1.621	1.318	06/28
	zero-shot	0.68	1.614	1.307	12/28
	few-shot	0.408	1.243	0.886	11/28

Table 1: Quantitative evaluation metrics of the proposed system.

BLEU was used to quantify syntactic similarity based on n-gram overlap between generated outputs and reference texts^23. Originally developed for machine translation, it has also been applied to a wide range of text generation tasks because it captures structural correspondence between texts. In the present evaluation, the BLEU scores indicate that the generated responses preserve syntactic characteristics consistent with the reference outputs.

ROUGE was used to evaluate recall-oriented similarity, with emphasis on the coverage of relevant information in the generated texts²⁴. In healthcare applications, this metric is particularly useful because preserving clinically relevant information is often more important than reproducing identical wording. As reported in Table 2, the ROUGE scores indicate that the generated responses retain relevant clinical content across the evaluated cases.

Judge	Prompt	ICC1	ICC2	ICC3	ICC1k	ICC2k	ICC3K
Mixtral	short	0.136	0.164	0.182	0.32	0.37	0.4
	zero-shot	0.28	0.353	0.507	0.539	0.621	0.755
	few-shot	0.224	0.323	0.522	0.463	0.588	0.766
LLaMA3	short	0.234	0.331	0.532	0.479	0.597	0.773
	zero-shot	0.244	0.34	0.551	0.492	0.607	0.786
	few-shot	0.218	0.321	0.531	0.456	0.587	0.772
LLaMA3.1	short	0.521	0.525	0.537	0.766	0.768	0.777
	zero-shot	0.246	0.343	0.56	0.495	0.611	0.792
	few-shot	0.599	0.597	0.589	0.817	0.816	0.811

Table 2: Inter-rater agreement metrics (ICC).

BERTScore was included to evaluate semantic similarity using contextual embeddings derived from transformer-based models²⁵. Unlike BLEU and ROUGE, this metric compares texts according to contextual meaning rather than lexical overlap, making it suitable for evaluating clinical language generation. The BERTScore values obtained in this study indicate a high degree of semantic alignment between the generated responses and the corresponding reference texts.

Additional analyses included perplexity and cosine similarity to complement the primary evaluation metrics. Perplexity was calculated to estimate the predictability of the generated outputs, with lower values indicating lower uncertainty during text generation²⁶. Cosine similarity was used to quantify the alignment between embedding vectors derived from the generated and reference texts, providing an additional measure of semantic coherence²⁷.

Taken together, the evaluation metrics provide complementary information on the syntactic, semantic, and contextual characteristics of the generated responses. Figure 5 summarizes the distribution of the evaluation results across the assessed metrics, providing an overall view of system performance under the tested conditions.

The evaluation results are consistent with the expected behavior of Retrieval-Augmented Generation (RAG) in applications that require access to external knowledge sources. By incorporating retrieved documents into the response generation process, the architecture provides additional contextual information that supports clinically relevant responses in knowledge-intensive scenarios^18,19. Structured prompting strategies are incorporated as complementary mechanisms to guide the reasoning process and contextual interpretation, consistent with approaches reported in recent studies²⁰.

Qualitative analysis complemented the quantitative evaluation by examining the interpretability and clinical relevance of the generated outputs. Representative examples of system responses are shown in Figure 3 for different types of clinical queries. These examples illustrate how the platform generates contextually coherent responses across the evaluated scenarios, including cases involving more complex clinical information.

The multi-agent architecture distributes processing tasks across specialized modules responsible for complementary functions within the workflow. This organization reduces reliance on a single language model and enables multiple stages of information processing and output verification, consistent with recent multi-agent approaches reported in the literature^21,22. Figure 5 summarizes the distribution of the evaluation results obtained with the different prompting strategies and language models considered in this study.

Figure 5: Performance distribution of the proposed system across different prompting strategies and language models. (A–F) The violin plots illustrate variations in response quality for short, zero-shot, and few-shot prompting approaches using LLaMA3, LLaMA3.1, and Mixtral models. The results highlight the impact of prompt design on output consistency and performance, demonstrating that structured prompting strategies tend to produce more stable and accurate responses in clinical tasks. Please click here to view a larger version of this figure.

The evaluation results are consistent with recent studies that advocate combining lexical and semantic metrics for assessing the performance of large language models^4,12. In particular, embedding-based metrics have become increasingly important for capturing contextual similarity in healthcare-related language generation, where semantic interpretation extends beyond lexical matching^28,29.

Overall, the evaluation indicates that the proposed platform integrates interoperability standards, Retrieval-Augmented Generation (RAG), and multi-agent processing within a unified framework for healthcare data analysis. The quantitative and qualitative results presented in this study support the feasibility of the proposed workflow under the evaluated experimental conditions and provide a basis for future validation in real-world clinical environments.

DATA AVAILABILITY

The datasets used in this study are publicly available. The chest X-ray dataset was obtained from the Indiana University Chest X-ray Collection (Open-i, U.S. National Library of Medicine), including the indiana_reports.csv and indiana_projections.csv files, available at https://openi.nlm.nih.gov/. The MedQA benchmark dataset is publicly available through its official repository. No proprietary or patient-identifiable clinical data were used in this study. All preprocessing procedures are described in the Protocol section to support reproducibility.

Discussion

The evaluation presented in this study represents a methodological validation performed using publicly available and fully anonymized benchmark healthcare datasets. No prospective clinical validation involving healthcare professionals or patient recruitment was conducted, as the objective of this work is to demonstrate a reproducible implementation protocol rather than clinical effectiveness.

The findings of this study suggest that combining healthcare interoperability standards with large language models provides a practical framework for integrating heterogeneous clinical information. The proposed architecture brings together structured data exchange mechanisms, including FHIR and HL7, with AI-based language processing in a unified workflow that is consistent with current developments in digital health systems^15,16,17.

An important aspect of the proposed workflow is the integration of Retrieval-Augmented Generation (RAG) within a multi-agent architecture. In this configuration, retrieved documents provide additional contextual information during response generation, supporting knowledge-intensive clinical tasks. This architectural design is consistent with recent studies that describe retrieval-based mechanisms as a strategy for improving contextual grounding and response reliability in large language model applications^18,19.

Structured prompting strategies were incorporated into the proposed workflow to support contextual interpretation during response generation. Previous studies have reported that prompt engineering and structured reasoning techniques can improve the performance of large language models in complex decision-making tasks²⁰. The approach adopted in this protocol is consistent with these developments and provides a structured framework for clinical language processing.

From an evaluation perspective, the use of complementary metrics allowed different dimensions of system performance to be examined. While BLEU and ROUGE provide information on syntactic similarity and content coverage^23,24, embedding-based metrics such as BERTScore capture semantic relationships that may not be reflected by lexical overlap alone²⁵. This multidimensional evaluation strategy is consistent with recent benchmarking studies that recommend combining lexical and semantic measures when assessing large language models in healthcare applications^4,12.

The proposed workflow provides a reproducible methodological framework for implementing interoperable AI-based clinical decision support systems. Rather than benchmarking different artificial intelligence architectures, this study focuses on documenting the complete implementation workflow, system architecture, interoperability mechanisms, and evaluation methodology to facilitate reproducibility and future adoption. Comparative analyses involving conventional LLMs, RAG-disabled configurations, fine-tuned models, or traditional machine learning approaches are beyond the scope of this methodological contribution and represent an important direction for future research.

Future translation of the proposed framework into real-world clinical environments will require additional validation with healthcare professionals as well as compliance with applicable regulatory and ethical requirements. Depending on the intended use, such systems may be subject to Software as a Medical Device (SaMD) regulations and should comply with the requirements established by regulatory authorities, including the U.S. Food and Drug Administration (FDA) and the European Medical Device Regulation (MDR). In addition, robust data governance, cybersecurity, transparency, and clinical safety practices will be essential for supporting safe and responsible deployment in healthcare settings.

The qualitative analysis complemented the quantitative evaluation by illustrating the characteristics of the generated responses across representative clinical scenarios. The examples presented indicate that the platform was able to produce contextually coherent responses under the evaluated conditions, providing additional insight into response interpretability beyond the quantitative metrics. Similar observations have been reported in studies applying large language models to clinical applications, including diagnostic assistance and patient interaction^28,29,30,31.

The proposed architecture distributes processing tasks across specialized modules responsible for complementary functions, including data validation, contextual filtering, clinical reasoning support, and output verification. This modular organization reduces reliance on a single language model and enables successive stages of information processing within the workflow. Similar architectural strategies have been described in recent studies on distributed AI systems and multi-agent frameworks designed for complex decision-making environments^21,22.

Several implementation considerations may facilitate the reproducibility and deployment of the proposed protocol. Consistent mapping of clinical information to the HL7 FHIR standard helps preserve semantic interoperability throughout the data processing pipeline. Similarly, document preprocessing, chunking strategies, embedding generation, and vector indexing influence the behavior of the Retrieval-Augmented Generation workflow and should be configured and validated according to the characteristics of the target application. Prompt engineering and multi-agent orchestration may also be refined iteratively using domain-specific knowledge and expert feedback to improve contextual grounding during response generation. These implementation practices are consistent with recent developments in trustworthy artificial intelligence and retrieval-enhanced language models^18,19,20.

Some limitations of this study should be acknowledged. Although the evaluation combined complementary quantitative and qualitative metrics, these measures may not fully capture all aspects of clinical reasoning. This observation is consistent with previous studies that recommend domain-specific evaluation frameworks for healthcare applications¹². In addition, the platform was evaluated under controlled conditions using publicly available and anonymized benchmark datasets, which may not fully represent the variability and complexity of real-world clinical environments.

The proposed platform was developed in accordance with established healthcare interoperability standards. The adoption of HL7 and FHIR supports structured data exchange across heterogeneous healthcare information systems, providing a standardized framework for integrating clinical information from multiple sources. This architectural approach is consistent with current efforts to develop interoperable AI systems for distributed healthcare environments^15,16,17.

The successful implementation of the proposed workflow depends on several critical methodological steps. First, clinical data should be consistently mapped to standardized HL7 FHIR resources to preserve semantic interoperability across heterogeneous healthcare systems^15,17. Second, document preprocessing, including normalization, chunking strategy, and embedding generation, should be carefully configured because these stages directly influence retrieval quality within the Retrieval-Augmented Generation (RAG) pipeline^19,32,33. Third, the vector database should be rebuilt whenever the knowledge base is updated to maintain consistency between indexed documents and retrieval results. Finally, prompt engineering and multi-agent orchestration should be iteratively validated using representative clinical scenarios to improve contextual accuracy, minimize hallucinations, and enhance the reproducibility of AI-assisted clinical decision support workflows^4,3,20,31.

From a troubleshooting perspective, common implementation challenges include incomplete FHIR resource mapping, reduced retrieval performance associated with document indexing or vector database configuration, and responses affected by insufficient contextual information or suboptimal prompt engineering. These issues may be addressed through validation of interoperability resources, optimization of retrieval parameters, periodic updating or rebuilding of the vector database after modifications to the knowledge base, and continuous evaluation using complementary lexical and semantic metrics. These practices support consistent system behavior and facilitate the deployment and maintenance of AI-assisted clinical decision support workflows^4,12,25,23..

From a practical perspective, deploying large language models in healthcare environments requires consideration of computational resources and infrastructure requirements. Although the proposed architecture supports both cloud-based and local execution, additional optimization strategies may be necessary depending on the scale of deployment and the computational resources available, particularly in resource-constrained settings⁴.

Future research may extend the proposed workflow by evaluating the platform with real-world clinical datasets and routine healthcare workflows. Additional investigations may also explore domain-specific fine-tuning strategies and the integration of explainable AI methods to improve model interpretability and support transparency during clinical decision support. These directions may contribute to a broader assessment of the platform in practical healthcare environments.

Overall, this work presents a reproducible framework for integrating healthcare interoperability standards, Retrieval-Augmented Generation (RAG), and multi-agent language model architectures within a unified clinical data processing workflow. The proposed protocol provides a structured approach for implementing and evaluating AI-assisted medical data analysis systems and may serve as a reference for future developments in interoperable clinical decision support.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
FAISS	Meta Platforms Inc.	Version 1.8.0	Vector database used for similarity search in the Retrieval-Augmented Generation (RAG) pipeline.
FHIR Standard	HL7 International	Release 4 (R4)	Healthcare interoperability standard adopted for structured clinical data exchange.
Flask	Pallets Projects	Version 3.0.3	Python web framework used to implement the RESTful backend API.
HL7 Standard	HL7 International	Version 2.x	Legacy messaging protocol used for interoperability with hospital information systems.
HTTP REST API	Custom Implementation	N/A	RESTful communication layer between frontend, backend, database and AI services.
LangChain	LangChain Inc.	Version 0.3.x	Framework used to integrate LLMs, prompt engineering and Retrieval-Augmented Generation workflows.
LangGraph	LangChain Inc.	Version 0.2.x	Framework used for orchestrating the multi-agent clinical reasoning workflow.
LLaMA 3.1 8B Instruct	Meta Platforms Inc.	Version 3.1	Large Language Model employed for clinical reasoning and natural language generation.
MySQL Community Server	Oracle Corporation	Version 8.0	Relational database used to store structured clinical data.
Ollama	Ollama Inc.	Version 0.9.x	Local inference engine used for executing Large Language Models while preserving patient privacy.
Python	Python Software Foundation	Version 3.11	Programming language used for implementing the complete platform.
PyTorch	Linux Foundation	Version 2.4	Deep learning framework used for Large Language Model inference.
Retrieval-Augmented Generation (RAG) Pipeline	Custom Implementation	N/A	AI framework combining semantic retrieval with LLM inference for context-aware response generation.
Sentence Transformers	Hugging Face Inc.	all-MiniLM-L6-v2	Embedding model used to generate dense vector representations for semantic document retrieval.
Ubuntu Linux	Canonical Ltd.	Version 24.04 LTS	Operating system used for development and experimental evaluation.
Visual Studio Code	Microsoft Corporation	Version 1.100	Integrated development environment used for software implementation and debugging.