Foundation models could connect the multimodal data that already define gastroenterology, but their clinical value depends on local validation, auditability, privacy protection, uncertainty estimation, and human oversight. Near-term progress is more likely to come from focused, reviewable tools than from a single model for the entire specialty.
综述文章
Foundation Models in Gastroenterology: Emerging Applications and Future Directions
* These authors contributed equally
本文内容
摘要
摘要
Foundation models are being investigated in gastroenterology because the specialty already generates multimodal data, including endoscopy videos, pathology slides, abdominal images, laboratory results, molecular profiles, and longitudinal clinical notes. This review examines where these models may be useful in emerging applications, where evidence remains limited, and what is required before wider clinical deployment. Large language models (LLMs) can support document organization, report generation, guideline retrieval, and patient communication, but fluent output can conceal errors. Endoscopic models are beginning to move beyond single-task polyp detection towards reusable visual pretraining across gastrointestinal sites and procedures. Pathology and radiology foundation models may assist with biomarker prediction, image segmentation, and tumor detection, although performance remains sensitive to staining, scanners, acquisition protocols, and case mix. The immediate goal is not to build a single model that covers the entire specialty. Rather, gastroenterology needs systems that are locally validated, auditable, privacy-aware, uncertainty-calibrated, and sufficiently focused to remain within clinical responsibility.
引言
The term “foundation model” is now common in medical artificial intelligence (AI), although the available evidence does not always justify the confidence with which it is used. In a narrow sense, a foundation model is pretrained on large, diverse datasets and then adapted to multiple downstream tasks; model size or a transformer architecture alone does not confer this status. Medical examples process text, images, waveforms, molecular data, or combinations of these modalities1,2. Gastroenterology is a natural testbed because a single patient journey may include symptoms, endoscopy, pathology, radiology, treatment, and years of follow-up. The resulting record is inherently multimodal.
For interpretation, the framework distinguish traditional task-specific AI, pretrained domain-specific foundation models, multimodal/general-purpose models, and downstream applications (Figure 1). This is a conceptual progression, not a mandatory pipeline: a unimodal domain-specific model can support several applications without becoming multimodal. Datasets and adaptation methods are identified separately. Conventional AI establishes background feasibility; foundation-model evidence must demonstrate transferable pretraining and task-specific evaluation. Table 1 applies this framework to the major examples.
Large language models (LLMs) have begun to appear in clinical and educational settings. General medical models can answer examination questions and participate in simulated diagnostic dialogues3,4,5. These results indicate capability, but they do not establish clinical readiness. A model may perform well on a benchmark yet fail to interpret a post-polypectomy interval, a pancreatic cyst report, a steroid-refractory inflammatory bowel disease flare, or a cirrhosis record with missing laboratory data. Recent commentary, therefore, presents foundation models as a possible next step for digestive disease and endoscopy, rather than as replacements for focused clinical systems6.
This review has a deliberately narrow scope. It does not cover every application of artificial intelligence in digestive disease. Instead, it considers clinical scenarios in which foundation models may reshape gastroenterology, including language models, endoscopy, pathology, radiology, multi-omics and the requirements for safe deployment. The central argument is that near-term value will depend less on model scale than on whether systems can be tested locally, audited in practice, and kept within clearly defined clinical responsibilities.
综述与观点
Study selection
This is a focused narrative review using purposive selection of representative studies, not a systematic or exhaustive review. During revision, targeted PubMed and publisher-site searches were performed on 7 October 2026 for English-language journal literature from 2018 to that date. Search concepts combined foundation models, large language models or retrieval-augmented generation with gastroenterology, endoscopy, pathology, radiology or omics; named models and cited papers were also checked. Studies were chosen to illustrate reusable pretraining, downstream evaluation, evidence maturity or deployment requirements. Earlier task-specific AI studies were retained as explicitly labelled background. No systematic screening or formal risk-of-bias assessment was performed.
Distinctive features of gastroenterology
Earlier artificial intelligence systems in gastroenterology were usually designed for single tasks. A colonoscopy model detected polyps, a pathology model predicted microsatellite instability, and a radiology model segmented a lesion. Many such tools are useful, and some have undergone rigorous evaluation. However, this development pattern is expensive. Each task requires new labels, training, validation, integration, monitoring, and, often, a separate regulatory argument.
Foundation models offer a different trade-off. They do not remove the need for task-specific testing, but they may reduce how much must be rebuilt for every new application. A pretrained endoscopy model could support polyp detection, Barrett esophagus assessment, blind-spot monitoring, upper gastrointestinal lesion classification, and procedure-quality analysis. A pathology model trained on diverse whole-slide images might support the identification of several gastrointestinal cancer biomarkers. The value lies in reuse, not in eliminating clinical validation.
The risks are equally practical. Endoscope vendors, bowel preparation, staining protocols, scanners, patient populations, reporting habits, and local regulations vary across sites. A model may perform well where it was developed and poorly elsewhere. Text models pose an additional problem because an incorrect answer can sound reasonable. The relevant question is therefore whether broad pretraining provides enough support for a specific, governed use case.
Language models in the clinic
Language is present throughout digestive care. Referral notes, procedure reports, pathology descriptions, imaging impressions, multidisciplinary discussions, trial criteria, patient questions and guidelines are all text-rich. Recent studies have examined LLMs in gastroenterology and hepatology for question answering, education, documentation, research support and patient information7,8. A recent scoping review by Yazarkan et al. mapped 73 studies, with most evaluations based on simulated or literature-derived cases and limited but growing use of real-world clinical data9. This supports the distinction between technical capability and demonstrated clinical benefit. Other work has been explored by literature screening, cohort searches, variable extraction, and protocol preparation10. The discussion of GastroGPT illustrates the appeal of adapting a language model to specialty-specific terminology and failure modes; the cited commentary is not itself evidence of routine clinical effectiveness11.
Comparatively lower-risk applications remain close to reviewable office work, provided that a clinician checks the source and output before use. LLMs can summarize a long admission, extract structured attributes from free-text endoscopy reports, generate patient instructions or retrieve the relevant section of a guideline. These tasks are not risk-free. Patient-specific surveillance, diagnosis, and treatment recommendations are higher-risk uses and require separate validation, explicit handling of missing variables, and clinician authorization before action. A surveillance recommendation for a serrated lesion can be wrong if a pathology detail is missing. A Crohn disease recommendation can be inappropriate if the model does not account for previous biologic exposure, surgery, infection risk or local drug availability. The important safeguard is that the clinician can inspect the source record and correct the output.
Patient communication is another plausible use. Digestive diseases may be chronic, sensitive or difficult to explain during a short consultation. A language model could adapt colonoscopy-preparation instructions, explain histology results or translate a liver-imaging report into plain language. The patient-facing system should be connected to approved local content, display its sources and provide a clear route back to the clinical team. Tone is useful only when certainty and provenance are also controlled. Unsupervised personalized advice should not be inferred to be safe from performance on patient-education questions.
Retrieval-augmented generation and grounding
Retrieval-augmented generation (RAG) combines a language model with retrieval of relevant passages from a controlled knowledge source. The retrieved passages are supplied as context for generation, allowing answers to cite inspectable evidence. Almanac illustrates this architecture in a clinical question-answering evaluation; it provides methodological evidence, not proof of safe routine gastroenterology deployment12. For a gastrointestinal workflow, the retrieval source should contain approved, versioned local guidelines and relevant patient records, with access controls. The system should identify missing clinical variables, show the supporting passages and abstain when evidence is insufficient or conflicting. RAG can still retrieve outdated or irrelevant material, omit a critical passage or generate claims unsupported by its sources. Retrieval coverage, citation fidelity, answer correctness and abstention therefore require separate local evaluation, particularly for surveillance or treatment decisions.
Endoscopy: established task-specific AI and emerging foundation models
Gastroenterology is among the specialties with the longest record of clinical artificial intelligence evaluation in endoscopy. Early task-specific studies showed real-time polyp localization, and randomized trials evaluated adenoma detection and blind-spot monitoring13,14,15,16,17. These studies establish an evidence base for conventional endoscopic AI, not for foundation models. They are informative because they tested systems under conditions resembling real endoscopic work. They also show the limits of narrow tools. A polyp detector is not automatically a Barrett esophagus model, a gastric-cancer triage system or an upper gastrointestinal quality monitor.
Endoscopic foundation models aim to provide a reusable visual base. Recent work has compared pretrained visual representations across downstream endoscopy tasks, examining architectures, pretraining strategies and data efficiency18. GastroNet-5M is a multicentre pretraining dataset, not a foundation model or a clinical intervention; its associated experiments support representation learning for gastrointestinal endoscopy19. A foundation-model-based esophagogastric-junction adenocarcinoma study included held-out, external and prospectively collected test sets, providing early diagnostic-validation evidence for a defined downstream application20. These technical and diagnostic evaluations do not establish improved patient outcomes or routine deployment of a general endoscopic foundation model.
Accuracy alone is not enough. Endoscopic frames may be blurred, partially obscured by mucosa or mucus, altered by insufflation or affected by bleeding. Some examinations simply do not provide an adequate view. Style-transfer methods such as EndoStyle may help address vendor and site variation, but they remain development tools rather than evidence of safe deployment21. A clinically useful endoscopic model should be able to identify poor image quality and express uncertainty. Forced confidence is not a clinical virtue.
Pathology, imaging and biology
Pathology is another area in which foundation-model approaches are attractive. Earlier task-specific deep learning studies predicted microsatellite instability (MSI) from routine gastrointestinal and colorectal cancer histology22,23. Transformer-based biomarker prediction has also been evaluated in large multicentre colorectal cancer cohorts; architecture and cohort size alone do not make it a general-purpose foundation model24. These studies establish that histology can contain predictive biomarker information. Foundation-model evidence addresses a different question: whether broadly pretrained representations transfer across tasks. UNI, a self-supervised vision encoder, and CONCH, a visual-language model, have been evaluated across pathology classification and related tasks25,26. Such retrospective benchmark and transfer-learning evidence supports the reuse of representations, but does not establish prospective clinical benefit for gastrointestinal biomarker-guided treatment.
The clinical boundary remains important. Staining, fixation, scanner type, tissue processing, and case selection must be sufficiently consistent for a prediction to be trusted. If a biomarker prediction is wrong, a patient may be incorrectly triaged or considered for an inappropriate treatment pathway. Until they are validated for a defined clinical decision, pathology foundation models should therefore be regarded as triage or decision-support tools.
Radiology also distinguishes conventional task-specific image analysis from broadly pretrained representations and their downstream adaptations. Abdominal imaging includes computed tomography (CT), magnetic resonance imaging (MRI), ultrasonography (US), elastography and endoscopic ultrasound (EUS), often acquired with multiphase protocols and uneven annotations. MedSAM is a medical adaptation of a promptable foundation segmentation model, evaluated across medical-image segmentation tasks27. A multicentre colorectal liver metastasis pipeline adapted pretrained encoders for CT detection and classification, with an independent test set and an external cohort28. These studies provide segmentation benchmarks and retrospective diagnostic-validation evidence, rather than proof of prospective workflow or patient benefit. Local validation remains essential because acquisition protocols and scanner settings are not incidental details.
Biology is a research and future-direction domain. AlphaFold is included as a foundation-model-adjacent context: its protein-structure prediction advances do not, by themselves, demonstrate reusable representations across clinical tasks. In contrast, scGPT and Geneformer are broadly pretrained single-cell models that support transfer to molecular and cellular tasks29,30,31. These approaches may help connect inflammation, fibrosis, cancer development, microbiome-host interactions, and drug response in digestive disease. A realistic near-term role is to rank hypotheses, integrate multiple omics layers, and direct researchers towards mechanism-based experiments. The cited studies do not establish a clinical decision-making benefit in gastrointestinal care, and these tools should not be treated as substitutes for those experiments.
Table 1 separates established task-specific AI, actual foundation models, foundation-model-enabled applications, and supporting datasets or methods. Its evidence descriptions distinguish commentary, technical benchmarks, retrospective validation, and prospectively collected diagnostic tests; none is equated with established routine clinical benefit.
Before clinical deployment
The deployment problem begins with the use case. A model that summarizes endoscopy reports is not equivalent to a model that recommends surveillance or flags malignancy. Study design and reporting should reflect this difference. Consolidated Standards of Reporting Trials–Artificial Intelligence (CONSORT-AI), Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence (SPIRIT-AI), and Developmental and Exploratory Clinical Investigations of DEcision support systems driven by Artificial Intelligence (DECIDE-AI) provide useful frameworks for reporting trials, protocols, and early clinical evaluations of artificial intelligence systems32,33,34. Gastroenterology studies should specify the clinical context, comparator, clinician role, failure plan, update policy and post-release monitoring strategy.
Interpretability should be practical. A heat map designed for developers may not help an endoscopist decide whether to biopsy, resect, wash or re-examine. A language-model answer without a source passage cannot be verified. For endoscopy, pathology and radiology, outputs should identify the area of interest, image-quality problems and calibrated uncertainty estimates, with possible reasons when they are known. For text systems, each important statement should be traceable to the record or guideline passage that supports it. Uncertainty estimation and calibration should be tested against observed errors; displaying confidence value alone is insufficient.
Privacy and governance are also part of accuracy. Digestive-care data may include videos, histology slides, images, laboratory values, genetic information, medication history, and sensitive symptoms. Multicentre studies can improve generalizability, but they raise questions about consent, anonymization, vendor access, cybersecurity, and institutional responsibility. Federated learning and local fine-tuning may reduce some technical barriers. A deployable model also needs an owner and a plan for drifting, complaints, updates, and withdrawal.
Foundation models can make poor data practices look sophisticated. Inconsistent endoscopy reports, incomplete follow-up, nonstandardized pathology labels and weak image-quality control may simply be learned by a larger model. Good artificial intelligence begins with ordinary clinical discipline: reliable records, clear labels, appropriate quality control and accountable workflows.
Future directions
The most useful future systems will probably connect multiple modalities. Few colorectal cancer decisions depend on a single data type. Pathology, molecular tests, stage, operative findings, previous treatments, health status, and follow-up are interdependent. A model that prepares a tumor-board summary from these sources, flags missing information, and shows the origin of each statement would be more practical than another stand-alone classifier.
Endoscopy may also move from frame-level assistance towards procedure-level assessment. A procedure-level system could record mucosal exposure, withdrawal quality, blind spots, lesion context, optical diagnosis, documentation completeness, and follow-up recommendations during the examination. Its value should be judged by procedure quality, workload, and patient outcomes together, not only by sensitivity on selected frames.
Language models need grounding more than polishing. The retrieval-augmented architecture described above should retrieve the current local guideline, cite the relevant passages, identify missing information, and refuse to answer when the record is insufficient. Post-polypectomy management, pancreatic-cyst surveillance, and liver disease in inflammatory bowel disease are examples in which calibrated refusal may be safer than an unsupported estimate.
Pathology and radiology models should be evaluated by how they change clinical work. Are they available in emergencies? Do they reduce interobserver variability? Do they improve lesion measurement or biomarker triage? Do they support multidisciplinary discussion? A higher area under the curve in a retrospective study is informative, but it does not by itself establish real-world usefulness.
结论
Foundation models offer reusable representations for gastroenterology, but established evidence for task-specific AI cannot be transferred to them without direct evaluation. Near-term opportunities include reviewable documentation, grounded patient communication, procedure-quality support and multimodal record organization. Molecular applications remain primarily research tools for hypothesis generation and experimental prioritization. Progress should be judged by prospectively demonstrated improvements in workflow and patient care, with local validation, auditability, privacy protection, uncertainty estimation and calibration, and accountable human oversight. Defined tasks and explicit failure plans will be essential as models are adapted across institutions, modalities and patient populations. Technical benchmark gains alone are insufficient to establish clinical benefit.
Figure 1: Multimodal foundation models in gastroenterology. The gastroenterology data ecosystem integrates heterogeneous clinical data, including endoscopy videos, clinical text, pathology slides, laboratory results, radiology images, and molecular/omics data. These inputs can support vision, language, and multimodal foundation models through large-scale pretraining and task-specific adaptation. Panels A–D show emerging clinical applications whose maturity varies, rather than established routine uses. These include language support (report summarization, guideline retrieval, and patient communication), endoscopy (polyp detection, Barrett esophagus assessment, blind-spot monitoring, and procedure-quality assessment), pathology (biomarker prediction, tumor triage and histology-based decision support), radiology (lesion segmentation, colorectal liver-metastasis detection and measurement assistance); panel E denotes research/future directions (hypothesis generation, multi-omics integration and drug-response exploration), rather than clinical decision-making. Before clinical deployment, models should undergo local validation and be supported by auditability, privacy protection, human oversight, uncertainty estimation and calibration, workflow monitoring, and model-drift control. The figure is an original schematic; all graphical elements are original, and no elements are adapted or reproduced from previously published material. Pathology and radiology illustrations are schematic, not patient-derived microscopic images or quantitative data. Please click here to view a larger version of this figure.
Table 1: Representative evidence and translational maturity of foundation models and related technologies in gastroenterology. The table classifies major examples by modality, model or resource category, pretraining basis, downstream task, evaluation design and remaining translational gap. Conventional AI and foundation-model-adjacent examples provide context and are not counted as evidence of foundation-model clinical effectiveness. References correspond to the numbered manuscript bibliography. Please click here to download this file.
Abbreviations: AI, artificial intelligence; CRLM, colorectal liver metastases; CT, computed tomography; EGJA, esophagogastric-junction adenocarcinoma; LLM, large language model; MSI, microsatellite instability; RAG, retrieval-augmented generation.
披露
The authors declare no conflicts of interest.
参考文献
- Moor M et al. Foundation models for generalist medical artificial intelligence. Nature. 2023;616(7956):259–265.
- Acosta JN, Falcone GJ, Rajpurkar P, Topol EJ. Multimodal biomedical AI. Nat Med. 2022;28(9):1773–1784.
- Singhal K et al. Large language models encode clinical knowledge. Nature. 2023;620(7972):172–180.
- Singhal K et al. Toward expert-level medical question answering with large language models. Nat Med. 2025;31(3):943–950.
- Tu T et al. Towards conversational diagnostic artificial intelligence. Nature. 2025;642(8067):442–450.
- Rafiee Sevyeri L, von Renteln D. Foundation models in medicine, gastroenterology, and endoscopy: the next frontier? Gastroenterology. 2026;170(1):16–18.
- Gong EJ et al. Large language models in gastroenterology: systematic review. J Med Internet Res. 2024;26:e66648.
- Omar M et al. Emerging applications of NLP and large language models in gastroenterology and hepatology: a systematic review. Front Med (Lausanne). 2024;11:1512824.
- Yazarkan Y, Sonmez G, Simsek C. Artificial intelligence in gastroenterology clinical practice: Scoping review of large language model applications. Int J Med Inform. 2026;214:106413.
- Berry P, Dhanakshirur RR, Khanna S. Utilizing large language models for gastroenterology research: a conceptual framework. Ther Adv Gastroenterol. 2025;18:17562848251328577.
- Tiwari A, Bharadwaj HR, Ahmed KAHM, Dahiya DS. GastroGPT Pioneering Specialized AI in Gastroenterology: Strengths, Pitfalls, and the Road to Clinical Integration. JGH Open. 2025;9(11):e70306.
- Zakka C et al. Almanac — Retrieval-Augmented Language Models for Clinical Medicine. NEJM AI. 2024;1(2):AIoa2300068.
- Urban G et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology. 2018;155(4):1069–1078.e8.
- Wang P et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut. 2019;68(10):1813–1819.
- Wu L et al. Randomised controlled trial of WISENSE, a real-time quality improving system for monitoring blind spots during esophagogastroduodenoscopy. Gut. 2019;68(12):2161–2169.
- Repici A et al. Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology. 2020;159(2):512–520.e7.
- Wang P et al. Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol Hepatol. 2020;5(4):343–351.
- Boers TGW et al. Foundation models in gastrointestinal endoscopic AI: Impact of architecture, pre-training approach and data efficiency. Med Image Anal. 2024;98:103298.
- Jong MR et al. GastroNet-5M: a multicenter dataset for developing foundation models in gastrointestinal endoscopy. Gastroenterology. 2026;170(1):174–187.
- Ma Y et al. Development and validation of an AI foundation model for endoscopic diagnosis of esophagogastric junction adenocarcinoma: a cohort and deep learning study. EClinicalMedicine. 2025;89:103524.
- Troya J et al. Gastrointestinal endoscopic image style transfer using EndoStyle to improve artificial intelligence prediction models. NPJ Digit Med. 2026;9(1):340.
- Kather JN et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat Med. 2019;25(7):1054–1056.
- Echle A et al. Clinical-grade detection of microsatellite instability in colorectal tumors by deep learning. Gastroenterology. 2020;159(4):1406–1416.e11.
- Wagner SJ et al. Transformer-based biomarker prediction from colorectal cancer histology: a large-scale multicentric study. Cancer Cell. 2023;41(9):1650–1661.e4.
- Chen RJ et al. Towards a general-purpose foundation model for computational pathology. Nat Med. 2024;30(3):850–862.
- Lu MY et al. A visual-language foundation model for computational pathology. Nat Med. 2024;30(3):863–874.
- Ma J et al. Segment anything in medical images. Nat Commun. 2024;15(1):654.
- Mali SA et al. Robust multicentre detection and classification of colorectal liver metastases on CT: application of foundation models. NPJ Precis Oncol. 2026;10(1):326.
- Jumper J et al. Highly accurate protein structure prediction with AlphaFold. Nature. 2021;596(7873):583–589.
- Cui H et al. scGPT: toward building a foundation model for single-cell multi-omics using generative AI. Nat Methods. 2024;21(8):1470–1480.
- Theodoris CV et al. Transfer learning enables predictions in network biology. Nature. 2023;618(7965):616–624.
- Liu X et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med. 2020;26(9):1364–1374.
- Cruz Rivera S et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med. 2020;26(9):1351–1363.
- Vasey B et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med. 2022;28(5):924–933.