Verification feedback and self-revision for omissions in clinical recommendations by large language models: A paired evaluation
Large language models (LLMs) in clinical decision support can omit required actions (omission) or recommend wrong ones (commission). Verification layers check responses, and models can revise without feedback. Based on 176 test cases, we delivered the first responses of three LLMs (GPT-6 Astra, Gemini 3.1 Pro, Claude Opus 5.5) in six versions: no feedback, the full findings of a rule-and-agent verification framework, or alerts calibrated to a 10% false-alert budget, each appended or returned to the model for revision. Three LLM judges scored the share of high-grade (high-certainty) items on LLM-generated checklists that each text omitted; physicians reviewed samples. Revision without feedback (self-revision) lowered omission from 8.1% to 7.1% (−0.9 percentage points; 95% confidence interval [CI]: −1.7 to −0.1). In post hoc comparisons, revision with the full findings omitted 2.2 percentage points more than self-revision (95% CI: 1.3 to 3.2), or 0.8 (95% CI: −0.1 to 1.8) without the Gemini 3.1 Pro revisions that replaced content with status phrases. Revision with the alerts, which reached 56 of 528 first responses, did not differ from self-revision (0.5 percentage points; 95% CI: −0.3 to 1.3). Appended, the full findings changed judged omission by −0.6 percentage points (95% CI: −1.20 to −0.04). Based on 354 responses to physician-reviewed commission probes, both deliveries of the full findings lowered commission, mostly citations to nonexistent guidelines, from 6.8% to 3.4% (appended) and 4.2% (revised); with hedged commission counted, only revision lowered it (post hoc). Revision with verification feedback did not reduce omission beyond self-revision, which evaluations should include as a baseline.

- Singhal K, Azizi S, Tu T, et al. Large language models encode clinical knowledge. Nature. 2023;620(7972):172-180. doi: 10.1038/s41586-023-06291-2
- Goh E, Gallo R, Hom J, et al. Large language model influence on diagnostic reasoning: A randomized clinical trial. JAMA Netw Open. 2024;7(10):e2440969. doi: 10.1001/jamanetworkopen.2024.40969
- Brodeur PG, Buckley TA, Kanjee Z, et al. Performance of a large language model on the reasoning tasks of a physician. Science. 2026;392(6797):524-527. doi: 10.1126/science.adz4433
- Fischer HD, Juurlink DN, Mamdani MM, Kopp A, Laupacis A. Hemorrhage during warfarin therapy associated with cotrimoxazole and other urinary tract anti-infective agents: A population-based study. Arch Intern Med. 2010;170(7):617-621. doi: 10.1001/archinternmed.2010.37
- Liu NF, Lin K, Hewitt J, et al. Lost in the middle: How language models use long contexts. Trans Assoc Comput Linguist. 2024;12:157-173. doi: 10.1162/tacl_a_00638
- Bhattacharyya M, Miller VM, Bhattacharyya D, Miller LE. High rates of fabricated and inaccurate references in ChatGPT-generated medical content. Cureus. 2023;15(5):e39238. doi: 10.7759/cureus.39238
- Wu K, Wu E, Wei K, et al. An automated framework for assessing how well LLMs cite relevant medical references. Nat Commun. 2025;16(1):3615. doi: 10.1038/s41467-025-58551-6
- Henry K, Xu S, Blotske K, et al. Large language models management of medications: three performance analyses. arXiv. Preprint posted online 2025. doi: 10.48550/arXiv.2509.22926
- Chen S, Kann BH, Foote MB, et al. Use of artificial intelligence chatbots for cancer treatment information. JAMA Oncol. 2023;9(10):1459-1462. doi: 10.1001/jamaoncol.2023.2954
- Sendak MP, Kim JY, Hasan A, et al. Empowering US healthcare delivery organizations: Cultivating a community of practice to harness AI and advance health equity. PLoS Digit Health. 2024;3(6):e0000513. doi: 10.1371/journal.pdig.0000513
- Shah NH, Halamka JD, Saria S, et al. A nationwide network of health AI assurance laboratories. JAMA. 2024;331(3):245-249. doi: 10.1001/jama.2023.26930
- Coalition for Health AI. Responsible AI Guidance. Coalition for Health AI; 2023. Accessed October 20, 2026. https://www.chai.org/workgroup/responsible-ai/blueprint-for-trustworthy-ai
- OpenAI. Usage Policies. OpenAI; 2025. Accessed October 19, 2026. https://openai.com/policies/usage-policies
- Anthropic. Usage Policy. Anthropic; 2025. Accessed October 19, 2026. https://www.anthropic.com/legal/archive/22742366-2ef0-4c7a-a833-6523f10d3944
- Google. Generative AI Prohibited Use Policy. Google; 2024. Accessed October 19, 2026. https://policies.google.com/terms/generative-ai/use-policy
- Banerjee S, Agarwal A, Singla S. LLMs will always hallucinate, and we need to live with this. In: Lecture Notes in Networks and Systems. Springer Nature Switzerland; 2025:624-648. doi: 10.1007/978-3-031-99965-9_39
- Xu Z, Jain S, Kankanhalli M. Hallucination is Inevitable: An Innate Limitation of Large Language Models. arXiv. Preprint posted online 2024. doi: 10.48550/arXiv.2401.11817
- Karpowicz MP. On the Fundamental Impossibility of Hallucination Control in Large Language Models. arXiv. Preprint posted online 2025. doi: 10.48550/arXiv.2506.06382
- Kalai AT, Vempala SS. Calibrated language models must hallucinate. In: Proceedings of the 56th Annual ACM Symposium on Theory of Computing. ACM; 2024:160-171. doi: 10.1145/3618260.3649777
- Riegel R, Gray A, Luus F, et al. Logical Neural Networks. arXiv. Preprint posted online 2020. doi: 10.48550/arXiv.2006.13155
- van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006;13(2):138-147. doi: 10.1197/jamia.M1809
- Qazi IA, Ali A, Khawaja AU, Akhtar MJ, Sheikh AZ, Alizai MH. Automation Bias in Large Language Model–Assisted Diagnostic Reasoning among Physicians Trained in AI Literacy — A Randomized Clinical Trial. NEJM AI. 2026;3(5). doi: 10.1056/aioa2501001
- Madaan A, Tandon N, Gupta P, et al. Self-Refine: Iterative Refinement with Self-Feedback. In: Advances in Neural Information Processing Systems 36. Neural Information Processing Systems Foundation, Inc. (NeurIPS); 2023:46534-46594. doi: 10.52202/075280-2019
- Huang J, Chen X, Mishra S, et al. Large Language Models Cannot Self-Correct Reasoning Yet. arXiv. Preprint posted online 2023. doi: 10.48550/ARXIV.2310.01798
- Kamoi R, Zhang Y, Zhang N, Han J, Zhang R. When can LLMs actually correct their own mistakes? A critical survey of self-correction of LLMs. Trans Assoc Comput Linguist. 2024;12:1417-1440. doi: 10.1162/tacl_a_00713
- Ning J, Li X, Yu C. Revision or Re-Solving? Decomposing Second-Pass Gains in Multi-LLM Pipelines. arXiv. Preprint posted online 2026. doi: 10.48550/ARXIV.2604.01029
- Gallifant J, Afshar M, Ameen S, et al. The TRIPOD-LLM reporting guideline for studies using large language models. Nat Med. 2025;31(1):60-69. doi: 10.1038/s41591-024-03425-5
- Jin D, Pan E, Oufattole N, Weng WH, Fang H, Szolovits P. What disease does this patient have? A large-scale open domain question answering dataset from medical exams. Appl Sci. 2021;11(14):6421. doi: 10.3390/app11146421
- Hendrycks D, Burns C, Basart S, et al. Measuring Massive Multitask Language Understanding. arXiv. Preprint posted online 2020. doi: 10.48550/arXiv.2009.03300
- Jin Q, Dhingra B, Liu Z, Cohen W, Lu X. PubMedQA: A dataset for biomedical research question answering. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics; 2019:2567-2577. doi: 10.18653/v1/d19-1259
- Baum A, Batniji R, Ratcliffe H, DeGosztonyi M, Basu S. Supporting rising-risk Medicaid patients through early intervention. NEJM Catal Innov Care Deliv. 2024;5(11). doi: 10.1056/CAT.24.0060
- Bedi S, Cui H, Fuentes M, et al. Holistic evaluation of large language models for medical tasks with MedHELM. Nat Med. 2026;32(3):943-951. doi: 10.1038/s41591-025-04151-2
- Guyatt GH, Oxman AD, Vist GE, et al. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. doi: 10.1136/bmj.39489.470347.AD
- Chen S, Gao M, Sasse K, et al. When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior. npj Digit Med. 2025;8(1):605. doi: 10.1038/s41746-025-02008-z
- Angelopoulos AN, Bates S, Fisch A, Lei L, Schuster T. Conformal Risk Control. arXiv. Published online 2022. doi: 10.48550/ARXIV.2208.02814
- Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20(1):37-46. doi: 10.1177/001316446002000104
- Benjamini Y, Hochberg Y. Controlling the false discovery rate: A practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol. 1995;57(1):289-300. doi: 10.1111/j.2517-6161.1995.tb02031.x
- Wilson EB. Probable inference, the law of succession, and statistical inference. J Am Stat Assoc. 1927;22(158):209-212. doi: 10.1080/01621459.1927.10502953
- Cochran WG. The combination of estimates from different experiments. Biometrics. 1954;10(1):101-129. doi: 10.2307/3001666
- McNemar Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika. 1947;12(2):153-157. doi: 10.1007/BF02295996
- Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76(5):378-382. doi: 10.1037/h0031619
- Zhan Z, Cui M, Zhang R. Can Large Language Models Self-Correct in Medical Question Answering? An Exploratory Study. arXiv. Preprint posted online 2026. doi: 10.48550/arXiv.2604.00261
- Stechly K, Valmeekam K, Kambhampati S. On the self-verification limitations of large language models on reasoning and planning tasks. In: The Thirteenth International Conference on Learning Representations (ICLR 2025); 2025:98190-98243.
- Basu S, Huynh B. Mitigating hallucinations in healthcare AI: a systematic review of evidence-based strategies. BMC Health Serv Res. 2026;26(1). doi: 10.1186/s12913-026-14851-1
- Liu S, McCoy AB, Wright A. Improving large language model applications in biomedicine with retrieval-augmented generation: A systematic review, meta-analysis, and clinical development guidelines. J Am Med Inform Assoc. 2025;32(4):605-615. doi: 10.1093/jamia/ocaf008
- Xiong G, Jin Q, Lu Z, Zhang A. Benchmarking retrieval-augmented generation for medicine. In: Findings of the Association for Computational Linguistics ACL 2024. Association for Computational Linguistics; 2024:6233-6251. doi: 10.18653/v1/2024.findings-acl.372
- Kim H, Sohn J, Gilson A, et al. Rethinking Retrieval-Augmented Generation for Medicine: A Large-Scale, Systematic Expert Evaluation and Practical Insights. arXiv. Preprint posted online 2025. doi: 10.48550/arXiv.2511.06738
- Wu K, Wu E, Zou J. ClashEval: Quantifying the tug-of-war between an LLM’s internal prior and external evidence. In: Advances in Neural Information Processing Systems 37. Neural Information Processing Systems Foundation, Inc. (NeurIPS); 2024:33402-33422. doi: 10.52202/079017-1053
- Barnett S, Kurniawan S, Thudumu S, Brannelly Z, Abdelrazek M. Seven failure points when engineering a retrieval augmented generation system. In: Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI. ACM; 2024:194-199. doi: 10.1145/3644815.3644945
- Magesh V, Surani F, Dahl M, Suzgun M, Manning CD, Ho DE. Hallucination-free? Assessing the reliability of leading AI legal research tools. J Empir Leg Stud. 2025;22(2):216-242. doi: 10.1111/jels.12413
- Asai A, Wu Z, Wang Y, Sil A, Hajishirzi H. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In: The Twelfth International Conference on Learning Representations (ICLR 2024); 2024:9112-9141.
- Su H, Wang R, Ji DY, Lee J. RC-RAG: A Reliability-Centered Retrieval-Augmented LLM Framework for PHM Question Answering. IEEE Trans Rel. 2026;75:2159-2173. doi: 10.1109/tr.2026.3702056
- Fox S, Markham L, Lail R, Karotsieris M. LLM Judges Verify Presence, Not Absence: Omission Blindness in AI Clinical Notes and What Recovers It. arXiv. Preprint posted online 2026. doi: 10.48550/ARXIV.2608.31016
