This is an unedited manuscript accepted for publication and provided as an Article in Press for early access at the author’s request. The article will undergo copyediting, typesetting, and galley proof review before final publication. Please be aware that errors may be identified during production that could affect the content. All legal disclaimers of the journal apply.
Nadia Tahseen,
Aneeza,
- Student, Department of Computer Science, University of Engineering & Technology, Lahore, Pakistan
- Student, Department of Computer Science, University of Engineering & Technology, Lahore, Pakistan
Abstract
A Large Language Model (LLM) is likely to produce sentences that are fluent and confident in many instances, but completely wrong. In many cases a Large Language Model (LLM) will produce a fluent and confident sentence that is completely incorrect. This review tries to analyze this phenomenon using the most recent literature regarding natural language generation, computational learning theory and benchmark experiments. The argument is that it is not a problem with the engineering but that such problems are inevitable consequences of how LLM’s are trained; predicting what is likely to continue next rather than checking that it is factually correct and that testing on benchmark datasets can reinforce confidence when training LLM’s, which encourages them to be more confident than they may be due. This review looks at factors contributing to hallucination at every phase of an LLM’s life cycle—data curation, training goals and reinforcement learning, and inference-time generation. Next, the most important families of methods for detecting and mitigating the problem (retrieval augmented generation, self-consistency and sampling checking, and calibrating model outputs) are discussed, and examples are provided of how they can be used effectively in practice. This discussion takes place in the context of actual examples using fictionalized legal references and fictionalized biographies to illustrate how believable hallucinations might be mistaken for facts. The review ends with an analysis of domain-specific consequences of hallucination in domains like medicine, legal practice, journalism, and teaching, where precision in facts cannot be compromised, and some open questions for research, such as the need for improved benchmarking, multilingual hallucination, and the trade-off between calibrated honesty and desired certainty of response. All in all, the review illustrates the need for novel training and scoring approaches, rather than just bigger models, to tackle problem of hallucination.
Keywords: Large Language Models (LLMs); Hallucination; Artificial Intelligence; Natural Language Generation; Retrieval-Augmented Generation (RAG); AI Reliability; Model Calibration; Factuality; Machine Learning; Trustworthy AI
References
- Kalai AT, Nachum O, Vempala SS, Zhang E. Why language models hallucinate. arXiv preprint arXiv:2509.04664. 2025 Sep 4.
- Bani-Harouni D, Pellegrini C, Stangel P, Özsoy E, Zaripova K, Navab N, Keicher M. Rewarding doubt: A reinforcement learning approach to calibrated confidence expression of large language models. InInternational Conference on Learning Representations 2026 Apr 20 (Vol. 2026, pp. 117927-117943).
- Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, Ishii E, Bang YJ, Madotto A, Fung P. Survey of hallucination in natural language generation. ACM computing surveys. 2023 Mar 3;55(12):1-38.
- Huang L, Yu W, Ma W, Zhong W, Feng Z, Wang H, Chen Q, Peng W, Feng X, Qin B, Liu T. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM transactions on information systems. 2025 Jan 24;43(2):1-55.
- Lin S, Hilton J, Evans O. Truthfulqa: Measuring how models mimic human falsehoods. InProceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers) 2022 May (pp. 3214-3252).
- Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, Küttler H, Lewis M, Yih WT, Rocktäschel T, Riedel S. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems. 2020;33:9459-74.
- Manakul P, Liusie A, Gales M. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. InProceedings of the 2023 conference on empirical methods in natural language processing 2023 Dec (pp. 9004-9017).
- Alansari A, Luqman H. Large language models hallucination: A comprehensive survey. Computer Science Review. 2026 Aug 1;61:100970, 10.1016/j.cosrev.2026.100970.
- Kandpal N, Deng H, Roberts A, Wallace E, Raffel C. Large language models struggle to learn long-tail knowledge. InInternational conference on machine learning 2023 Jul 3 (pp. 15696-15707). PMLR.
- Kadavath S, Conerly T, Askell A, Henighan T, Drain D, Perez E, Schiefer N, Hatfield-Dodds Z, DasSarma N, Tran-Johnson E, Johnston S. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221. 2022 Jul 11.
Journal of Artificial Intelligence Research & Advances
| Volume | 13 | |
| 03 | ||
| Received | 22/07/2026 | |
| Accepted | 22/08/2026 | |
| Published | 25/09/2026 | |
| Publication Time | 65 Days |