Sujal Gupta,
Hardik Sharma,
Shivansh Narayan,
Dhruv Kumar,
Aayush Bisht,
- Student, Department of CSE, Greater Noida Institute of Technology, Greater Noida, Uttar Pradesh, India
- Student, Department of CSE, Greater Noida Institute of Technology, Greater Noida, Uttar Pradesh, India
- Student, Department of CSE, Greater Noida Institute of Technology, Greater Noida, Uttar Pradesh, India
- Student, Department of CSE, Greater Noida Institute of Technology, Greater Noida, Uttar Pradesh, India
- Student, Department of CSE, Greater Noida Institute of Technology, Greater Noida, Uttar Pradesh, India
Abstract
Large Language Models can generate fluent, contextually appropriate text across a range of NLP tasks, but they frequently produce outputs that are factually wrong while sounding confident and plausible. This problem, referred to as hallucination, poses serious risks in domains where accuracy matters. We propose a post-processing framework that detects hallucinated responses by comparing them against verified reference text using sentence embeddings. The system computes cosine similarity between the response and reference, and extracts four additional features: named entity mismatches, numerical inconsistencies, keyword overlap, and output perplexity. These features are used to train three classifiers: Logistic Regression, SVM, and Random Forest. Random Forest achieved the best results with 87% accuracy and 0.84 F1-score. The framework requires no access to the language model’s internals and works as a standalone verification layer, which makes it applicable to any LLM including API-based services. To evaluate the robustness of the proposed approach, experiments were conducted on a benchmark dataset containing both factual and hallucinated responses across diverse domains. The results demonstrate that combining semantic similarity with linguistic and statistical features significantly improves hallucination detection compared to relying on embedding similarity alone. The proposed framework is computationally efficient, easily deployable, and scalable, making it suitable for integration into real-world applications where reliable and trustworthy AI-generated content is essential.
Keywords: Large language models, hallucination detection, semantic consistency, machine learning, generative AI, AI reliability, natural language processing.
[This article belongs to Emerging Trends in Languages ]
References
- Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, Ishii E, Bang YJ, Madotto A, Fung P. Survey of hallucination in natural language generation. ACM computing surveys. 2023 Mar 3;55(12):1-38.
- Maynez J, Narayan S, Bohnet B, McDonald R. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th annual meeting of the association for computational linguistics 2020 Jul (pp. 1906-1919).
- Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, Küttler H, Lewis M, Yih WT, Rocktäschel T, Riedel S. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems. 2020; 33:9459-74.
- Roller S, Dinan E, Goyal N, Ju D, Williamson M, Liu Y, Xu J, Ott M, Smith EM, Boureau YL, Weston J. Recipes for building an open-domain chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume 2021 Apr (pp. 300-325).
- Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Advances in neural information processing systems. 2017;30 5998–6008
- Devlin J, Chang MW, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers) 2019 Jun (pp. 4171-4186).
- Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S. Language models are few-shot learners. Advances in neural information processing systems. 2020; 33:1877-901.
- Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, Ishii E, Bang YJ, Madotto A, Fung P. Survey of hallucination in natural language generation. ACM computing surveys. 2023 Mar 3;55(12):1-38.
- Farquhar S, Kossen J, Kuhn L, Gal Y. Detecting hallucinations in large language models using semantic entropy. Nature. 2024 Jun 20;630(8017):625-30.
- Manakul P, Liusie A, Gales M. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 conference on empirical methods in natural language processing 2023 Dec (pp. 9004-9017).
- Yang B, Al Mamun MA, Zhang JM, Uddin G. Hallucination detection in large language models with metamorphic relations. Proceedings of the ACM on Software Engineering. 2025 Jun 19;2(FSE):425-45.
- Reimers N, Gurevych I. Sentence-bert: Sentence embeddings using siamese bert-networks. InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) 2019 Nov (pp. 3982-3992).
- Sun Z, Zang X, Zheng K, Xu J, Zhang X, Yu W, Song Y, Li H. Redeep: Detecting hallucination in retrieval-augmented generation via mechanistic interpretability. In International Conference on Learning Representations 2025 May 1 (Vol. 2025, pp. 50250-50279).
Emerging Trends in Languages
| Volume | 03 | |
| Issue | 02 | |
| Received | 29/06/2026 | |
| Accepted | 04/07/2026 | |
| Published | 04/07/2026 | |
| Publication Time | 5 Days |