MentaLLaMA: Advancing Mental Health Insights with Instruction-Finetuned Large Language Models

Year : 2025 | Volume : 12 | Issue : 03 | Page : 08 15
    By

    Himanshu Kumar Singh,

  • Bhanu Prakash Lohani,

  1. Student, Department of Computer Science and Engineering, Amity School of Engineering & Technology Amity University, Greater Noida, Uttar Pradesh, India
  2. Assistant Professor, Department of Computer Science and Engineering, Amity School of Engineering & Technology Amity University, Greater Noida, Uttar Pradesh, India

Abstract

The growing prevalence of mental health challenges in contemporary society has highlighted the urgent need for advanced, interpretable, and reliable artificial intelligence solutions that can support mental health assessment and intervention. In response to this need, this research introduces a novel collection of open-source, instruction-tuned large language models (LLMs) specifically designed to facilitate transparent and accurate mental health evaluations. Leveraging a newly developed dataset, which integrates multiple tasks and diverse sources and contains over 105,000 instances, the models were fine-tuned to deliver precise predictions while providing human-level explanations for their outputs across a variety of mental health-related tasks. Experimental results demonstrate that these models achieve high performance in terms of accuracy and consistency, while also displaying notable adaptability when applied to previously unseen tasks. Additionally, this study investigates both single-dataset and multi-dataset fine-tuning strategies to optimize LLMs for domain-specific applications, ensuring they outperform existing approaches in both efficiency and generalization. Despite these promising outcomes, the models are not without limitations, including gaps in domain-specific knowledge and the potential for biased outputs, underscoring the importance of ongoing pre-training and the development of more robust evaluation metrics for future research.

Keywords: Mental health analysis, large language models (LLMs), instruction tuning, multi-task learning, fine-tuning strategies, generalizability in AI, domain-specific AI models, open-source mental health tools

[This article belongs to Recent Trends in Programming languages ]

How to cite this article:
Himanshu Kumar Singh, Bhanu Prakash Lohani. MentaLLaMA: Advancing Mental Health Insights with Instruction-Finetuned Large Language Models. Recent Trends in Programming languages. 2025; 12(03):08-15.
How to cite this URL:
Himanshu Kumar Singh, Bhanu Prakash Lohani. MentaLLaMA: Advancing Mental Health Insights with Instruction-Finetuned Large Language Models. Recent Trends in Programming languages. 2025; 12(03):08-15. Available from: https://journals.stmjournals.com/rtpl/article=2025/view=232660


References

  1. Agrawal M, Hegselmann S, Lang H, Kim Y, Sontag D. Large language models are few-shot clinical information extractors. arXiv preprint arXiv:2205.12689. 2022 May 25.
  2. Markov AA. On a Remarkable Case of Samples Connected in a Chain. Appendix on the statistical investigation of a text by Aksakov. Sci Context. 2006 Dec; 19(4): 601–4.
  3. Pakshina NA. Aleksandr Lyapunov: remembered by his contemporaries. IFAC-PapersOnLine. 2017 Jul 1; 50(1): 5208–18.
  4. Abd-Alrazaq AA, Alajlani M, Ali N, Denecke K, Bewick BM, Househ M. Perceptions and opinions of patients about mental health chatbots: scoping review. J Med Internet Res. 2021 Jan 13; 23(1): e17828.
  5. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN. Attention is all you need. Adv Neural Inf Process Syst. 2017 Dec 4; 30(1): 261–72.
  6. Bommasani R. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. 2021.
  7. Lindberg DA, Humphreys BL, McCray AT. The unified medical language system. Yearb Med Inform. 1993; 2(01): 41–51.
  8. Zhang Y, Sun S, Galley M, Chen YC, Brockett C, Gao X, Gao J, Liu J, Dolan B. Dialogpt: Large-scale generative pre-training for conversational response generation. arXiv preprint arXiv:1911. 00536. 2019 Nov 1.
  9. Liu H. Towards automated psychotherapy via language modeling. arXiv preprint arXiv:2104.10661. 2021 Apr 5.
  10. Gillblad D. Language Models for Everyone—Responsible and Transparent Development of Open Large Language Models. Comput Sci Math Forum (MDPI). 2023 Sep 7; 8(1): 51.
  11. Abid A, Farooqi M, Zou J. Large language models associate Muslims with violence. Nat Mach Intell. 2021 Jun; 3(6): 461–3.
  12. Gao L, Biderman S, Black S, Golding L, Hoppe T, Foster C, Phang J, He H, Thite A, Nabeshima N, Presser S. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027. 2020 Dec 31.
  13. Devlin J, Chang MW, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). 2019 Jun; 4171–4186.
  14. Kenton JD, Toutanova LK. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT. 2019 Jun 2; 1(2).
  15. Abid A, Farooqi M, Zou J. Persistent anti-Muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 2021 Jul 21; 298–306.
  16. Das A, Selek S, Warner AR, Zuo X, Hu Y, Keloth VK, Li J, Zheng WJ, Xu H. Conversational bots for psychotherapy: a study of generative transformer models using domain-specific dialogues. In Proceedings of the 21st workshop on biomedical language processing. 2022 May; 285–297.
  17. Ahmed A, Aziz S, Toro CT, Alzubaidi M, Irshaidat S, Serhan HA, Abd-Alrazaq AA, Househ M. Machine learning models to detect anxiety and depression through social media: A scoping review. Comput Methods Programs Biomed Update. 2022 Jan 1; 2: 100066.
Regular Issue Subscription Original Research
Volume 12
Issue 03
Received 19/06/2025
Accepted 14/07/2025
Published 17/10/2025
Publication Time 120 Days
103

Login


My IP

PlumX Metrics