AI-News 4.0: Most Suitable LLM for UPSC Aspirants

Year : 2024 | Volume : 02 | Issue : 02 | Page : 47 53
    By

    Priyansh Joshi,

  1. Student, Department of Computer Science, Shri Gujrati Samaj Ajmera Mukesh Nemichand Bhai School, Indore, Madhya Pradesh, India

Abstract

In recent years, the domain of interactive artificial intelligence (AI) has experienced a significant surge with large language models (LLMs) at the forefront of this evolution. AI systems, including those based on the GPT-3.5 framework, have been engineered to address various tasks such as responding to intricate inquiries, participating in conversations, and executing sophisticated natural language processing (NLP) operations. A prominent LLM, known for its adaptability, prompts an essential inquiry: is it capable of serving as a full-time assistant, particularly in demanding contexts such as Union Public Service Commission (UPSC) examination preparation? This investigation aims to evaluate the capabilities of LLM-based systems as potential aids for UPSC candidates by scrutinizing their competencies in areas such as subject knowledge, practice examinations, contemporary affairs interpretation, and response composition abilities. The study is further contextualized by the United Nations Educational, Scientific and Cultural Organization (UNESCO) Report on AI in Higher Education (2023), which addresses crucial aspects including Pedagogy and Learning, Scientific Inquiry, Academic Honesty, Data Protection Concerns, Gender Equality and Inclusivity, and Regulatory Shortcomings. A range of assessments was conducted to gauge the LLM’s aptitude in addressing questions pertinent to the Union Public Service Commission (UPSC) curriculum, encompassing topics from general studies, elective subjects, ethics, and composition writing. The evaluation considered factors such as precision, analytical thoroughness, and capacity to engage in subtle discussions essential for UPSC preparation. Our results indicate that, while LLMs exhibit significant promise as UPSC study aids, they face several constraints. For instance, they occasionally lack the analytical depth necessary for essay composition and struggle with real-time updates of current events. The research concludes by emphasizing these limitations and stressing the necessity for further enhancements in LLMs to improve their efficacy as full-time assistants during UPSC exam preparation.

Keywords: LLMs, conversational AI, summarization, newspapers, natural language processing (NLP), UPSC examination

[This article belongs to International Journal of Computer Science Languages ]

How to cite this article:
Priyansh Joshi. AI-News 4.0: Most Suitable LLM for UPSC Aspirants. International Journal of Computer Science Languages. 2024; 02(02):47-53.
How to cite this URL:
Priyansh Joshi. AI-News 4.0: Most Suitable LLM for UPSC Aspirants. International Journal of Computer Science Languages. 2024; 02(02):47-53. Available from: https://journals.stmjournals.com/ijcsl/article=2024/view=178917


References

  1. Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, et al. Language models are few-shot learners. [Preprint] arXiv:2005.14165 [cs.CL]. 2020. DOI: 10.48550/arXiv.2005.14165.
  2. Wang S, Xu T, Li H, Zhang C, Liang J, Tang J, et al. Large language models for education: a survey and outlook. [Preprint]. arXiv:2403.18105 [cs.CL]. 2024. DOI: 10.48550/arXiv.2403.18105.
  3. Harrer S. Attention is not all you need: The complicated case of ethically using large language models in healthcare and medicine. EBioMedicine. 2023;90. DOI: 10.1016/j.ebiom.2023.104512.
  4. Yang Z, Gan Z, Wang J, Hu X, Lu Y, Liu Z, et al. An empirical study of GPT-3 for few-shot knowledge-based VQA. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2022 Jun 28;36(3):3081–9. DOI: 10.1609/aaai.v36i3.20215.
  5. Chen Y, Liu Y, Yan J, Bai X, Zhong M, Yang Y, et al. See what LLMs cannot answer: a self-challenge framework for uncovering LLM weaknesses. [Preprint]. arXiv:2408.08978 [cs.CL]. 2024. DOI: 10.48550/arXiv.2408.08978.
  6. Li H, Appleby G, Suh A. A preliminary roadmap for LLMs as assistants in exploring, analyzing, and visualizing knowledge graphs. [Preprint] arXiv:2404.01425 [cs.HC]. 2024. DOI: 10.48550/arXiv.2404.01425.
  7. Honda H, Hagiwara M. Deep-learning-based fuzzy symbolic processing with agents capable of knowledge communication. In: Proceedings of the 14th International Conference on Agents and Artificial Intelligence. SCITEPRESS – Science and Technology Publications; 2022. p. 172–9. DOI: 10.5220/0010796300003116.
  8. Bender EM, Gebru T, McMillan-Major A, Shmitchell S. On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. New York (NY): Association for Computing Machinery; 2021. p. 610-23. DOI: 10.1145/3442188.3445922.
  9. Sánchez-Adame LM, Mendoza S, Urquiza J, Rodríguez J, Meneses-Viveros A. Towards a set of heuristics for evaluating chatbots. IEEE Latin America Transactions. 2021;19:2037–45. DOI: 10.1109/TLA.2021.9480145.
  10. Susnjak T, Maddigan P. Forecasting patient demand at urgent care clinics using explainable machine learning. CAAI Transactions on Intelligence Technology. 2023;8:712–33. DOI: 10.1049/cit2.12258.
  11. Joosten J, Bilgram V, Hahn A, Totzek D. Comparing the ideation quality of humans with generative artificial intelligence. IEEE Engineering Management Review. 2024;52:153–64. DOI: 10.1109/EMR.2024.3353338.
  12. Wu T, He S, Liu J, Sun S, Liu K, Han QL, et al. A brief overview of ChatGPT: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica. 2023;10:1122–36. DOI: 10.1109/JAS.2023.123618.
  13. Casheekar A, Lahiri A, Rath K, Prabhakar KS, Srinivasan K. A contemporary review on chatbots, AI-powered virtual conversational agents, ChatGPT: Applications, open challenges and future research directions. Computer Science Review. 2024;52:100632. DOI: 10.1016/j.cosrev.2024.100632.
Regular Issue Subscription Review Article
Volume 02
Issue 02
Received 11/10/2024
Accepted 19/10/2024
Published 21/10/2024
570

Login


My IP

PlumX Metrics