Evaluating the Efficiency of LLMs-SA (Sentiment Analysis) via Social Media Texts

Notice

This is an unedited manuscript accepted for publication and provided as an Article in Press for early access at the author’s request. The article will undergo copyediting, typesetting, and galley proof review before final publication. Please be aware that errors may be identified during production that could affect the content. All legal disclaimers of the journal apply.

Year : 2025 | Volume : 16 | 03 | Page :
    By

    Nilesh Jain,

  1. Associate Professor, Department of Computer Science and Applications, Mandsaur University, Mandsaur, Madhya Pradesh, India

Abstract

Sentiment analysis (SA) is becoming popular in business and scientific communities as the processing of natural language (NLP), computational linguistics, text analytics, image-based processing or video- based processing is used in extracting and mining subjective information in the web, social network, etc. It is able to detect positive, negative or neutral information and can be selected to absorb polarity, sentiments, urgency and goals of mount importance. The majority of the world’s data is unstructured, and this makes it challenging to determine the feelings of the people. This paper outlines a sentiment taxonomy based on a transformer-type large language model, RoBERTa, to classify textual data from tweets. The process includes data gathering, tokenization, feature extraction with the help of the Bag of Words approach, and classification with the RoBERTa model. The provided RoBERTa model has been experimentally evaluated using a Twitter dataset, and results show that it outperforms well- known classifiers such as Support Vector Classifier (SVC) and Naive Bayes (NB) with accuracies of 99.9% and F1-score of 99.89%. The outcomes support the usefulness of the model in contextual semantics on informal and noisy social media text. The proposed solution will provide an effective and scalable mechanism to analyze sentiments in real time in different areas, such as business intelligence, political analysis, and health monitoring.

Keywords: Sentiment analysis, social media, RoBERTa, large language models (LLMs), text classification, natural language processing (NLP), machine learning, deep learning, twitter data

How to cite this article:
Nilesh Jain. Evaluating the Efficiency of LLMs-SA (Sentiment Analysis) via Social Media Texts. Journal of Computer Technology & Applications. 2025; 16(03):-.
How to cite this URL:
Nilesh Jain. Evaluating the Efficiency of LLMs-SA (Sentiment Analysis) via Social Media Texts. Journal of Computer Technology & Applications. 2025; 16(03):-. Available from: https://journals.stmjournals.com/jocta/article=2025/view=228386


References

  1. Drus Z, Khalid H. Sentiment analysis in social media and its application: Systematic literature review. Procedia computer science. 2019 Jan 1;161:707-14.
  2. McGarry K. Analyzing social media data using sentiment mining and bigram analysis for the recommendation of youtube videos. Information. 2023 Jul 16;14(7):408.
  3. Chaudhary JK, Tyagi S, Sharma HP, Akram SV, Sisodia DR, Kapila D. Machine learning model- based financial market sentiment prediction and application. In2023 3rd international conference on advance computing and innovative technologies in engineering (ICACITE) 2023 May 12 (pp. 1456-1459). IEEE.
  4. Ranjan M, Tiwari S, Sattar AM, Tatkar NS. A new approach for carrying out sentiment analysis of social media comments using natural language processing. Engineering Proceedings. 2024 Jan 17;59(1):181.
  5. RS NN, Priyadarshini J achine earning lgorithms for the diagnosis of lzheimer’s and Parkinson’s Disease International Journal Jul; ( )
  6. Rongala S, Pahune SA, Velu H, Mathur S. Leveraging Natural Language Processing and Machine Learning for Consumer Insights from Amazon Product Reviews. In2025 3rd International Conference on Smart Systems for applications in Electrical Sciences (ICSSES) 2025 Mar 21 (pp. 1-6). IEEE.
  7. Aryal SK, Prioleau H, Aryal S. Sentiment analysis across multiple african languages: A current benchmark. arXiv preprint arXiv:2310.14120. 2023 Oct 21.
  8. Pandya S. Comparative Analysis of Large Language Models and Traditional Methods for Sentiment Analysis of Tweets Dataset. Int. J. Innov. Sci. Res. Technol. 2024;9(12):1647-57.
  9. Taherdoost H, Madanchian M. Artificial intelligence and sentiment analysis: A review in competitive research. Computers. 2023 Feb 7;12(2):37.
  10. Pahune S, Chandrasekharan M. Several categories of large language models (llms): A short survey. arXiv preprint arXiv:2307.10188. 2023 Jul 5.
  11. Deokar OR, Gaikwad TR, Gawali KR, Ghanghav PD, Shivarkar SA, Muzumdar AA. Social Media Sentiment Analysis Using Machine Learning. In2025 3rd International Conference on Disruptive Technologies (ICDT) 2025 Mar 7 (pp. 89-93). IEEE.
  12. Gupta Y. Influence of Feature Selection Techniques for Social Media Data Analysis (Text and Image). In2025 17th International Conference on Knowledge and Smart Technology (KST) 2025 Feb 26 (pp. 88-91). IEEE.
  13. Lee S. Social media text sentiment classification based on bag-of-words models with multiple machine learning algorithms. In2024 5th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) 2024 Mar 29 (pp. 1775-1779). IEEE.
  14. Hidayani N, Mantoro T, Ayu MA. Deep learning model for sentiment analysis in the use of informal language and slang on social media. In2024 10th International Conference on Computing, Engineering and Design (ICCED) 2024 Dec 11 (pp. 1-5). IEEE.
  15. Mridula B, Juliet AH, Legapriyadharshini N. Deciphering social media sentiment for enhanced analytical accuracy: Leveraging random forest, KNN, and Naive Bayes. In2024 10th International Conference on Communication and Signal Processing (ICCSP) 2024 Apr 12 (pp. 1410-1415). IEEE.
  16. Pardeshi SM, Jain DC, Wagh RB. A Qualified Study of Machine Learning Algorithms for Sentiment Analysis in Social Media. In2023 10th IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON) 2023 Dec 1 (Vol. 10, pp. 577-581). IEEE.
  17. Kurniawan I, Nurhayati IK, Fahreza MY, Ikhsan N, Lhaksmana KM. Imbalanced Text Classification based on Corporate Culture by using Support Vector Machine, Case Study: PT XYZ, Indonesia. In2022 5th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI) 2022 Dec 8 (pp. 425-429). IEEE.
  18. Alqahtani A, Khan SB, Alqahtani J, AlYami S, Alfayez F. Sentiment analysis of semantically interoperable social media platforms using computational intelligence techniques. Applied Sciences. 2023 Jun 27;13(13):7599.
  19. Mostafa AM, Aljasir M, Alruily M, Alsayat A, Ezz M. Innovative forward fusion feature selection algorithm for sentiment analysis using supervised classification. Applied Sciences. 2023 Feb 5;13(4):2074.
  20. Ghatora PS, Hosseini SE, Pervez S, Iqbal MJ, Shaukat N. Sentiment analysis of product reviews using machine learning and pre-trained llm. Big Data and Cognitive Computing. 2024 Dec 23;8(12):199.
  21. Tan KL, Lee CP, Lim KM. Roberta-gru: A hybrid deep learning model for enhanced sentiment analysis. Applied Sciences. 2023 Mar 19;13(6):3915.
  22. Kaur G, Sharma A. A deep learning-based model using hybrid feature extraction approach for consumer sentiment analysis. Journal of big data. 2023 Jan 13;10(1):5.
  23. Padhy M, Modibbo UM, Rautray R, Tripathy SS, Bebortta S. Application of Machine Learning Techniques to Classify Twitter Sentiments Using Vectorization Techniques, Preprints.org, 2024, 20 August;pp-1:19.
Ahead of Print Subscription Review Article
Volume 16
03
Received 07/07/2025
Accepted 14/07/2025
Published 29/09/2025
Publication Time 84 Days
61


My IP

PlumX Metrics