Real-time Facial Recognition with Convolutional Neural Networks for Personalized Music Therapy

Year : 2024 | Volume : 11 | Issue : 02 | Page : 67 76
    By

    Shivam Kumar,

  • Rameshber Goswami,

  • Yash Tripathi,

  • Srishti Malu,

  • Anjuli Dubey,

  1. Student, Department of Computer Engineering, Poornima College of Engineering, Jaipur, Rajasthan, India
  2. Student, Department of Computer Engineering, Poornima College of Engineering, Jaipur, Rajasthan, India
  3. Student, Department of Computer Engineering, Poornima College of Engineering, Jaipur, Rajasthan, India
  4. Student, Department of Computer Engineering, Poornima College of Engineering, Jaipur, Rajasthan, India
  5. Assistant Professor, Department of Computer Engineering, Poornima College of Engineering, Jaipur, Rajasthan, India

Abstract

In this project, a web-based application has been developed that integrates computer vision-based facial recognition, multiple algorithms, and machine learning approaches. The given system obtains a user’s emotions in the real-time frame by analyzing facial expressions such as eyes, mouth, the forehead, and so on. It detects emotions like happiness, sadness, that is neutrality, or rock. For a given detected emotion, language, and a user’s chosen artist, the system recommends a song or music that matches a user’s mood. The deep learning model used in the given system is trained on the FER-2013 dataset, which is nothing more than an annotated dataset of facial images. The system is based on real-time video feeds, which recommend an emotional level based on a user’s emotional state. Thus, the presented system is a widely innovative tool that has the potential to revolutionize music consumption and generally improve the user’s state through accurate mood-based music selection. A performance assessment involving a labeled image dataset resulted in a detection accuracy rate of around 81%. Subsequent user studies ensured the system’s ability to recommend music tunes, which accurately reflect the user’s mood and personality.

Keywords: Deep learning, computer vision, deep CNN, facial recognition technology, emotion recognition, music recommendation system, vision, machine learning, convolutional neural network, mood detection

[This article belongs to Journal of Artificial Intelligence Research & Advances ]

How to cite this article:
Shivam Kumar, Rameshber Goswami, Yash Tripathi, Srishti Malu, Anjuli Dubey. Real-time Facial Recognition with Convolutional Neural Networks for Personalized Music Therapy. Journal of Artificial Intelligence Research & Advances. 2024; 11(02):67-76.
How to cite this URL:
Shivam Kumar, Rameshber Goswami, Yash Tripathi, Srishti Malu, Anjuli Dubey. Real-time Facial Recognition with Convolutional Neural Networks for Personalized Music Therapy. Journal of Artificial Intelligence Research & Advances. 2024; 11(02):67-76. Available from: https://journals.stmjournals.com/joaira/article=2024/view=155855


References

  1. Baddur RP, Shedole S. A novel approach for sentiment analysis using deep recurrent networks and sequence modeling. Recent Pat Eng. 2020 Dec 1; 14(3): 403–11.
  2. Ni R, Yang B, Zhou X, Cangelosi A, Liu X. Facial expression recognition through cross-modality attention fusion. IEEE Trans Cogn Develop Syst. 2022 Feb 9; 15(1): 175–85.
  3. Narimisaei J, Naeim M, Imannezhad S, Samian P, Sobhani M. Exploring emotional intelligence in artificial intelligence systems: A comprehensive analysis of emotion recognition and response mechanisms. Ann Med Surg. 2024 Jun 21. 86(8):4657–63.
  4. De Witte M, Pinho AD, Stams GJ, Moonen X, Bos AE, Van Hooren S. Music therapy for stress reduction: a systematic review and meta-analysis. Health Psychol Rev. 2022 Jan 2; 16(1): 134–59.
  5. Yang X, Dong Y, Li J. Review of data features-based music emotion recognition methods. Multimedia systems. 2018 Jul; 24: 365–89.
  6. Sharma VP, Gaded AS, Chaudhary D, Kumar S, Sharma S. Emotion-based music recommendation system. In 2021 IEEE 9th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO). 2021 Sep 3; 1–5.
  7. Sana SK, Sruthi G, Suresh D, Rajesh G, Reddy GS. Facial emotion recognition based music system using convolutional neural networks. Materials Today: Proceedings. 2022 Jan 1; 62: 4699–706.
  8. Vaishnavi G, Sumathi R, Anvitha K, Bathineed D, Nikhitha B, Vanaja K. Music Recommendation Based on Facial Expressions and Mood Detection using CNN. In 2023 IEEE International Conference on Computer Communication and Informatics (ICCCI). 2023 Jan 23; 1–4.
  9. Visnu Dharsini S, Balaji B, Kirubha Hari KS. Music recommendation system based on facial emotion recognition. J Comput Theor Nanosci. 2020 Apr 1; 17(4): 1662–5.
  10. Kullayappa KC, NaikCh.Hima Bindu M.Hari Babu J.Sriram Pavan. Emotion Based Music Recommendation System using CNN. Int J Appl Eng Res. 2020; 5(2): 2666–2795.
  11. Samuvel DJ, Perumal B, Elangovan M. Music recommendation system based on facial emotion recognition. 3C Tecnologia. 2020 Mar 1: 261–71.
  12. Athavle M, Mudale D, Shrivastav U, Gupta M. Music recommendation based on face emotion recognition. J Infor Electr Electron Eng. 2021 Jun 9; 2(2): 1–11.
  13. Mahadik A, Milgir S, Patel J, Jagan VB, Kavathekar V. Mood based music recommendation system. Int J Eng Res Technol. 2021 Jun; 10(06): 553–559.
  14. Joshi S, Jain T, Nair N. Emotion based music recommendation system using LSTM-CNN architecture. In 2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT). 2021 Jul 6; 01–06.
  15. Plichoski GF, Chidambaram C, Parpinelli RS. A face recognition framework based on a pool of techniques and differential evolution. Inf Sci. 2021 Jan 8; 543: 219–41.
  16. Sashank MS, Maddila VS, Krishnasai P, Boddu V, Karthika G. Mood-Based Music Recommendation System Using Facial Expression Recognition and Text Sentiment Analysis. J Theor Appl Inf Technol. 2022 Oct 15; 100(19): 5667–5674.
  17. Supriya LP, Khilar R. Affective music player for multiple emotion recognition using facial expressions with SVM. In 2021 IEEE 5th international conference on I-SMAC (IoT in social, Mobile, analytics and cloud) (I-SMAC). 2021 Nov 11; 622–626.
  18. Eliyajer G, Natarajan B, Bhuvaneswari R, Elakkiya R. A Novel Approach for Song Recommendation System Using Deep Neural Networks. In 2023 IEEE World Conference on Applied Intelligence and Computing (AIC). 2023 Jul 29; 382–387.
  19. Shabu SJ, Janaardhan C, Bhaskar K, Mary AV, Refonaa J, Dhamodaran S. Music Recommendation System based on Facial Expression. In 2023 IEEE 4th International Conference on Electronics and Sustainable Communication Systems (ICESC). 2023 Jul 6; 908–912.
  20. Gupta M, Venkatesh SN, Suraskar R, Praveena K, Jegadesan S, Suneetha S. Enhancing Music Recommendations with Emotional Insight: A Facial Expression Approach in AI. In 2023 7th IEEE International Conference on Electronics, Communication and Aerospace Technology (ICECA). 2023 Nov 22; 450–457.
  21. Chen X, Tang TY. Combining content and sentiment analysis on lyrics for a lightweight emotion-aware Chinese song recommendation system. In Proceedings of the 2018 10th International Conference on Machine Learning and Computing. 2018 Feb 26; 85–89.
Regular Issue Subscription Review Article
Volume 11
Issue 02
Received 06/05/2024
Accepted 30/06/2024
Published 10/07/2024
396

Login


My IP

PlumX Metrics