Recipe-fusion: Multimodal Food Recipe Recommendation System

Year : 2024 | Volume : 11 | Issue : 03 | Page : 82 91
    By

    Karthik Anilkumar Nair,

  • Sanjog Gajanan Chavhan,

  • Tejas Nilkanth Pawar,

  • Anmol Jagdish Reddy,

  • K.S. Charumathi,

  1. Student, Department of Computer Engineering, Pillai College of Engineering, New Panvel, Maharashtra, India
  2. Student, Department of Computer Engineering, Pillai College of Engineering, New Panvel, Maharashtra, India
  3. Student, Department of Computer Engineering, Pillai College of Engineering, New Panvel, Maharashtra, India
  4. Student, Department of Computer Engineering, Pillai College of Engineering, New Panvel, Maharashtra, India
  5. Professor, Department of Computer Engineering, Pillai College of Engineering, New Panvel, Maharashtra, India

Abstract

The food recipe recommendation system using data science is a software solution designed to help users discover new and delicious food options based on their food history and other relevant data. This system recommends various recipes based on the input given by the user and it helps to filter out the recipes on course type, diet type, and nature of the food (including non-veg, and veg) using a recommendation technique. The system also analyzes food trends and popular dishes that the users might not have considered before. The system can recognize the input ingredients provided by the user and give the recipe based on the ingredients mentioned. Various machine learning and deep learning approaches like convolutional neural networks (CNN) and recurrent neural networks (RNN) will be used for the implementation of this system. The CNN model is trained and implemented to recommend the recipes based on the image similarity score. The RNN model is trained and implemented for the prediction of the recipes from the input (recipe name) given by the user. Both models (CNN and RNN) are used simultaneously for the implementation of the system, which is why it is named as “Multimodal Food Recipe Recommendation System”. This system will help the person learn and make recipes based on the input given to it. This system will simplify finding relevant recipes for the user.

Keywords: Recommendation, content-based, machine learning, deep learning approaches, convolutional neural network

[This article belongs to Journal of Artificial Intelligence Research & Advances ]

How to cite this article:
Karthik Anilkumar Nair, Sanjog Gajanan Chavhan, Tejas Nilkanth Pawar, Anmol Jagdish Reddy, K.S. Charumathi. Recipe-fusion: Multimodal Food Recipe Recommendation System. Journal of Artificial Intelligence Research & Advances. 2024; 11(03):82-91.
How to cite this URL:
Karthik Anilkumar Nair, Sanjog Gajanan Chavhan, Tejas Nilkanth Pawar, Anmol Jagdish Reddy, K.S. Charumathi. Recipe-fusion: Multimodal Food Recipe Recommendation System. Journal of Artificial Intelligence Research & Advances. 2024; 11(03):82-91. Available from: https://journals.stmjournals.com/joaira/article=2024/view=171716


References

  1. Patel R, Thakkar P, Ukani V. CNNRec: Convolutional neural network-based recommender systems: A survey. Eng Appl Artif Intell. 2024;133:108062. doi:10.1016/j.engappai.2024.108062.
  2. Forouzandeh S, Rostami M, Sheikhpour R. Hfrs-Han: Health-aware food recommendation system based on the heterogeneous attention network. SSRN. 2023. doi:10.2139/ssrn.4378052.
  3. Khilji AFUR, Sinha U, Singh P, Ali A, Dadure P, Manna R, Pakray P. Multimodal recipe recommendation system using deep learning and rule-based approach. SN Comput Sci. 2023;4:421. doi:10.1007/s42979-023-01870-6.
  4. Castiglia G, El Majjodi A, Starke AD, Narducci F, Deldjoo Y, Calò F. Nudging towards health in a conversational food recommender system using multi-modal interactions and nutrition labels. 4th edn of Knowledge-aware and Conversational Recommender Systems (KaRS) Workshop @ RecSys, Seattle, USA, September 2023.
  5. Zhang J, Li M, Liu W, Lauria S, Liu X. Many-objective optimization meets recommendation systems: A food recommendation scenario. Neurocomputing. 2022;503:109-117. doi:10.1016/j.
    2022.06.081.
  6. Jia N, Chen J, Wang R. An attention-based convolutional neural network for recipe recommendation. Expert Syst Appl. 2022;201:116979. doi:10.1016/j.eswa.2022.116979.
  7. Lei Z, Ul Haq AU, Zeb A, Suzauddola M, Zhang D. Is the suggested food your desired?: Multi-modal recipe recommendation with demand-based knowledge graph. Expert Syst Appl. 2021;186:115708. doi:10.1016/j.eswa.2021.115708.
  8. Chavan P, Thoms B, Isaacs J. A recommender system for healthy food choices: Building a hybrid model for recipe recommendations using big data sets. In: Proc 54th Hawaii Int Conf Syst Sci, HICCS-54, CA, USA, Jan 2021. doi:10.24251/HICSS.2021.458.
  9. Manna R, Das D, Gelbukh A. Question-answering and recommendation system on cooking recipes. Computacion y Sistemas. 2021;25:223-235. doi:10.13053/cys-25-1-3899.
  10. Wang W, Duan LY, Jiang H, Jing P, Song X, Nie L. Market2Dish: Health-aware food recommendation. ACM Trans Multimedia Comput Commun Appl. 2021;17:1-19. doi:10.1145/3418211.
  11. Pham HX, Guerrero R, Pavlovic V, Li J. CHEF: Cross-modal hierarchical embeddings for food domain retrieval. In: Proc AAAI Conf Artif Intell. 2021 May 18;35:2423-2430. doi:10.1609/aaai.v35i3.16343.
  12. Deldjoo Y, Schedl M, Hidasi B, Wei Y, He X. Multimedia recommender systems: Algorithms and challenges. In: Recommender Systems Handbook. Springer: New York, USA; 2021. p. 973-1014.
  13. Marin J, Biswas A, Ofli F, Hynes N, Salvador A, Aytar Y, Weber I, Torralba A. Recipe1M+: A dataset for learning cross-modal embeddings for cooking recipes and food images. arXiv Preprint arXiv:1810.06553. 2018 Oct 14.
  14. Xie Z, Liu L, Wu Y, Li L, Zhong L. Learning TFIDF enhanced joint embedding for recipe-image cross-modal retrieval service. IEEE Trans Serv Comput. 2021;15:3304-3316. doi:10.1109/TSC.2021.3098834.
  15. Tian Y, Zhang C, Metoyer R, Chawla NV. Recipe representation learning with networks. In: Proc 30th ACM Int Conf Inf Knowl Manag. 2021 Oct 26;1824-1833. doi:10.1145/3459637.3482468.
  16. Chen Y, Zhou D, Li L, Han JM. Multimodal encoders for food-oriented cross-modal retrieval. In: Web and Big Data. Proc 5th Int Joint Conf, APWeb-WAIM 2021, Guangzhou, China, 23–25 Aug 2021, Vol 5. Springer International Publishing; 2021. p. 253-266.
  17. Pan L, Chen J, Liu S, Ngo CW, Kan MY, Chua TS. A hybrid approach for detecting prerequisite relations in multi-modal food recipes. IEEE Trans Multimedia. 2020;23:4491-4501. doi:10.1109/TMM.2020.3042706.
  18. Pecune F, Callebert L, Marsella S. A recommender system for healthy and personalized recipes recommendations. In: HealthRecSys@ RecSys. 2020 Sep 26;15-20.
  19. Rostami A, Pandey V, Nag N, Wang V, Jain R. Personal food model. In: RT-PA of the 28th ACM Int Conf Multimedia. 2020 Oct 12;4416-4424.
  20. Theodoridis T, Solachidis V, Dimitropoulos K, Daras P. A cross-modal variational framework for food image analysis. In: 2020 IEEE Int Conf Image Process (ICIP). IEEE Publications; 2020. p. 3244-3248. doi:10.1109/ICIP40778.2020.9190758.
  21. Twomey N, Fain M, Ponikar A, Sarraf N. Towards multi-language recipe personalisation and recommendation. In: Proc 14th ACM Conf Recommender Syst. 2020 Sep 22;708-713. doi:10.1145/3383313.3418478.
  22. Lin AS, et al. A recipe for creating multimodal aligned datasets for sequential tasks. arXiv Preprint arXiv:2005.09606. 2020.
  23. Liu A, Yuan S, Zhang C, Luo C, Liao Y, Bai K, Xu Z. Multi-level multimodal transformer network for multimodal recipe comprehension. In: Proc 43rd Int ACM SIGIR Conf Res Dev Inf Retrieval. 2020 Jul 25;1781-1784. doi:10.1145/3397271.3401247.
  24. Meng L, Feng F, He X, Gao X, Chua TS. Heterogeneous fusion of semantic and collaborative information for visually aware food recommendation. In: Proc 28th ACM Int Conf Multimedia. 2020 Oct 12;3460-3468. doi:10.1145/3394171.3413598.
  25. Fu H, Wu R, Liu C, Sun J. Mcen: Bridging cross-modal gap between cooking recipes and dish images with latent variable model. In: Proc IEEE/CVF Conf Comput Vis Pattern Recognit. 2020;14558-14568. doi:10.1109/CVPR42600.2020.01458.
Regular Issue Subscription Review Article
Volume 11
Issue 03
Received 28/06/2024
Accepted 21/08/2024
Published 11/09/2024
404

Login


My IP

PlumX Metrics