Sakshi Londhe,
Smita Palnitkar,
Vishnupriya Kannadkar,
Shubhangi Magar,
- Student Department of Electronics & Telecommunication Engineering, Smt. Kashibai Navale College of Engineering (SKNCOE), Savitribai Phule Pune University, Pune Maharashtra India
- Assistant Professor Department of Electronics & Telecommunication Engineering, Smt. Kashibai Navale College of Engineering (SKNCOE), Savitribai Phule Pune University, Pune Maharashtra India
- Student Department of Electronics & Telecommunication Engineering, Smt. Kashibai Navale College of Engineering (SKNCOE), Savitribai Phule Pune University, Pune Maharashtra India
- Student Department of Electronics & Telecommunication Engineering, Smt. Kashibai Navale College of Engineering (SKNCOE), Savitribai Phule Pune University, Pune Maharashtra India
Abstract
In the deep learning era, image understanding is advancing in sophistication, encompassing both semantic interpretation and the generation of meaningful image descriptions. To achieve this, deep neural networks must undergo specific cross-model training; these networks must be both simple enough to handle a wide range of inputs and complex enough to encode the fine contextual information associated with the image. An appropriate example of the previously described picture comprehension problem is the conversion of a food image to its preparation instructions. This study introduces a novel method that employs cross-model training of CNN, LSTM, and Bi-Directional LSTM to obtain a compressed representation of cooking instructions from a recipe image. Variable instruction length and the number of instructions per recipe is the biggest problem in this. The system addresses challenges such as varying instructions, differing numbers of instructions per recipe, and the presence of multiple food items in the image. The model successfully shrinks instructions with high similarity to original instructions, especially on Indian cuisine data. This model can be useful for information retrieval and automatic recipe recommendations. These results highlight the potential of deep learning in recipe generation from food images, offering a promising solution for culinary enthusiasts seeking convenient access to cooking instruction.
Keywords: Training, image coding, deep learning, context modeling neural network, information retrieval, CNN, LSTM
[This article belongs to Journal of Operating Systems Development & Trends(joosdt)]
References
- Salvador A, Drozdzal M, Giró-i-Nieto X, Romero A. Inverse cooking: Recipe generation from food images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019; 10453–10462.
- Salvador A, Hynes N, Aytar Y, Marin J, Ofli F, Weber I, Torralba A. Learning cross-modal embeddings for cooking recipes and food images. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2017; 3020–3028.
- Gao X, Feng F, He X, Huang H, Guan X, Feng C, Ming Z, Chua TS. Hierarchical attention network for visually-aware food recommendation. IEEE Trans Multimedia. 2019 Oct 4; 22(6): 1647–59.
- Chen JJ, Ngo CW, Chua TS. Cross-modal recipe retrieval with rich food attributes. In Proceedings of the 25th ACM international conference on Multimedia. 2017 Oct 23; 1771–1779.
- Li J, Monroe W, Ritter A, Galley M, Gao J, Jurafsky D. Deep reinforcement learning for dialogue generation. arXiv preprint arXiv:1606.01541. 2016 Jun 5.
- Carvalho M, Cadène R, Picard D, Soulier L, Thome N, Cord M. Cross-modal retrieval in the cooking context: Learning semantic text-image embeddings. In the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 2018 Jun 27; 35–44.
- Fujii T, Sei Y, Tahara Y, Orihara R, Ohsuga A. “Never fry carrots without chopping” Generating Cooking Recipes from Cooking Videos Using Deep Learning Considering Previous Process. Int J Networked Distrib Comput. 2019 Jul; 7(3): 107–12.
- Pinel F, Varshney LR, Bhattacharjya D. A culinary computational creativity system. Computational creativity research: Towards creative machines. Paris: Atlantis Press; 2015; 327–46.
- Teng CY, Lin YR, Adamic LA. Recipe recommendation using ingredient networks. In Proceedings of the 4th annual ACM web science conference. 2012 Jun 22; 298–307.
- Srinivasa CK, Harshitha S, et al. Recipe Finder Web Application Using Convolutional Neural Network. International Journal of Research Publication and Reviews (IJRPR). 2024; 5(5): 10071–10076.
Journal of Operating Systems Development & Trends
| Volume | 11 |
| Issue | 02 |
| Received | July 3, 2024 |
| Accepted | July 31, 2024 |
| Published | August 8, 2024 |