Advanced Anomaly Detection in Cloud Infrastructures Using Deep Learning Algorithms


Year : 2025 | Volume : 16 | Issue : 01 | Page : 1-11
    By

    Harshvardhan Chunawala,

  • Pratikkumar Chunawala,

  1. Cloud Infrastructure Architect, Amazon Web Services (AWS) – 10 Exchange Place, Jersey City, New Jersey, USA
  2. Principal Cloud Architect, Amazon Web Services (AWS) – 10 Exchange Place, Jersey City, New Jersey, USA

Abstract

It is critical to guarantee the stability and security of cloud environments as cloud computing is becoming the backbone of contemporary IT infrastructures. Neglecting to quickly identify and resolve anomalies, which might point to security breaches, performance problems, or system breakdowns, can lead to disastrous outcomes. The increasing size and complexity of cloud infrastructures are challenging the effectiveness of traditional anomaly detection methods. These approaches often depend on rule-based systems or statistical methodologies. In order to identify anomalies in cloud infrastructures, this study suggests a sophisticated method that makes use of deep learning algorithms. This research presents a framework that makes use of deep learning to identify and categorise abnormalities in real-time using CNNs and RNNs. A variety of indicators, including CPU utilisation, memory consumption, network traffic, and user behaviour patterns, are analysed by the suggested model. It learns to identify regular operating patterns and probable abnormalities by detecting deviations. We tested the framework on large datasets taken from actual cloud settings to see how well it worked. The results show that when compared to conventional approaches, the deep learning-based solution for anomaly identification is much more accurate and faster. In addition to lowering the rate of false positives, the suggested model enhances the identification of complicated and subtle abnormalities that may go unnoticed by more traditional methods. The model’s scalability also makes it well-suited for large-scale deployments, since it can adjust to the ever-changing cloud settings. By developing a powerful and effective method for detecting anomalies, this study adds to the continuing work in cloud security, which improves the stability and dependability of cloud infrastructures.

Keywords: Cloud infrastructures, anomaly detection, deep learning algorithms, CNNs, RNNs, cloud security, real-time monitoring

[This article belongs to Journal of Computer Technology & Applications (jocta)]

How to cite this article:
Harshvardhan Chunawala, Pratikkumar Chunawala. Advanced Anomaly Detection in Cloud Infrastructures Using Deep Learning Algorithms. Journal of Computer Technology & Applications. 2024; 16(01):1-11.
How to cite this URL:
Harshvardhan Chunawala, Pratikkumar Chunawala. Advanced Anomaly Detection in Cloud Infrastructures Using Deep Learning Algorithms. Journal of Computer Technology & Applications. 2024; 16(01):1-11. Available from: https://journals.stmjournals.com/jocta/article=2024/view=190561


References

  1. Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I. Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Future Gener Comput Syst. 2009 Jun 1; 25(6): 599–616.
  2. Mell P. The NIST definition of cloud computing. Recommendations of the National Institute of Standards and Technology. 2011.
  3. Hagemann T, Katsarou K. A systematic review on anomaly detection for cloud computing environments. In: Proceedings of the 2020 3rd Artificial Intelligence and Cloud Computing Conference. 2020 Dec 18; 83–96.
  4. Zhu M, Ye K, Xu CZ. Network anomaly detection and identification based on deep learning methods. In: Cloud Computing–CLOUD 2018: 11th International Conference, Held as Part of the Services Conference Federation, SCF 2018, Seattle, WA, USA, June 25–30, 2018, Proceedings 11. Springer Int Publ; 2018; 219–34.
  5. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015 May 28; 521(7553): 436–44.
  6. Chalapathy R, Chawla S. Deep learning for anomaly detection: A survey. arXiv Preprint arXiv:1901.03407. 2019 Jan 10.
  7. Ranshous S, Shen S, Koutra D, Harenberg S, Faloutsos C, Samatova NF. Anomaly detection in dynamic networks: A survey. Wiley Interdiscip Rev Comput Stat. 2015 May; 7(3): 223–47.
  8. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst. Commun ACM. 2017;60(6):84–90. doi: 10.1145/3065386.
  9. Chhabra GS, Guru A, Rajput BJ, Dewangan L, Swarnkar SK. Multimodal neuroimaging for early Alzheimer’s detection: A deep learning approach. In: Proceedings of the 2023 IEEE 14th International Conference on Computing Communication and Networking Technologies (ICCCNT). 2023 Jul 6; 1–5.
  10. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997; 9(8): 1735–78.
  11. Goldstein M, Uchida S. A comparative evaluation of unsupervised anomaly detection algorithms for multivariate data. PLoS One. 2016 Apr 19; 11(4): e0152173.
  12. Hinton G. Deep learning—a technology with the potential to transform health care. JAMA. 2018 Sep 18; 320(11): 1101–2.
  13. Swarnkar DM, Ambhaikar A. Improved convolutional neural network-based sign language recognition. Int J Adv Sci Technol. 2019 Aug; 27(1): 302–17.
  14. Zhang Q, Yang LT, Chen Z, Li P. A survey on deep learning for big data. Inf Fusion. 2018 Jul 1; 42: 146–57.
  15. Sutton RS, Barto AG. Reinforcement learning: An introduction. Cambridge: MIT Press; 2018 Nov 13.
  16. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, et al. Human-level control through deep reinforcement learning. Nature. 2015 Feb 26; 518(7540): 529–33.
  17. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, et al. Mastering the game of Go with deep neural networks and tree search. Nature. 2016 Jan; 529(7587): 484–9.
  18. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, et al. TensorFlow: A system for large-scale machine learning. In: Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). 2016; 265–83.
  19. Swarnkar SK, Ambhaikar A, Swarnkar VK, Sinha U. Optimized convolution neural network for voice-based sign language recognition: Optimization and regularization. In: Information and Communication Technology for Competitive Strategies (ICTCS 2020). Singapore: Springer; 2022; 633–9.
  20. Bengio Y. Learning deep architectures for AI. Found Trends Mach Learn. 2009; 2(1): 1–127.
  21. Devarajan HR, Balasubramanian S, Swarnkar SK, Kumar P, Jallepalli VR. Deep learning for automated detection of lung cancer from medical imaging data. In: Proceedings of the 2023 IEEE International Conference on Artificial Intelligence for Innovations in Healthcare Industries (ICAIIHI). 2023 Dec 29; 1–5.
  22. Gaikwad VS, Deore SS, Poddar GM, Patil R, Hirolikar DS, Borawake MP, et al. Unveiling market dynamics through machine learning: Strategic insights and analysis. Int J Intell Syst Appl Eng. 2024; 12(14s): 388–97.
  23. Garg S, Kaur K, Kumar N, Rodrigues JJ. Hybrid deep-learning-based anomaly detection scheme for suspicious flow detection in SDN: A social multimedia perspective. IEEE Trans Multimedia. 2019 Jan 16; 21(3): 566–78.
  24. Wang R, Qiu H, Cheng X, Liu X. Anomaly detection with a container-based stream processing framework for industrial internet of things. J Ind Inf Integr. 2023 Oct 1; 35: 100507.
  25. Chen M, Hao Y, Hwang K, Wang L, Wang L. Disease prediction by machine learning over big data from healthcare communities. IEEE Access. 2017 Apr 26; 5: 8869–79.
  26. Dhaygude AD, Varma RA, Yerpude P, Swarnkar SK, Jindal RK, Rabbi F. Deep learning approaches for feature extraction in big data analytics. In: Proceedings of the 2023 10th IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON). 2023 Dec 1; 964–9.
  27. Chkirbene Z, Erbad A, Hamila R, Gouissem A, Mohamed A, Hamdi M. Machine learning-based cloud computing anomalies detection. IEEE Netw. 2020 Sep 1; 34(6): 178–83.
  28. Kim B, Wattenberg M, Gilmer J, Cai C, Wexler J, Viegas F. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In: Proceedings of the International Conference on Machine Learning (PMLR 2018). 2018 Jul 3; 2668–77.
  29. Gunning D, Aha D. DARPA’s explainable artificial intelligence (XAI) program. AI Mag. 2019 Jun 24; 40(2): 44–58.
  30. Swarnkar SK, Dewangan L, Dewangan O, Prajapati TM, Rabbi F. AI-enabled crop health monitoring and nutrient management in smart agriculture. In: Proceedings of the 2023 IEEE 6th International Conference on Contemporary Computing and Informatics (IC3I). 2023 Sep 14; 2679–83.
  31. Agrawal B, Wiktorski T, Rong C. Adaptive real-time anomaly detection in cloud infrastructures. Concurr Comput Pract Exp. 2017 Dec 25; 29(24): e4193.
  32. Hinojosa-Palafox EA, Rodríguez-Elías OM, Hoyo-Montaño JA, Pacheco-Ramírez JH, Nieto-Jalil JM. An analytics environment architecture for industrial cyber-physical systems big data solutions. Sensors. 2021 Jun 23; 21(13): 4282.
  33. Habeeb RA, Nasaruddin F, Gani A, Hashem IA, Ahmed E, Imran M. Real-time big data processing for anomaly detection: A survey. Int J Inf Manag. 2019 Apr 1; 45: 289–307.

Regular Issue Subscription Review Article
Volume 16
Issue 01
Received 18/10/2024
Accepted 08/12/2024
Published 21/12/2024


Loading citations…