Adversarial Attacks on Machine Learning Models in Cybersecurity: A Systematic Literature Review

Year : 2025 | Volume : 13 | Issue : 01 | Page : 23 38
    By

    Sanjay Singh,

  • Anamika Gupta,

  1. Pg Cybersecurity Student, Department of Computer Science, Shaheed Sukhdev College of Business Studies, University of Delhi, New Delhi, Delhi, India
  2. Professor, Department of Computer Science, Shaheed Sukhdev College of Business Studies, University of Delhi, New Delhi, Delhi, India

Abstract

Adversarial machine learning (AML) is a field that is growing swiftly, especially as machine learning models are employed more and more in places where security is critical. This review goes into great depth over 746 publications from the Scopus database, with an emphasis on the connection between AML and network security. Using Biblioshiny and Scopus tools, we looked at trends in publications, study fields, productive authors, collaboration networks, and theme concentrations. Thirteen charts show how AML research has developed over time by highlighting notable contributions, main journals, popular keywords, and citation trends. Our research demonstrates that the topic is multidisciplinary and has research centers all around the world. It also shows that scholars are not working together as much anymore and that the focus is moving from pure attack modeling to robustness and interpretability. At the end of the study, some major gaps are pointed out, such as the clichés about real correspondent implementation and ethical governance. The report also offers approaches for future research to make the AML research ecosystem stronger and more accessible to everyone.

Keywords: Adversarial attack, adversarial machine learning, machine learning models, machine learning security, network security

[This article belongs to Journal of Artificial Intelligence Research & Advances ]

How to cite this article:
Sanjay Singh, Anamika Gupta. Adversarial Attacks on Machine Learning Models in Cybersecurity: A Systematic Literature Review. Journal of Artificial Intelligence Research & Advances. 2025; 13(01):23-38.
How to cite this URL:
Sanjay Singh, Anamika Gupta. Adversarial Attacks on Machine Learning Models in Cybersecurity: A Systematic Literature Review. Journal of Artificial Intelligence Research & Advances. 2025; 13(01):23-38. Available from: https://journals.stmjournals.com/joaira/article=2025/view=233970


References

  1. Babatunde LA, Etim ED, Essien IA, Cadet E, Ajayi JO, Erigha ED, et al. Adversarial machine learning in cybersecurity: Vulnerabilities and defense strategies. J Front Multidiscip Res. 2020;1:31–45. doi:10.54660/.JFMR.2020.1.2.31-45.
  2.  Eykholt K, Evtimov I, Fernandes E, Li B, Rahmati A, Xiao C, et al. Robust physical-world attacks on deep learning visual classification. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. 2018. p. 1625–1634. doi:10.1109/CVPR.2018.00175.
  3.  Biggio B, Roli F. Wild patterns: Ten years after the rise of adversarial machine learning. In: Proc ACM SIGSAC Conf Comput Commun Secur. 2018. p. 2154–2156. doi:10.1145/3243734.3264418.
  4. Akhtar N, Mian A. Threat of adversarial attacks on deep learning in computer vision: A survey. IEEE Access. 2018;6:14410–14430. doi:10.1109/ACCESS.2018.2807385.
  5.  Zhang J, Hu J, Liu J. Neural network with multiple connection weights. Pattern Recognit. 2020;107:107481. doi:10.1016/j.patcog.2020.107481.
  6.  Chen X, Liu C, Li B, Lu K, Song D. Targeted backdoor attacks on deep learning systems using data poisoning. [Preprint]. 2017. arXiv:1712.05526. doi:10.48550/arXiv.1712.05526.
  7. Dilhara M, Ketkar A, Dig D. Understanding software-2.0: a study of machine learning library usage and evolution. ACM Trans Softw Eng Methodol. 2021 Oct;30(4):55. doi:10.1145/3453478.
  8. Hendrycks D, Gimpel K. A baseline for detecting misclassified and out-of-distribution examples in neural networks. [Preprint]. 2016. arXiv:1610.02136. doi:10.48550/arXiv.1610.02136.
  9. Dong Y, Liao F, Pang T, Su H, Zhu J, Hu X, Li J. Boosting adversarial attacks with momentum. In: Proc IEEE Conf Comput Vis Pattern Recognit (CVPR). 2018. p. 9185–9193. doi:10.1109/CVPR.2018.00957.
  10. He J, Li X, Liao L, Song D, Cheung W. Inferring a personalized next point-of-interest recommendation model with latent behavior patterns. Proc AAAI Conf Artif Intell. 2016;30(1). doi:10.1609/aaai.v30i1.9994.
  11. Gu T, Dolan-Gavitt B, Garg S. Badnets: Identifying vulnerabilities in the machine learning model supply chain. [Preprint]. 2017. arXiv:1708.06733. doi:10.48550/arXiv.1708.06733.
  12. Liu Y, Chen X, Liu C, Song D. Delving into transferable adversarial examples and black-box attacks. [Preprint]. 2016. arXiv:1611.02770. doi:10.48550/arXiv.1611.02770.
  13.  Carlini N, Wagner D. Towards evaluating the robustness of neural networks. Proc IEEE Symp Secur Priv (SP). 2017;39–57. doi:10.1109/SP.2017.49.
  14.  Demontis A, Melis M, Pintor M, Jagielski M, Biggio B, Oprea A, et al. Why do adversarial attacks transfer? Explaining transferability of evasion and poisoning attacks. In: Proceedings of the 28th USENIX Security Symposium (USENIX Security 19). Berkeley (CA): USENIX Association; 2019. p. 321–338.
  15. Wang X, Li J, Kuang X, Tan YA, Li J. The security of machine learning in an adversarial setting: A survey. J Parallel Distrib Comput. 2019;130:12–23. doi:10.1016/j.jpdc.2019.03.003
  16.  Ilyas A, Santurkar S, Tsipras D, Engstrom L, Tran B, Madry A. Adversarial examples are not bugs, they are features. In: Wallach HM, Larochelle H, Beygelzimer A, d’Alché-Buc F, Fox E, Garnett R, editors. Proceedings of the 33rd International Conference on Neural Information Processing Systems (NeurIPS 2019); 2019 Dec 8–14; Vancouver, BC, Canada. Red Hook (NY): Curran Associates Inc.; 2019. P. 125–136.
  17. Jagielski M, Oprea A, Biggio B, Liu C, Nita-Rotaru C, Li B. Manipulating machine learning: Poisoning attacks and countermeasures for regression learning. 2018 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA. 2018. p. 19–35. doi:10.1109/SP.2018.00057.
  18. Li Y, Huang H, Guo X, Yuan Y. An empirical study on group fairness metrics of judicial data. IEEE Access. 2021;9:149043–149049. doi:10.1109/ACCESS.2021.3122443.
  19. Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A. Towards deep learning models resistant to adversarial attacks. [Preprint]. 2017. arXiv:1706.06083. doi:10.48550/arXiv.1706.06083.
  20.  Nasr M, Shokri R, Houmansadr A. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. 2019 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA. 2019. p. 739–753. doi:10.1109/SP.2019.00065.
  21. Kurita K, Vyas N, Pareek A, Black AW, Tsvetkov Y. Measuring bias in contextualized word representations. In: Costa-jussà MR, Hardmeier C, Radford W, Webster K, editors. Proceedings of the First Workshop on Gender Bias in Natural Language Processing; 2019 Aug; Florence, Italy. Stroudsburg (PA): Association for Computational Linguistics; 2019. p. 166–172. doi:10.18653/v1/W19-3823.
  22. Su J, Vargas DV, Sakurai K. One pixel attack for fooling deep neural networks. IEEE Trans Evol Comput. 2019;23:828–841. doi:10.1109/TEVC.2019.2890858.
  23. Li J, Gao J, Jiang Q, He G. Adversarial defense networks via Gaussian noise and RBF. In: Sun X, Zhang X, Xia Z, Bertino E, editors. Artificial Intelligence and Security. ICAIS 2021. Lecture Notes in Computer Science. Vol. 12736. Cham: Springer; 2021. p. 480–491. doi:10.1007/978-3-030-78609-0_42.
  24.  Xiao C, Li B, Zhu JY, He W, Liu M, Song D. Generating adversarial examples with adversarial networks. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI-18). Stockholm, Sweden: International Joint Conferences on Artificial Intelligence Organization; 2018. p. 3905-3911. doi:10.24963/ijcai.2018/543.
  25.  Sadeghi AR, Wachsmann C, Waidner M. Security and privacy challenges in industrial Internet of Thin s n: Proceedin s of the 5 nd Annual esi n Automation Conference AC ‘ 5); 5 un; San Francisco, CA, USA. New York (NY): Association for Computing Machinery; 2015. Article 54, 6 p. doi:10.1145/2744769.2747942.
  26. Cheng G, Sun X, Li K, Guo L, Han J. Perturbation-seeking generative adversarial networks: A defense framework for remote sensing image scene classification. IEEE Trans Geosci Remote Sens. 2022;60:1–11. doi:10.1109/TGRS.2021.3081421.
  27. Alfakih T, Hassan MM, Gumaei A, Savaglio C, Fortino G. Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA. IEEE Access. 2020;8:54074–54084. doi:10.1109/ACCESS.2020.2981434.
  28. Zhao ZQ, Zheng P, Xu ST, Wu X. Object detection with deep learning: A review. IEEE Trans Neural Netw Learn Syst. 2019;30:3212–3232. doi:10.1109/TNNLS.2018.2876865.
  29. Li L. Comprehensive survey on adversarial examples in cybersecurity: Impacts, challenges, and mitigation strategies. [Preprint]. 2024. arXiv:2412.12217. doi:10.48550/arXiv.2412.12217.
  30. Papernot N, McDaniel P, Jha S, Fredrikson M, Celik ZB, Swami A. The limitations of deep learning in adversarial settings. 2016 IEEE European Symposium on Security and Privacy (EuroS&P), Saarbruecken, Germany. 2016. p. 372–387. doi:10.1109/EuroSP.2016.36.
  31. Jia R, Liang P. Adversarial examples for evaluating reading comprehension systems. In: Palmer M, Hwa R, Riedel S, editors. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017); 2017 Sep; Copenhagen, Denmark. Stroudsburg (PA): Association for Computational Linguistics; 2017. p. 2021–2031. doi:10.18653/v1/D17-1215.
  32.  Goodfellow IJ, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. [Preprint]. 2014. arXiv:1412.6572. doi:10.48550/arXiv.1412.6572.
  33. Finlayson SG, Bowers JD, Ito J, Zittrain JL, Beam AL, Kohane IS. Adversarial attacks on medical machine learning. Science. 2019;363(6433):1287–1289. doi:10.1126/science.aaw4399.
  34. Pei K, Cao Y, Yang J, Jana S. DeepXplore: automated whitebox testing of deep learning systems. n: Proceedin s of the th ymposium on Operatin ystems Principles O P ‘ 7); 7 Oct; Shanghai, China. New York (NY): Association for Computing Machinery; 2017. p. 1–18. doi:10.1145/3132747.3132785.
  35. Cortés-Pérez N, Torres-Méndez LA. A low-cost mirror-based active perception system for effective collision-free underwater robotic navigation. 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Las Vegas, NV, USA. 2016. p. 61–68. doi:10.1109/CVPRW.2016.15.
  36.  Xie S, Girshick R, Dollár P, Tu Z, He K. Aggregated residual transformations for deep neural networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA. 2017. p. 5987–5995. doi:10.1109/CVPR.2017.634.
  37. Nelson B, Barreno M, Chi FJ, Joseph AD, Rubinstein BIP, Saini U, et al. Exploiting machine learning to subvert your spam filter. In: Monrose F, editor. Proceedings of the First USENIX Workshop on Large-Scale Exploits and Emergent Threats (LEET ‘08); 2008 Apr 15; San Francisco, CA, USA. Berkeley (CA): USENIX Association; 2008.
  38. . Papernot N, McDaniel P, Sinha A, Wellman MP. SoK: Security and privacy in machine learning. 2018 IEEE European Symposium on Security and Privacy (EuroS&P), London, UK. 2018. p. 399–414. doi:10.1109/EuroSP.2018.00035.
  39. Sharif M, Bhagavatula S, Bauer L, Reiter MK. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security; 2016; Vienna, Austria. New York (NY): ACM; 2016. p. 1528–1540. doi:10.1145/2976749.2978392.
  40. Shokri R, Shmatikov V. Privacy-preserving deep learning. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security; 2015; Denver, CO, USA. New York (NY): ACM; 2015. p. 1310–1321. doi:10.1145/2810103.2813687.
  41. Song C, Ristenpart T, Shmatikov V. Machine learning models that remember too much. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security; 2017; Dallas, TX, USA. New York (NY): ACM; 2017. p. 587–601. doi:10.1145/3133956.3134077.
  42. . Gan S, Zhang C, Qin X, Tu X, Li K, Pei Z, et al. CollaFL: Path sensitive fuzzing. 2018 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA. 2018. p. 679–696. doi:10.1109/SP.2018.00040.
  43. Tramèr F, Carlini N, Brendel W, Madry A. On adaptive attacks to adversarial example defenses. Adv Neural Inf Process Syst. 2020;33:1633–1645.
  44. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, et al. Intriguing properties of neural networks. [Preprint]. 2013. arXiv:1312.6199. doi:10.48550/arXiv.1312.6199.
  45. Recht B, Roelofs R, Schmidt L, Shankar V. Do ImageNet classifiers generalize to ImageNet? In: Chaudhuri K, Salakhutdinov R, editors. Proceedings of the 36th International Conference on Machine Learning (ICML); 2019 Jun 9-15; Long Beach, CA, USA. Proc Mach Learn Res. 2019;97:5389-5400.
  46. Zhang C, Bengio S, Hardt M, Recht B, Vinyals O. Understanding deep learning requires rethinking generalization. [Preprint]. 2016. arXiv:1611.03530. doi:10.48550/arXiv.1611.03530.
  47. Zhang H, Chen H, Song Z, Boning D, Dhillon IS, Hsieh CJ. The limitations of adversarial training and the blind-spot attack. [Preprint]. 2019. arXiv:1901.04684. doi:10.48550/arXiv.1901.04684.
  48. Xu H, Caramanis C, Mannor S. Robustness and regularization of support vector machines. J Mach Learn Res. 2009;10:1485-1510.
  49. Ren K, Zheng T, Qin Z, Liu X. Adversarial attacks and defenses in deep learning. Engineering. 2020;6(3):346–360. doi:10.1016/j.eng.2019.12.012.
  50. Ruiz N, Bargal SA, Sclaroff S. Disrupting deepfakes: adversarial attacks against conditional image translation networks and facial manipulation systems. [Preprint]. 2020 Mar 3. arXiv:2003.01279. doi:10.48550/arXiv.2003.01279.
  51. Aria M, Cuccurullo C. Bibliometrix: An R-tool for comprehensive science mapping analysis. J Informetr. 2017;11:959–975. doi:10.1016/j.joi.2017.08.007. 52. Li N, Zhai L, Ma Z, Zhu X, Li Y. Lyapunov-guided deep reinforcement learning for service caching and task offloading in mobile edge computing. Comput Netw. 2024;250:110593. doi:10.1016/j.comnet.2024.110593.
Regular Issue Subscription Review Article
Volume 13
Issue 01
Received 03/11/2025
Accepted 12/11/2025
Published 11/12/2025
Publication Time 38 Days
288

Login


My IP

PlumX Metrics