Adversarial attacks on machine learning models in cybersecurity: a systematic literature review

Notice

This is an unedited manuscript accepted for publication and provided as an Article in Press for early access at the author’s request. The article will undergo copyediting, typesetting, and galley proof review before final publication. Please be aware that errors may be identified during production that could affect the content. All legal disclaimers of the journal apply.

Year : 2025 | Volume : 13 | 01 | Page :
    By

    Sanjay Singh,

  • Anamika Gupta,

  1. Pg Cybersecurity Student, Department of Computer Science, Shaheed Sukhdev College of Business Studies, University of Delhi, Delhi, India
  2. Professor, Department of Computer Science, Shaheed Sukhdev College of Business Studies, University of Delhi, Delhi, India

Abstract

Adversarial Machine Learning (AML) is a field that is growing swiftly, especially as machine learning models are employed more and more in places where security is critical. This review goes into great depth over 746 publications from the Scopus database, with an emphasis on the connection between AML and network security. Using the Biblioshiny, and Scopus tools, we looked at trends in publications, study fields, productive authors, collaboration networks, and theme concentrations. Thirteen charts show how AML research has developed over time by highlighting notable contributions, main journals, popular keywords, and citation trends. Our research demonstrates that the topic is multidisciplinary and has research centers all around the world. It also shows that scholars are not working together as much anymore and that the focus is moving from pure attack modeling to robustness and interpretability. At the end of the study, some major gaps are pointed out, such as the clichés about real correspondent implementation and ethical governance. The report also offers approaches for future research to make the AML research ecosystem stronger and more accessible to everyone.

Keywords: Adversarial Machine Learning, Machine Learning Security, Adversarial Attack, Network Security, Machine Learning Models

How to cite this article:
Sanjay Singh, Anamika Gupta. Adversarial attacks on machine learning models in cybersecurity: a systematic literature review. Journal of Artificial Intelligence Research & Advances. 2025; 13(01):-.
How to cite this URL:
Sanjay Singh, Anamika Gupta. Adversarial attacks on machine learning models in cybersecurity: a systematic literature review. Journal of Artificial Intelligence Research & Advances. 2025; 13(01):-. Available from: https://journals.stmjournals.com/joaira/article=2025/view=233970


References

  1. Akhtar N, Mian A. Threat of adversarial attacks on deep learning in computer vision: A survey. Ieee Access. 2018 Feb 19;6:14410-30.
  2. He J, Li X, Liao L, Song D, Cheung W. Inferring a personalized next point-of-interest recommendation model with latent behavior patterns. InProceedings of the AAAI conference on artificial intelligence 2016 Feb 21 (Vol. 30, No. 1).
  3. Biggio B, Roli F. Wild patterns: Ten years after the rise of adversarial machine learning. InProceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security 2018 Oct 15 (pp. 2154-2156).
  4. Carlini N, Wagner D. Towards evaluating the robustness of neural networks. In2017 ieee symposium on security and privacy (sp) 2017 May 22 (pp. 39-57). Ieee.
  5. Zhang J, Hu J, Liu J. Neural network with multiple connection weights. Pattern Recognition. 2020 Nov 1;107:107481.
  6. Chen X, Liu C, Li B, Lu K, Song D. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526. 2017 Dec 15.
  7. Babatunde LA, Etim ED, Essien IA, Cadet E, Ajayi JO, Erigha ED, Obuse E. Adversarial machine learning in cybersecurity: Vulnerabilities and defense strategies. Journal of Frontiers in Multidisciplinary Research. 2020 Jul;1(2):31-45.
  8. Demontis A, Melis M, Pintor M, Jagielski M, Biggio B, Oprea A, Nita-Rotaru C, Roli F. Why do adversarial attacks transfer? explaining transferability of evasion and poisoning attacks. In28th USENIX security symposium (USENIX security 19) 2019 (pp. 321-338).
  9. Dong Y, Liao F, Pang T, Su H, Zhu J, Hu X, Li J. Boosting adversarial attacks with momentum. InProceedings of the IEEE conference on computer vision and pattern recognition 2018 (pp. 9185-9193).
  10. Eykholt K, Evtimov I, Fernandes E, Li B, Rahmati A, Xiao C, Prakash A, Kohno T, Song D. Robust physical-world attacks on deep learning visual classification. InProceedings of the IEEE conference on computer vision and pattern recognition 2018 (pp. 1625-1634).
  11. Finlayson SG, Bowers JD, Ito J, Zittrain JL, Beam AL, Kohane IS. Adversarial attacks on medical machine learning. Science. 2019 Mar 22;363(6433):1287-9.
  12. Dilhara M, Ketkar A, Dig D. Understanding software-2.0: A study of machine learning library usage and evolution. ACM Transactions on Software Engineering and Methodology (TOSEM). 2021 Jul 23;30(4):1-42.
  13. Goodfellow IJ, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. 2014 Dec 20.
  14. Gu T, Dolan-Gavitt B, Garg S. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733. 2017 Aug 22.
  15. Hendrycks D, Gimpel K. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136. 2016 Oct 7.
  16. Ilyas A, Santurkar S, Tsipras D, Engstrom L, Tran B, Madry A. Adversarial examples are not bugs, they are features. Advances in neural information processing systems. 2019;32.
  17. Jagielski M, Oprea A, Biggio B, Liu C, Nita-Rotaru C, Li B. Manipulating machine learning: Poisoning attacks and countermeasures for regression learning. In2018 IEEE symposium on security and privacy (SP) 2018 May 20 (pp. 19-35). IEEE.
  18. Jia R, Liang P. Adversarial examples for evaluating reading comprehension systems. arXiv preprint arXiv:1707.07328. 2017 Jul 23.
  19. Wang X, Li J, Kuang X, Tan YA, Li J. The security of machine learning in an adversarial setting: A survey. Journal of Parallel and Distributed Computing. 2019 Aug 1;130:12-23.
  20. Li Y, Huang H, Guo X, Yuan Y. An Empirical Study on Group Fairness Metrics of Judicial Data. IEEE Access. 2021 Oct 25;9:149043-9.
  21. Li L. Comprehensive survey on adversarial examples in cybersecurity: Impacts, challenges, and mitigation strategies. arXiv preprint arXiv:2412.12217. 2024 Dec 16.
  22. Liu Y, Chen X, Liu C, Song D. Delving into transferable adversarial examples and black-box attacks. arXiv preprint arXiv:1611.02770. 2016 Nov 8.
  23. Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083. 2017 Jun 19.
  24. Xie S, Girshick R, Dollár P, Tu Z, He K. Aggregated residual transformations for deep neural networks. InProceedings of the IEEE conference on computer vision and pattern recognition 2017 (pp. 1492-1500).
  25. Nasr M, Shokri R, Houmansadr A. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In2019 IEEE symposium on security and privacy (SP) 2019 May 19 (pp. 739-753). IEEE.
  26. Nelson B, Barreno M, Chi FJ, Joseph AD, Rubinstein BI, Saini U, Sutton C, Tygar JD, Xia K. Exploiting machine learning to subvert your spam filter. Leet. 2008 Apr 15;8(1):9.
  27. Papernot N, McDaniel P, Jha S, Fredrikson M, Celik ZB, Swami A. The limitations of deep learning in adversarial settings. In2016 IEEE European symposium on security and privacy (EuroS&P) 2016 Mar 21 (pp. 372-387). IEEE.
  28. Papernot N, McDaniel P, Sinha A, Wellman MP. Sok: Security and privacy in machine learning. In2018 IEEE European symposium on security and privacy (EuroS&P) 2018 Apr 24 (pp. 399-414). IEEE.
  29. Kurita K, Vyas N, Pareek A, Black AW, Tsvetkov Y. Measuring bias in contextualized word representations. arXiv preprint arXiv:1906.07337. 2019 Jun 18.
  30. Pei K, Cao Y, Yang J, Jana S. Deepxplore: Automated whitebox testing of deep learning systems. Inproceedings of the 26th Symposium on Operating Systems Principles 2017 Oct 14 (pp. 1-18).
  31. Cortes Perez N, Abril Torres Mendez L. A low-cost mirror-based active perception system for effective collision free underwater robotic navigation. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops 2016 (pp. 61-68).
  32. Sadeghi AR, Wachsmann C, Waidner M. Security and privacy challenges in industrial internet of things. InProceedings of the 52nd annual design automation conference 2015 Jun 7 (pp. 1-6).
  33. Sharif M, Bhagavatula S, Bauer L, Reiter MK. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. InProceedings of the 2016 acm sigsac conference on computer and communications security 2016 Oct 24 (pp. 1528-1540).
  34. Shokri R, Shmatikov V. Privacy-preserving deep learning. InProceedings of the 22nd ACM SIGSAC conference on computer and communications security 2015 Oct 12 (pp. 1310-1321).
  35. Song C, Ristenpart T, Shmatikov V. Machine learning models that remember too much. InProceedings of the 2017 ACM SIGSAC Conference on computer and communications security 2017 Oct 30 (pp. 587-601).
  36. Su J, Vargas DV, Sakurai K. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation. 2019 Jan 4;23(5):828-41.
  37. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199. 2013 Dec 21.
  38. Tramer F, Carlini N, Brendel W, Madry A. On adaptive attacks to adversarial example defenses. Advances in neural information processing systems. 2020;33:1633-45.
  39. Gan S, Zhang C, Qin X, Tu X, Li K, Pei Z, Chen Z. Collafl: Path sensitive fuzzing. In2018 IEEE Symposium on Security and Privacy (SP) 2018 May 20 (pp. 679-696). IEEE.
  40. Cheng G, Sun X, Li K, Guo L, Han J. Perturbation-seeking generative adversarial networks: A defense framework for remote sensing image scene classification. IEEE Transactions on Geoscience and Remote Sensing. 2021 May 27;60:1-1.
  41. Li J, Gao J, Jiang Q, He G. Adversarial Defense Networks via Gaussian Noise and RBF. InInternational Conference on Artificial Intelligence and Security 2021 Jul 9 (pp. 494-504). Cham: Springer International Publishing.
  42. Xiao C, Li B, Zhu JY, He W, Liu M, Song D. Generating adversarial examples with adversarial networks. arXiv preprint arXiv:1801.02610. 2018 Jan 8.
  43. Xu H, Caramanis C, Mannor S. Robustness and regularization of support vector machines. Journal of machine learning research. 2009 Jul 1;10(7).
  44. Alfakih T, Hassan MM, Gumaei A, Savaglio C, Fortino G. Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA. Ieee Access. 2020 Mar 17;8:54074-84.
  45. Zhao ZQ, Zheng P, Xu ST, Wu X. Object detection with deep learning: A review. IEEE transactions on neural networks and learning systems. 2019 Jan 28;30(11):3212-32.
  46. Zhang C, Bengio S, Hardt M, Recht B, Vinyals O. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530. 2016 Nov 10.
  47. Zhang H, Chen H, Song Z, Boning D, Dhillon IS, Hsieh CJ. The limitations of adversarial training and the blind-spot attack. arXiv preprint arXiv:1901.04684. 2019 Jan 15.
  48. Recht B, Roelofs R, Schmidt L, Shankar V. Do imagenet classifiers generalize to imagenet?. InInternational conference on machine learning 2019 May 24 (pp. 5389-5400). PMLR.
  49. Ren K, Zheng T, Qin Z, Liu X. Adversarial attacks and defenses in deep learning. Engineering. 2020 Mar 1;6(3):346-60.
  50. Ruiz N, Bargal SA, Sclaroff S. Disrupting deepfakes: Adversarial attacks against conditional image translation networks and facial manipulation systems. InEuropean conference on computer vision 2020 Aug 23 (pp. 236-251). Cham: Springer International Publishing.
  51. Aria M, Cuccurullo C. bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of informetrics. 2017 Nov 1;11(4):959-75.
  52. ACAD_SC_VID_CSAB_purpose_expertcuration (1).mp4. www.elsevier.com. Elsevier; 2025. Available from: https://www.elsevier.com/en-in/products/scopus/content
Ahead of Print Subscription Review Article
Volume 13
01
Received 03/11/2025
Accepted 12/11/2025
Published 11/12/2025
Publication Time 38 Days
146

Login


My IP

PlumX Metrics