Bias Detection and Accuracy Enhancement in Voice-based Banking Authentication Using Deep Learning

Notice

This is an unedited manuscript accepted for publication and provided as an Article in Press for early access at the author’s request. The article will undergo copyediting, typesetting, and galley proof review before final publication. Please be aware that errors may be identified during production that could affect the content. All legal disclaimers of the journal apply.

Year : 2026 | Volume : 4 | 02 | Page :
    By

    Vishal Sanjay Yadav,

  • Gaurang Gopal Vasoya,

  • Anusri Mukhopadhyay,

  1. Research Scholar, Department of MCA Thakur Institute of Management Studies, Career Development & Research (TIMSCDR) Mumbai, Maharashtra, India
  2. Professor, Department of MCA Thakur Institute of Management Studies, Career Development & Research (TIMSCDR) Mumbai, Maharashtra, India
  3. Assistant Professor, Department of MCA Thakur Institute of Management Studies, Career Development & Research (TIMSCDR) Mumbai, Maharashtra, India

Abstract

Biometric systems have become an integral part of how many people access banking services today, and voice verification systems can be a secure and easy-to-use source of banking authentication that does not require any physical contact with the bank or any other person. From the security perspective, these systems would normally provide an effective means of identifying an individual but frequently exhibit bias with respect to demographics such as the individual’s gender, age, accent, or speech patterns. Consequently, this bias can lead to unequal performance among the different user groups and create a level of distrust toward the system as well as reduce the perceived reliability and inclusiveness of banking systems. This paper examines the problem of demographic bias in voice-based banking authentication and describes an approach to increase both fairness and recognition accuracy in voice verification systems. Specifically, we developed a deep learning framework based on a combination of Convolutional Neural Networks (CNNs) and Bidirectional Long Short Term Memory (BiLSTM) networks trained using a voice dataset that is balanced with respect to gender, age, and other demographic factors. Furthermore, we incorporated fair machine learning techniques such as adversarial debiasing and data reweighting directly into the model development process with an emphasis on minimizing bias. The features from the users’ voices were extracted using Mel Frequency Cepstral Coefficients (MFCCs) to obtain a representation of the speech signals while increasing the robustness of the developed models. The performance of the system is analyzed based on several performance indicators including equal error rate (EER), F1 score, accuracy, and demographic parity difference among various demographics. The experimental results show that the suggested technique is capable of decreasing bias-based error rates by about 20 percent and increasing overall accuracy by approximately 12 percent in comparison with existing voice authentication schemes. In addition, the proposed method shows better consistency among different accent and gender users.

Keywords: Voice Authentication, Deep Learning, Bias Detection, Fairness-aware Models, Banking Security, CNN

How to cite this article:
Vishal Sanjay Yadav, Gaurang Gopal Vasoya, Anusri Mukhopadhyay. Bias Detection and Accuracy Enhancement in Voice-based Banking Authentication Using Deep Learning. International Journal of Information Security Engineering. 2026; 04(02):-.
How to cite this URL:
Vishal Sanjay Yadav, Gaurang Gopal Vasoya, Anusri Mukhopadhyay. Bias Detection and Accuracy Enhancement in Voice-based Banking Authentication Using Deep Learning. International Journal of Information Security Engineering. 2026; 04(02):-. Available from: https://journals.stmjournals.com/ijise/article=2026/view=247673


References

  1. Nagrani, J. S. Chung, and A. Zisserman, “VoxCeleb: A large-scale speaker identification dataset,” in Proc. Interspeech, 2017, pp. 2616–2620.
  2. S. Chung, A. Nagrani, and A. Zisserman, “VoxCeleb2: Deep speaker recognition,” in Proc. Interspeech, 2018, pp. 1086–1090.
  3. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, “X-vectors: Robust DNN embeddings for speaker recognition,” in Proc. IEEE ICASSP, 2018, pp. 5329–5333.
  4. Heigold, I. L. Moreno, S. Bengio, and N. Shazeer, “End-to-end text-dependent speaker verification,” in Proc. IEEE ICASSP, 2016, pp. 5115–5119.
  5. Pascual, M. Ravanelli, J. Serrà, A. Bonafonte, and Y. Bengio, “Learning problem-agnostic speech representations from multiple self-supervised tasks,” in Proc. Interspeech, 2019, pp. 161–165.
  6. Kinnunen and H. Li, “An overview of text-independent speaker recognition: From features to supervectors,” Speech Commun., vol. 52, no. 1, pp. 12–40, Jan. 2010.
  7. Zhang, R. K. Bellamy, and C. Kulkarni, “Ethical and societal implications of voice biometrics,” J. Inf., Commun. Ethics Soc., vol. 18, no. 4, pp. 589–606, 2020.
  8. Dehak, P. J. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, “Front-end factor analysis for speaker verification,” IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 4, pp. 788–798, May 2011
  9. Deng, Z. Chen, and H. Xu, “A hybrid CNN-BiLSTM model for speaker verification,” Appl. Sci., vol. 12, no. 3, p. 1457, 2022.
  10. X. Li, J. Ma, and C.-H. Lee, “Deep learning in speaker recognition: Advances and emerging challenges,” IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 29, pp. 1701–1715, 2021.
Ahead of Print Subscription Original Research
Volume 04
02
Received 10/03/2026
Accepted 19/06/2026
Published 26/06/2026
Publication Time 108 Days
43

Login


My IP

PlumX Metrics