Binit Behl,
- Student, Department of Computer Science & Engineering, Greater Noida Institute of Technology, Greater Noida, Uttar Pradesh, India, Uttar Pradesh, India
Abstract
Deep learning systems achieve strong predictive performance under independently and identically distributed conditions; however, their reliability can decline significantly when exposed to distribution shifts during deployment. This decline is often accompanied by overconfident predictions, which can introduce serious risks in safety-critical applications. In this work, we explore whether explicitly modeling the structure of the latent feature space can improve reliability through more effective out-ofdistribution detection. A statistically grounded framework based on class-conditional density estimation in feature space is proposed and evaluated across a range of semantic and covariate shift scenarios. Comparative results against established softmax-based and energy-based baselines show consistent improvements in detection performance, calibration stability, and control of false positives. In addition, supervised contrastive regularization is incorporated to produce more compact embeddings and improve the separation between in-distribution and out-of-distribution samples. The findings indicate that robustness under distribution shift is closely related to the quality of learned representations rather than relying solely on post hoc confidence measures. Overall, this study provides both theoretical and empirical insights into feature-space modeling as a foundation for building more reliable deep learning systems.
Keywords: Distribution shift, out-of-distribution detection, feature representation, reliability, representation learning, robustness
[This article belongs to International Journal of Bioinformatics and Computational Biology ]
References
1. Hendrycks D, Gimpel K. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136. 2016 Oct 7.
2. Liang S, Li Y, Srikant R. Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv:1706.02690. 2017 Jun 8.
3. Lee K, Lee K, Lee H, Shin J. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Adv Neural Inf Process Syst. 2018;31.
4. Liu W, Wang X, Owens J, Li Y. Energy-based out-of-distribution detection. Adv Neural Inf Process Syst. 2020;33:21464–21475.
5. Ovadia Y, Fertig E, Ren J, Nado Z, Sculley D, Nowozin S, et al. Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift. Adv Neural Inf Process Syst. 2019;32.
6. Lakshminarayanan B, Pritzel A, Blundell C. Simple and scalable predictive uncertainty estimation using deep ensembles. Adv Neural Inf Process Syst. 2017;30.
7. Quiñonero-Candela J, Sugiyama M, Schwaighofer A, Lawrence ND, editors. Dataset shift in machine learning. MIT Press; 2008 Dec 12.
8. Khosla P, Teterwak P, Chen W, Sarna A, Tian Y, Isola P, et al. Supervised contrastive learning. Adv Neural Inf Process Syst. 2020;33:18661–73. Available from: proceedings.neurips.cc
9. Papernot N, McDaniel P. Deep k-nearest neighbors: Towards confident, interpretable and robust deep learning. arXiv preprint arXiv:1803.04765. 2018 Mar 13.
10. Amodei D, Olah C, Steinhardt J, Christiano P, Schulman J, Mané D. Concrete problems in AI safety. arXiv preprint arXiv:1606.06565. 2016 Jun 21.
11. Nam Y, Mingard C, Seok Hyeong L, Hayou S, Ard L. Visualising feature learning in deep neural networks by diagonalizing the forward feature map. 2024. Available from: arxiv.org.
12. Hein M, Andriushchenko M, Bitterwolf J. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2019 Jun 15. p. 41–50. IEEE.
13. Lee K, Lee H, Lee K, Shin J. Training confidence-calibrated classifiers for detecting out-of-distribution samples. arXiv preprint arXiv:1711.09325. 2017 Nov 26.
14. DeVries T, Taylor GW. Learning confidence for out-of-distribution detection in neural networks. arXiv preprint arXiv:1802.04865. 2018 Feb 13. Gal Y, Ghahramani Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In: International Conference on Machine Learning; 2016 Jun 11. p. 1050–1059. PMLR.
| Volume | 04 | |
| Issue | 02 | |
| Received | 30/06/2026 | |
| Accepted | 08/09/2026 | |
| Published | 18/09/2026 | |
| Publication Time | 80 Days |