Accelerating Unsupervised Feature Learning: Parallelized Training of Denoising Autoencoders

Year : 2025 | Volume : 12 | Issue : 03 | Page : 56 64
    By

    Nagajayant Nagamani,

  1. Software Engagement, Cognizant, Chennai, Tamil Nadu, India

Abstract

Unsupervised representation learning has become a cornerstone of contemporary machine learning, enabling algorithms to extract informative features from un-labelled, high-dimensional data. This work investigates the efficacy of stacked denoising autoencoders (SDAEs) trained via parallelized stochastic gradient descent (SGD) as a scalable approach to feature extraction. By strategically leveraging multi-threaded computation, our study systematically examines the trade-offs between increased parallelism, training efficiency, and the preservation of model accuracy. Experiments on the MNIST digit classification benchmark demonstrate that parallelization of the autoencoder training process significantly reduces convergence time, particularly as the dimensionality of the hidden layers grows. Speedup due to multi-threading, while substantial, was occasionally sublinear due to inherent sequential dependencies in the backpropagation algorithm and resource contention during memory access. Moreover, visualizations of learned filters illustrate the model’s ability to capture meaningful patterns in the input space, while reconstructions from noisy inputs underline the robustness of the denoising criterion. Classification using stacked representations achieves competitive accuracy compared to state-of-the-art supervised models such as SVMs, highlighting the practical utility of unsupervised pretraining. Beyond SGD, the study discusses the promise of evolutionary optimization algorithms, in particular, genetic algorithms, as highly parallelizable and potentially more robust alternatives for training deep architectures on challenging noisy datasets. Overall, the study emphasizes scalable training techniques for deep unsupervised models and outlines future directions for enhancing training speed, generalizability, and resilience to complex data variations. These findings contribute to the broader field of deep learning by showcasing pathways for improving both the computational efficiency and the qualitative performance of neural network-based feature learning systems.

Keywords: Autoencoders, machine learning, SVM, Supervised learning, algorithm

[This article belongs to Journal of Image Processing & Pattern Recognition Progress ]

How to cite this article:
Nagajayant Nagamani. Accelerating Unsupervised Feature Learning: Parallelized Training of Denoising Autoencoders. Journal of Image Processing & Pattern Recognition Progress. 2025; 12(03):56-64.
How to cite this URL:
Nagajayant Nagamani. Accelerating Unsupervised Feature Learning: Parallelized Training of Denoising Autoencoders. Journal of Image Processing & Pattern Recognition Progress. 2025; 12(03):56-64. Available from: https://journals.stmjournals.com/joipprp/article=2025/view=224982


References

  1. Bengio Y, Courville A, Vincent P. Representation learning: A review and new perspectives. IEEE Trans Pattern Anal Mach Intell. 2013 Mar 7; 35(8): 1798–828.
  2. Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA, Bottou L. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res. 2010 Dec 1; 11(12): 3371–3408.
  3. Hinton GE, Osindero S, Teh YW. A fast learning algorithm for deep belief nets. Neural Comput. 2006 Jul; 18(7): 1527–54.
  4. Hecht-Nielsen R. Theory of the backpropagation neural network. In: Neural networks for perception. Academic Press; Massachusetts. 1992 Jan 1; 65–93.
  5. Bottou L. Stochastic gradient learning in neural networks. Proceedings of Neuro-Nımes. 1991 Nov; 91(8): 12.
  6. Srinivas M, Patnaik LM. Genetic algorithms: A survey. Computer. 2002 Aug 6; 27(6): 17–26.
  7. Cantú-Paz E. A survey of parallel genetic algorithms. Calculateurs paralleles, reseaux et systems repartis. 1998 May; 10(2): 141–71.
  8. Cantu-Paz E. Efficient and accurate parallel genetic algorithms. New York, NY: Springer Science & Business Media; 2000 Nov 30.
  9. Amari SI. Backpropagation and stochastic gradient descent method. Neurocomputing. 1993 Jun 1; 5(4–5): 185–96.
  10. Ranzato MA, Krizhevsky A, Hinton G. Factored 3-way restricted Boltzmann machines for modeling natural images. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings. 2010 Mar 31; 621–628.
Regular Issue Subscription Review Article
Volume 12
Issue 03
Received 11/08/2025
Accepted 19/08/2025
Published 28/08/2025
Publication Time 17 Days
214

Login


My IP

PlumX Metrics