Loading...

A new framework to train autoencoders through non-smooth regularization

Amini, S ; Sharif University of Technology | 2019

253 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/TSP.2019.2899294
  3. Publisher: Institute of Electrical and Electronics Engineers Inc , 2019
  4. Abstract:
  5. Deep structures consisting of many layers of nonlinearities have a high potential of expressing complex relations if properly initialized. Autoencoders play a complementary role in training a deep structure by initializing each layer in a greedy unsupervised manner. Due to the high capacity presented by autoencoders, these structures need to be regularized. While mathematical regularizers (based on weight decay, sparsity, etc.) and structural ones (by way of, e.g., denoising and dropout) have been well studied in the literature, quite a few papers have addressed the problem of training autoencoder with non-smooth regularization. In this paper, we address the problem of training autoencoder with non-smooth regularization. We propose an efficient algorithm and mathematically prove that it is convergent, where the regularizer needs to be proximable. As one of major applications of the proposed method, we get focused on the problem of sparse autoencoders and show that the new training method leads to better disentangling of factors of variation
  6. Keywords:
  7. Autoencoder ; Regularizer ; Electronics engineering ; Signal processing ; Auto encoders ; Deep structure ; Gradient descent ; High capacity ; High potential ; Proximal operator ; Training methods ; Learning systems
  8. Source: IEEE Transactions on Signal Processing ; Volume 67, Issue 7 , 2019 , Pages 1860-1874 ; 1053587X (ISSN)
  9. URL: https://ieeexplore.ieee.org/document/8642441