Loading...
| Friend's email | |
| Your name | |
| Your email | |
| enter code | |
This page was sent successfuly
- Type of Document: Ph.D. Dissertation
- Language: Farsi
- Document No: 52939 (05)
- University: Sharif University of Technology
- Department: Electrical Engineering
- Advisor(s): Ghaemmaghami, Shahrokh
- Abstract:
- Deep neural networks have not progresses comparative until last decade due to computational complexity and principal challenges as gradient vanishing. Thanks to newly designed hardware architecture and great breakthroughs in 2000s leading to the solution of principal challenges, we currently face a tsunami of deep architecture utilization in various machine learning applications. Sparsity of a representation as a feature to make it more descriptive has been considered in different deep learning architectures leading to different formulations where sparsity is impose on specific representations. Due to the gradient based optimization methods for training deep architecture, smooth regularizers have been utilized to impose sparsity while non-smooth ones have shown their efficiency in different applications including sparse coding. This shows one current challenge of deep learning architectures with sparsity regularizer. Although deep architectures accuracy is now highly progressed on clean input patterns, various papers have shown their weak robustness against adversarial perturbations in the input pattern. Different solutions have been proposed to improve the robustness on deep architectures among them Parseval networks shown impressive results. These networks impose Parseval tightness on the weight matrices of convolutional and fully-connected layers using a smooth regularizer. For the purpose of imposing Parseval tightness on the weight matrices, non-smooth regularizer would be better options. Because of gradient based optimization methods for the training of deep architectures, these non-smooth regularizers cannot be added to training formulations which shows the other current challenge of deep learning architectures. The above mentioned challenges have a common aspect. In both of them a training problem with a non-smooth regularizer term must be solved. In the first step of this thesis, we propose a new learning framework to train a deep architecture with non-smooth regularizer. In a specific set of constraints, the proposed framework is guaranteed to converge in a limited number of iterations to a critical point of learning cost function. In the second step we customize this framework to solve the problem of non-smooth regularized sparse autoencoder. Comparing to the previous formulations, the proposed one present more sparse code vectors in an equal reconstruction error and the deep network initializes using proposed framework lead to higher classification accuracy. In the sequel, we customized the framework to train a robust convolutional neural network. Comparing to previous learning formulation, the proposed framework shows boosted results for untargeted adversarial samples and comparative results for targeted ones
- Keywords:
- Deep Learning ; Sparse Representation ; Perturbed Pattern ; Proximal Mapping ; Deep Neural Networks
Digital Object List
-
محتواي کتاب
- view
Bookmark