Loading...

Partition pruning: Parallelization-aware pruning for dense neural networks

Shahhosseini, S ; Sharif University of Technology | 2020

347 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/PDP50117.2020.00053
  3. Publisher: Institute of Electrical and Electronics Engineers Inc , 2020
  4. Abstract:
  5. As recent neural networks are being improved to be more accurate, their model's size is exponentially growing. Thus, a huge number of parameters requires to be loaded and stored from/in memory hierarchy and computed in processors to perform training or inference phase of neural network processing. Increasing the number of parameters causes a big challenge for real-time deployment since the memory bandwidth improvement's trend cannot keep up with models' complexity growing trend. Although some operations in neural networks processing are computational intensive such as convolutional layer computing, computing dense layers face with memory bandwidth bottleneck. To address the issue, the paper has proposed Partition Pruning for dense layers to reduce the required parameters while taking into consideration parallelization. We evaluated the performance and energy consumption of parallel inference of partitioned models, which showed a 7.72x speedup of performance and a 2.73x reduction in the energy used for computing pruned fully connected layers in TinyVGG16 model in comparison to running the unpruned model on a single accelerator. Besides, our method showed a limited reduction in accuracy while partitioning fully connected layers. © 2020 IEEE
  6. Keywords:
  7. Deep Neural Network ; Hardware Accelerator ; Parallelization ; Partitioning ; Pruning ; Bandwidth ; Convolutional neural networks ; Energy utilization ; Green computing ; Dense layer ; Memory bandwidths ; Memory hierarchy ; Neural-network processing ; Parallelizations ; Partition pruning ; Real time ; Multilayer neural networks
  8. Source: 28th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2020, 11 March 2020 through 13 March 2020 ; 2020 , Pages 307-311
  9. URL: https://ieeexplore.ieee.org/document/9092354