Loading...

Multi-head relu implicit neural representation networks

Aftab, A ; Sharif University of Technology | 2022

19 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/ICASSP43922.2022.9747352
  3. Publisher: Institute of Electrical and Electronics Engineers Inc , 2022
  4. Abstract:
  5. In this paper, a novel multi-head multi-layer perceptron (MLP) structure is presented for implicit neural representation (INR). Since conventional rectified linear unit (ReLU) networks are shown to exhibit spectral bias towards learning low-frequency features of the signal, we aim at mitigating this defect by taking advantage of local structure of the signals. To be more specific, an MLP is used to capture the global features of the underlying generator function of the desired signal. Then, several heads are utilized to reconstruct disjoint local features of the signal, and to reduce the computational complexity, sparse layers are deployed for attaching heads to the body. Through various experiments, we show that the proposed model does not suffer from the special bias of conventional ReLU networks and has superior generalization capabilities. Finally, simulation results confirm that the proposed multi-head structure outperforms existing INR methods with considerably less computational cost. The source code is available at https://github.com/AlirezaMorsali/MH-RELU-INR. © 2022 IEEE
  6. Keywords:
  7. Multi-head MLP ; Multi-layer perceptron ; ReLU network ; Implicit neural representation ; Linear units ; Local structure ; Low frequency features ; Multi-head multi-layer perceptron ; Multilayers perceptrons ; Neural representations ; Rectified linear unit network ; Spectral bias ; Multilayer neural networks
  8. Source: 47th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2022, 23 May 2022 through 27 May 2022 ; Volume 2022-May , 2022 , Pages 2510-2514 ; 15206149 (ISSN); 9781665405409 (ISBN)
  9. URL: https://ieeexplore.ieee.org/document/9747352