Loading...

Masked autoencoder for distribution estimation on small structured data sets

Khajenezhad, A ; Sharif University of Technology | 2020

477 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/TNNLS.2020.3026572
  3. Publisher: Institute of Electrical and Electronics Engineers Inc , 2020
  4. Abstract:
  5. Autoregressive models are among the most successful neural network methods for estimating a distribution from a set of samples. However, these models, such as other neural methods, need large data sets to provide good estimations. We believe that knowing structural information about the data can improve their performance on small data sets. Masked autoencoder for distribution estimation (MADE) is a well-structured density estimator, which alters a simple autoencoder by setting a set of masks on its connections to satisfy the autoregressive condition. Nevertheless, this model does not benefit from extra information that we might know about the structure of the data. This information can especially be advantageous in case of training on small data sets. In this article, we propose two autoencoders for estimating the density of a small set of observations, where the data have a known Markov random field (MRF) structure. These methods modify the masking process of MADE, according to conditional dependencies inferred from the MRF structure, to reduce either the model complexity or the problem complexity. We compare the proposed methods with some related binary, discrete, and continuous density estimators on MNIST, binarized MNIST, OCR-letters, and two synthetic data sets. IEEE
  6. Keywords:
  7. Deep learning ; Density estimation ; Markov random field (MRF) ; Masked autoencoder ; Complex networks ; Magnetorheological fluids ; Markov processes ; Auto regressive models ; Density estimator ; Distribution estimation ; Markov Random Fields ; Neural network method ; Problem complexity ; Structural information ; Synthetic datasets ; Learning systems
  8. Source: IEEE Transactions on Neural Networks and Learning Systems ; 2020
  9. URL: https://ieeexplore.ieee.org/document/9222344