Loading...

Learning of gaussian processes in distributed and communication limited systems

Tavassolipour, M ; Sharif University of Technology | 2020

619 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/TPAMI.2019.2906207
  3. Publisher: IEEE Computer Society , 2020
  4. Abstract:
  5. It is of fundamental importance to find algorithms obtaining optimal performance for learning of statistical models in distributed and communication limited systems. Aiming at characterizing the optimal strategies, we consider learning of Gaussian Processes (GP) in distributed systems as a pivotal example. We first address a very basic problem: how many bits are required to estimate the inner-products of some Gaussian vectors across distributed machines? Using information theoretic bounds, we obtain an optimal solution for the problem which is based on vector quantization. Two suboptimal and more practical schemes are also presented as substitutes for the vector quantization scheme. In particular, it is shown that the performance of one of the practical schemes which is called per-symbol quantization is very close to the optimal one. Schemes provided for the inner-product calculations are incorporated into our proposed distributed learning methods for GPs. Experimental results show that with spending few bits per symbol in our communication scheme, our proposed methods outperform previous zero rate distributed GP learning schemes such as Bayesian Committee Model (BCM) and Product of experts (PoE). © 1979-2012 IEEE
  6. Keywords:
  7. Communication constraints ; Distributed learning ; Gaussian processes ; Vector quantization ; Distributed database systems ; Gaussian distribution ; Gaussian noise (electronic) ; Optimal systems ; Communication schemes ; Distributed systems ; Gaussian Processes ; Information theoretic bounds ; Optimal performance ; Optimal strategies ; Product of experts ; Learning systems
  8. Source: IEEE Transactions on Pattern Analysis and Machine Intelligence ; Volume 42, Issue 8 , 2020 , Pages 1928-1941
  9. URL: https://ieeexplore.ieee.org/document/8669734