Loading...

Low-complexity stochastic Generalized Belief Propagation

Haddadpour, F ; Sharif University of Technology

463 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/ISIT.2016.7541406
  3. Publisher: Institute of Electrical and Electronics Engineers Inc
  4. Abstract:
  5. The generalized belief propagation (GBP), introduced by Yedidia et al., is an extension of the belief propagation (BP) algorithm, which is widely used in different problems involved in calculating exact or approximate marginals of probability distributions. In many problems, it has been observed that the accuracy of GBP outperforms that of BP considerably. However, due to its generally higher complexity compared to BP, its application is limited in practice. In this paper, we introduce a stochastic version of GBP called stochastic generalized belief propagation (SGBP) that can be considered as an extension to the stochastic BP (SBP) algorithm introduced by Noorshams et al. They have shown that SBP reduces the complexity per iteration of BP by an order of magnitude in alphabet size. In contrast to SBP, SGBP can reduce the computation complexity if certain topological conditions are met by the region graph associated to a graphical model. However, this reduction can be larger than only one order of magnitude in alphabet size. In this paper, we characterize these conditions and the amount of complexity gain that one can obtain by using SGBP. Finally, using similar proof techniques employed by Noorshams et al., for general graphical models satisfy contraction conditions, we prove the asymptotic convergence of SGBP to the unique GBP fixed point, as well as providing non-asymptotic upper bounds on the mean square error and on the high probability error
  6. Keywords:
  7. Aluminum ; Graphic methods ; Information theory ; Iterative methods ; Mean square error ; Probability distributions ; Speech recognition ; Stochastic systems ; Topology ; Asymptotic convergence ; Belief propagation algorithm ; Computation complexity ; Contraction conditions ; Generalized belief propagation ; High probability ; ITS applications ; Topological conditions ; Backpropagation
  8. Source: 2016 IEEE International Symposium on Information Theory, ISIT 2016, 10 July 2016 through 15 July 2016 ; Volume 2016-August , 2016 , Pages 785-789 ; 21578095 (ISSN) ; 9781509018062 (ISBN)
  9. URL: http://ieeexplore.ieee.org/document/7541406/?reload=true