Loading...

CodedSketch: Coded distributed computation of approximated matrix multiplication

Jahani Nezhad, T ; Sharif University of Technology | 2019

305 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/ISIT.2019.8849370
  3. Publisher: Institute of Electrical and Electronics Engineers Inc , 2019
  4. Abstract:
  5. In this paper, we propose CodedSketch, as a distributed straggler-resistant scheme to compute an approximation of the multiplication of two massive matrices. The objective is to reduce the recovery threshold, defined as the total number of worker nodes that the master node needs to wait for to be able to recover the final result. To exploit the fact that only an approximated result is required, in reducing the recovery threshold, some sorts of pre-compression are required. However, compression inherently involves some randomness that would lose the structure of the matrices. On the other hand, considering the structure of the matrices is crucial to reduce the recovery threshold. In CodedSketch, we use count-sketch, as a hash-based compression scheme, on the rows of the first and columns of the second matrix, and a structured polynomial code on the columns of the first and rows of the second matrix. This arrangement allows us to exploit the gain of both in reducing the recovery threshold. To increase the accuracy of computation, multiple independent count-sketches are needed. This independency allows us to theoretically characterize the accuracy of the result and establish the recovery threshold achieved by the proposed scheme. To guarantee the independency of resulting count-sketches in the output, while keeping its cost on the recovery threshold minimum, we use another layer of structured codes. © 2019 IEEE
  6. Keywords:
  7. Computation theory ; Distributed computer systems ; Information theory ; Recovery ; Compression scheme ; Distributed computations ; MAtrix multiplication ; Polynomial codes ; Pre-compression ; Recovery thresholds ; Structured codes ; Worker nodes ; Matrix algebra
  8. Source: 2019 IEEE International Symposium on Information Theory, ISIT 2019, 7 July 2019 through 12 July 2019 ; Volume 2019-July , 2019 , Pages 2489-2493 ; 21578095 (ISSN); 9781538692912 (ISBN)
  9. URL: https://ieeexplore.ieee.org/document/8849370