Loading...

Reduction of Communication Cost and Effect of Heterogeneity in Federated Learning Via Efficient Clustering of Users

Babaei Vavdareh, Mehdi | 2022

100 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 55501 (05)
  4. University: Sharif University of Technology
  5. Department: Electrical Engineering
  6. Advisor(s): Behroozi, Hamid; Hossein Khalaj, Babak
  7. Abstract:
  8. Nowadays sharing data between users and organizations can be difficult due to privacy and legal reasons. Therefore, real-world data is not fully exploited by machine learning methods. A novel model training method called Federated Learning was proposed to alleviate the aforementioned problem by enabling users to jointly train learning models without sharing data. Federated learning utilization faces many challenges among which high communication cost, statistical heterogeneity and system heterogeneity are most important. This research deals with these challenges by proposing two methods of Reduced Clustering Federated Learning (RCFL) and Weighted Federated Distillation (WFD). Moreover, we use cluster-based aggregation and compression method to reduce communication cost further. Examining RCFL in different scenarios indicates that the accuracy of the model improves by about 10 to 15% compared to regular federated learning. Moreover, RCFL can achieve the target accuracy in about one-third the number of rounds required by state-of-the-art clustering methods to achieve that accuracy. The weights proposed in WFD improve the accuracy of the model by about 5% compared to federated distillation. Furthermore, communication cost can be reduced up to three order of magnitude in WFD compared to regular federated learning. Therefore, the results demonstrate the efficacy of our proposed approaches.
  9. Keywords:
  10. Communication Cost ; Compression ; Knowledge Distillation ; Clustering ; Federated Learning ; Statistical Heterogeneity ; System Heterogeneity

 Digital Object List

 Bookmark

No TOC