Loading...

Evaluation and optimization of distributed machine learning techniques for internet of things

Gao, Y ; Sharif University of Technology | 2021

196 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/TC.2021.3135752
  3. Publisher: IEEE Computer Society , 2021
  4. Abstract:
  5. Federated learning (FL) and split learning (SL) are state-of-the-art distributed machine learning techniques to enable machine learning without accessing raw data on clients or end devices. However, their comparative training performance under real-world resource-restricted Internet of Things (IoT) device settings, e.g., Raspberry Pi, remains barely studied, which, to our knowledge, have not yet been evaluated and compared, rendering inconvenient reference for practitioner. This work firstly provides empirical comparisons of FL and SL in real-world IoT settings regarding learning performance and on-device execution overhead. Our analyses demonstrate that the learning performance of SL is better than FL under an imbalanced data distribution but worse than FL under an extreme non-IID data distribution. Recently, FL and SL are combined to form spltifed learning (SFL) to leverage each of their benefits (e.g., faster training time than SL). This work considers FL, SL, and SFL, and mount them on Raspberry Pi devices to evaluate their performance. Besides, we apply two optimizations. Firstly, we generalize SFL by carefully examining the possibility of hybrid type of model training at the server-side to better fit large-scaled devices. Secondly, we propose pragmatic techniques to substantially reduce the communication overhead (by 4 times) of the generalized SFL. IEEE
  6. Keywords:
  7. Artificial intelligence ; Distance education ; Learning algorithms ; Learning systems ; AND splits ; Distance-learning ; Distributed database ; Distributed machine learning ; Federated learning ; Internet of thing ; Optimisations ; Performances evaluation ; Split federated learning ; Split learning ; Internet of things
  8. Source: IEEE Transactions on Computers ; 2021 ; 00189340 (ISSN)
  9. URL: https://ieeexplore.ieee.org.com/document/9652119