Loading...

A classification of hadoop job schedulers based on performance optimization approaches

Ghazali, R ; Sharif University of Technology | 2021

208 Viewed
  1. Type of Document: Article
  2. DOI: 10.1007/s10586-021-03339-8
  3. Publisher: Springer , 2021
  4. Abstract:
  5. Job scheduling in MapReduce plays a vital role in Hadoop performance. In recent years, many researchers have presented job scheduler algorithms to improve Hadoop performance. Designing a job scheduler that minimizes job execution time with maximum resource utilization is not a straightforward task. The primary purpose of this paper is to investigate agents affecting job scheduler efficiency and present a novel classification for job schedulers based on these factors. We provide a comprehensive overview of existing job schedulers in each group, evaluating their approaches, their effects on Hadoop performance, and comparing their advantages and disadvantages. Finally, we provide recommendations on choosing a preferred job scheduler in different environments for improving Hadoop performance. © 2021, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature
  6. Keywords:
  7. Computer networks ; Software engineering ; Job execution ; Job scheduler ; Job scheduling ; Map-reduce ; Performance optimizations ; Resource utilizations ; Scheduling
  8. Source: Cluster Computing ; Volume 24, Issue 4 , 2021 , Pages 3381-3403 ; 13867857 (ISSN)
  9. URL: https://link.springer.com/article/10.1007/s10586-021-03339-8