Loading...

A combined analytical modeling machine learning approach for performance prediction of MapReduce jobs in cloud environment

Ataie, E ; Sharif University of Technology | 2017

1011 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/SYNASC.2016.072
  3. Publisher: Institute of Electrical and Electronics Engineers Inc , 2017
  4. Abstract:
  5. Nowadays MapReduce and its open source implementation, Apache Hadoop, are the most widespread solutions for handling massive dataset on clusters of commodity hardware. At the expense of a somewhat reduced performance in comparison to HPC technologies, the MapReduce framework provides fault tolerance and automatic parallelization without any efforts by developers. Since in many cases Hadoop is adopted to support business critical activities, it is often important to predict with fair confidence the execution time of submitted jobs, for instance when SLAs are established with end-users. In this work, we propose and validate a hybrid approach exploiting both queuing networks and support vector regression, in order to achieve a good accuracy without too many costly experiments on a real setup. The experimental results show how the proposed approach attains a 21% improvement in accuracy over applying machine learning techniques without any support from analytical models. © 2016 IEEE
  6. Keywords:
  7. Analytical performance modeling ; Cloud computing ; Machine learning ; MapReduce ; Artificial intelligence ; Cloud computing ; Computer software ; Fault tolerance ; Learning systems ; Open systems ; Analytical performance model ; Automatic Parallelization ; Machine learning techniques ; Map-reduce ; Mapreduce frameworks ; Open source implementation ; Performance prediction ; Support vector regression (SVR) ; Analytical models
  8. Source: 18th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, SYNASC 2016, 24 September 2016 through 27 September 2016 ; 2017 , Pages 431-439 ; 9781509057078 (ISBN)
  9. URL: https://ieeexplore.ieee.org/document/7829644