Loading...

Optimal capacity allocation for executing mapreduce jobs in cloud systems

Malekimajd, M ; Sharif University of Technology | 2015

605 Viewed
  1. Type of Document: Article
  2. DOI: 10.1109/SYNASC.2014.58
  3. Publisher: Institute of Electrical and Electronics Engineers Inc , 2015
  4. Abstract:
  5. Nowadays, analyzing large amount of data is of paramount importance for many companies. Big data and business intelligence applications are facilitated by the MapReduce programming model while, at infrastructural layer, cloud computing provides flexible and cost effective solutions for allocating on demand large clusters. Capacity allocation in such systems is a key challenge to providing performance for MapReduce jobs and minimize cloud resource cost. The contribution of this paper is twofold: (i) we formulate a linear programming model able to minimize cloud resources cost and job rejection penalties for the execution of jobs of multiple classes with (soft) deadline guarantees, (ii) we provide new upper and lower bounds for MapReduce job execution time in shared Hadoop clusters. Moreover, our solutions are validated by a large set of experiments. We demonstrate that our method is able to determine the global optimal solution for systems including up to 1000 user classes in less than 0.5 seconds. Moreover, the execution time of MapReduce jobs are within 19% of our upper bounds on average
  6. Keywords:
  7. MapReduce ; Performance bounds ; Cloud computing ; Cost effectiveness ; Costs ; Linear programming ; Business intelligence applications ; Capacity allocation ; Cost-effective solutions ; Global optimal solutions ; Linear programming models ; Optimal capacity allocation ; Big data
  8. Source: Proceedings - 16th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, SYNASC 2014, 22 September 2014 through 25 September 2014 ; Sept , 2015 , Pages 385-392 ; 9781479984480 (ISBN)
  9. URL: http://ieeexplore.ieee.org/document/7034708