Loading...

Accelerating Big Data Stream Processing by FPGA-implementation of Parts of the Topology Graph

Kavand, Nima | 2017

596 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 50191 (19)
  4. University: Sharif University of Technology
  5. Department: Computer Engineering
  6. Advisor(s): Goudarzi, Maziar
  7. Abstract:
  8. In recent years, big data processing plays an important role in the era of information technology. The exponential growth of big data volume increases the need for data centers and infrastructures with more processing power. Due to dark silicon and scalability limitations in deep-submicron, the increasing trend of server performance slows down. Therefore, hardware accelerators such as FPGA and GPU are become increasingly popular for improvement of data center processing power. There are two types of big data processing based on the application: stream processing and batch processing. With the widespread use of social networks, online control systems and internet of things services, the stream data processing becomes more important. In stream data processing, addressing real time constraints is more vital compared to batch data processing. As a result, hardware accelerators becomes popular to improve performance of distributed stream processing frameworks such as apache storm. Although, researchers in previous studies try to employ FPGA in storm, they doesn`t propose a general solution for all applications. In this study, we propose an architecture to accelerate some parts of apache storm topology. Proposed architecture preserves scalability and generality of storm and its user interfaces are simple. The data transfer overhead between FPGA and host computer is low and data is transferred via PCIe link and DMA method. The overhead of hardware layer implementation on FPGA is extremely low and most of FPGA resources remains free for implementing user applications. In addition, an efficient algorithm is proposed to select some parts of topology for minimizing processor workload and decreasing server demand to obtain desired throughput. The integrated implementation of connected bolts instead of separated one decreases data transfer overhead and resourse utilization. This results in throughput improvement in most applications. Finally, we evaluate the proposed architecture with image processing applications and observed up to 38x improvement in throughput. According to simulation results, by running some parts of topology on FPGA, the number of required servers decreases from 2x to 83x without throughput degradation
  9. Keywords:
  10. Big Data ; Accelerators ; Field Programmable Gate Array (FPGA) ; Big Data Proccessing ; Apache Storm ; Stream Data Processing

 Digital Object List

 Bookmark

No TOC