Big data processing tools: An experimental performance evaluation

Big Data is currently a hot topic of research and development across several business areas mainly due to recent innovations in information and communication technologies. One of the main challenges of Big Data relates to how one should efficiently handle massive volumes of complex data. Due to the...

ver descrição completa

Detalhes bibliográficos
Autor principal: Rodrigues, Mário (author)
Outros Autores: Santos, Maribel Yasmina (author), Bernardino, Jorge (author)
Formato: article
Idioma:eng
Publicado em: 2019
Assuntos:
Texto completo:http://hdl.handle.net/1822/60417
País:Portugal
Oai:oai:repositorium.sdum.uminho.pt:1822/60417
Descrição
Resumo:Big Data is currently a hot topic of research and development across several business areas mainly due to recent innovations in information and communication technologies. One of the main challenges of Big Data relates to how one should efficiently handle massive volumes of complex data. Due to the notorious complexity of the data that can be collected from multiple sources, usually motivated by increasing data volumes gathered at high velocity, efficient processing mechanisms are needed for data analysis purposes. Motivated by the rapid growth in technology, development of tools, and frameworks for Big Data, there is much discussion about Big Data querying tools and, specifically, those that are more appropriated for specific analytical needs. This paper describes and evaluates the following popular Big Data processing tools: Drill, HAWQ, Hive, Impala, Presto, and Spark. An experimental evaluation using the Transaction Processing Council (TPC-H) benchmark is presented and discussed, highlighting the performance of each tool, according to different workloads and query types. This article is categorized under: Technologies > Computer Architectures for Data Mining Fundamental Concepts of Data and Knowledge > Big Data Mining Technologies > Data Preprocessing Application Areas > Data Mining Software Tools.