Evaluating SQL-on-Hadoop for Big Data warehousing on not-so-good hardware

Big Data is currently conceptualized as data whose volume, variety or velocity impose significant difficulties in traditional techniques and technologies. Big Data Warehousing is emerging as a new concept for Big Data analytics. In this context, SQL-on- Hadoop systems increased notoriety, providing...

ver descrição completa

Detalhes bibliográficos
Autor principal: Santos, Maribel Yasmina (author)
Outros Autores: Costa, Carlos (author), Galvão, João (author), Andrade, Carina (author), Martinho, Bruno Augusto (author), Lima, Francisca Vale (author), Costa, Eduarda (author)
Formato: conferencePaper
Idioma:eng
Publicado em: 2017
Assuntos:
Texto completo:http://hdl.handle.net/1822/46856
País:Portugal
Oai:oai:repositorium.sdum.uminho.pt:1822/46856
Descrição
Resumo:Big Data is currently conceptualized as data whose volume, variety or velocity impose significant difficulties in traditional techniques and technologies. Big Data Warehousing is emerging as a new concept for Big Data analytics. In this context, SQL-on- Hadoop systems increased notoriety, providing Structured #ery Language (SQL) interfaces and interactive queries on Hadoop. A benchmark based on a denormalized version of the TPC-H is used to compare the performance of Hive on Tez, Spark, Presto and Drill. Some key contributions of this work include: the direct comparison of a vast set of technologies; unlike previous scientific works, SQL-on-Hadoop systems were connected to Hive tables instead of raw files; allow to understand the behaviour of these systems in scenarios with ever-increasing requirements, but not-so-good hardware. Besides these benchmark results, this paper also makes available interesting findings regarding an architecture and infrastructure in SQL-on- Hadoop for Big Data Warehousing, helping practitioners and fostering future research.