Modeling agents working on ETL processes

All data warehousing systems architects know that is quite difficult to build a "perfect" ETL system without having primarily a model well founded on real operational requisites. Thus ETL modeling is crucial to the success of populating a data warehouse, independently from their natural co...

ver descrição completa

Detalhes bibliográficos
Autor principal: Gomes, Nuno André Barbosa (author)
Outros Autores: Oliveira, Bruno (author), Belo, Orlando (author)
Formato: conferencePaper
Idioma:eng
Publicado em: 2016
Assuntos:
Texto completo:http://hdl.handle.net/1822/54407
País:Portugal
Oai:oai:repositorium.sdum.uminho.pt:1822/54407
Descrição
Resumo:All data warehousing systems architects know that is quite difficult to build a "perfect" ETL system without having primarily a model well founded on real operational requisites. Thus ETL modeling is crucial to the success of populating a data warehouse, independently from their natural complexity. Modeling will also reduce implementation costs once it allows for revealing eventual operational errors originated by misinterpreted requisites. In this demo we demonstrate the viability of modeling an ETL system, using a standard workflow language and an ETL pattern approach implemented based on a cooperative community of opportunistic agents. The demo shows how we modeled, configured and executed an ETL system prototype in YAWL, and how we used agents to implement a self-organizing general purpose ETL system.