Thomas Vanhove, Gregory Van Seghbroeck, Tim Wauters, Filip De Turck, Brecht Vermeulen, Piet Demeester
Big data applications have stringent service requirements for scalability and fault-tolerance and involve high volumes of data, high processing speeds and large varieties of database technologies. In order to test big data management solutions, large experimentation facilities are needed, which are expensive in terms of both resource cost and configuration time. This paper presents Tengu, an experimentation platform for big data applications that can automatically be instantiated on GENI (US federation of testbeds) and Fed4FIRE (EU federation of testbeds) compatible testbeds. Tengu allows for automatic deployments of several data processing, storage and cloud technologies, including Hadoop, Storm and Openstack. The paper discusses the Tengu architecture, the Tengu-as-a-service approach and a demonstration of an automated instantiation of the Tengu experimentation suite on the Virtual Wall, a large-scale Emulab testbed at the iMinds research institute in Europe.
Published in the Proceedings of the International Workshop on Computer and Networking Experimental Research Using Testbeds (CNERT), 2015