Get Started

IMPORTANT NOTICE: From December 3rd until December 8th maintenance will be performed on the automated setup of the Tengu platform. We don’t anticipate any downtime of the service, but in any case regular updates will be provided here and through Twitter.

Step 0: Easy Tengu

In what follows we advice to use cURL , a nice command line tool that can be used as HTTP client. If, however, cURL is too lowlevel, we also provide a simple easy-to-use webform to set up the different Tengu platforms.

 Step 1: Prerequisites

The only thing you currently need, is a valid Fed4FIRE account. Documentation concerning Fed4FIRE and especially how to obtain such an account, can be found here.

The RESTful API is a combination of GET and POST HTTP requests. For the GET requests it suffices to have a standard browser, but for the POST requests this is not enough. Some browsers already have extensions to do RESTful requests, but we advice to use a tool such as cURL for the RESTful API requests. All examples provided here and in the document section are shown using cURL.

Step 2: Deploy your first Tengu core setup

Setting up a Tengu core experiment is pretty straightforward, at least the HTTP POST command you need. Authentication is a bit less straightforward. Tengu uses Fed4FIRE’s speaks-for protocol to setup experiments on the Fed4FIRE testbeds. What this basically means, is that you (as a Fed4FIRE user) grants Tengu the sufficient rights to setup experiments on your behalf. So, you allow Tengu to speak for you. What this practically means, a lot of hassling and signing of certificates and encoding and encryption.

Luckily we created an easy way to get this signed speaks-for certificate for you: via the easy-to-use webform. After authentication, you can simply copy the base64 encoding of your signed speaks-for certificate. We will refer to this base64 encoded string as s4cert.

As already mentioned, the actual POST is very simple.

$ curl -kgi6 -X POST "http://[2001:6a8:1d80:23::141]:8280/tengu/core?hnodes=2&snodes=1&testbed=urn%3Apublicid%3AIDN%2Bwall2.ilabt.iminds.be%2Bauthority%2Bcm&s4cert=[s4cert]"

This will setup a Tengu core experiment with an Apache Hadoop cluster of size two and an Apache Storm cluster of size one.

Let us break down the URL.

  • The Tengu RESTful API can be reached using the ipv6 address 2001:6a8:1d80:23::141 and port 8280.
  • By using to path /tengu/core we tell the API to setup a Tengu core platform.
  • The Tengu core platform is configured using several parameters, provided as key-value pairs in the Query-string:
    • The testbed we want this Tengu core setup to be deployed to, is set by the testbed parameter. Its value is defined using the Fed4FIRE uuid of the testbed (here urn:publicid:IDN+wall2.ilabt.iminds.be+authority+cm, but be aware of URL encoding).
    • The size of the Apache Hadoop cluster is set via the hnodes parameter.
    • Similar for the size of the Apache Storm cluster, snodes.
    • Your signed base64 encoded speaks-for certificated (s4cert) is set via the parameter s4cert

A response similar to this one will be responded. This response includes the uuid (here urn:uuid:74584f5b-cc26-46cd-8ab9-5b42f53e95c0) that can be used to get information about the Tengu core setup.

HTTP/1.1 202 OK
Date: Wed, 11 Mar 2015 07:54:55 GMT
Content-Type: application/xml; charset=utf-8
Connection: close
Transfer-Encoding: chunked

<?xml version="1.0" encoding="UTF-8"?>
<ten:tengu xmlns:ten=http://tengu.ibcn.ugent.be/0/1/core xmlns:lnk="http://www.w3.org/1999/xhtml">
   <ten:platform>
      <ten:id>urn:uuid:74584f5b-cc26-46cd-8ab9-5b42f53e95c0</ten:id>
      <lnk:link method="get” href="/tengu/urn%3Auuid%3A74584f5b-cc26-46cd-8ab9-5b42f53e95c0” />
   </ten:platform>
</ten:tengu>

Step 3: Get information your deployed setup

Retrieving information (e.g. state information) about your deployed Tengu core setup is via the following HTTP GET.

$ curl -kig6 "http://[2001:6a8:1d80:23::141]:8280/tengu/urn%3Auuid%3A74584f5b-cc26-46cd-8ab9-5b42f53e95c0"

Depending on the current state of the deployment, the response will also include links to the interfaces of important components. In the case of a Tengu core setup, this will be the Hadoop Administration front end, the webUI of the HDFS Namenode, the webUI of the Storm cluster, the OpenStack Horizon UI and the webUI of the WSO2 ESB.

Also notice you did not have to provide any certificate anymore. This is because Tengu remembers you.

The format of the response will be as follows.

HTTP/1.1 200 OK
Date: Wed, 11 Mar 2015 08:22:03 GMT
Content-Type: application/xml; charset=utf-8
Connection: close
Transfer-Encoding: chunked

<?xml version="1.0" encoding="UTF-8"?>
<ten:tengu xmlns:ten=http://tengu.ibcn.ugent.be/0/1/core xmlns:lnk="http://www.w3.org/1999/xhtml">
   <ten:platform>
      <ten:id>{uuid}</ten:id>
      <ten:status>{UNKNOWN|READY|FAILED}</ten:status>
      <lnk:link method="...” rel="..." href="...” /> *
   </ten:platform>
</ten:tengu>

Step 4: Check the documentation

For more advanced usage of the RESTful API and more in-depth information about Tengu, we refer to the documentation.

For those eager to start deploying Hadoop MapReduce jobs, Apache Storm topologies, or other things, we have written an easy walkthrough to get acquainted with the different technologies.

Twitter