Run a workflow on Ranger using Airavata

This post will show you how to run a workflow on Ranger [1] using Apache Airavata.

Applies to : Airavata 0.5-SNAPSHOT

Checkout and build Airavata

1. Checkout the source from the svn trunk

svn co https://svn.apache.org/repos/asf/airavata/trunk

2. Move to the root directory and build the trunk using maven.

eg: build with tests

mvn clean install

eg: build skipping tests

mvn clean install -Dmaven.test.skip=true -o

3. Unzip the apache-airavata-0.5-SNAPSHOT-bin.zip pack. The unzipped directory will be called AIRAVATA_HOME hereafter.

How to start the Airavata System Components with MySQL

Airavata is running using Apache Derby as the backend database. You can use MySQL database instead of default derby database if you wish.

1. Create a new user called airavata with password airavata in running MySQL instance

mysql> CREATE USER 'airavata'@'localhost' IDENTIFIED BY 'airavata';

2. Create a mysql database in your server with required name for user airavata (Assume name is:persistent_data)

mysql> CREATE DATABASE persistent_data;

GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP
ON persistent_data.*
TO 'airavata'@'localhost'
IDENTIFIED BY 'airavata';

3. Copy mysql driver jar (-5.1.6.jar) to /standalone-server/lib

4. Edit jdbc connection URLs at repository.properties file that resides in /standalone-server/conf. You can achieve this by just uncommenting out parameters under "#configuration for mysql" section and commented-out the default derby configurations (If you followed the above instructions you might not have to change this repository.properties file.)

registry.jdbc.url=jdbc:mysql://localhost:3306/persistent_data
registry.jdbc.driver=com.mysql.jdbc.Driver

5. In the next section we will try to run a workflow in Ranger [1]. Configure the repository.properties file by modifying dummy values for following properties. The XSEDE cert files can be downloaded from [2].

trusted.cert.location=path to certificates for ranger
myproxy.server=myproxy.teragrid.org
myproxy.user=username
myproxy.pass=password
myproxy.life=3600

5. Go to /bin and start the script airavata-server.sh. Now Airavata server will start with MySQL database (in this case persistent_data database).

6. When starting XBaya, it should also point to the same database that we specified when starting Airavata server. In order to do that, copy the same repository.properties file that you have edited in the 2nd step, to /bin folder. Now you are ready to start XBaya. To start XBaya, go to /bin folder and run the script xbaya-gui.sh. Now XBaya will start with the same database that used to start Airavata server.

Registering An application which will run on Ranger

1. Click on XBaya --> Register Application.  Then fill in the following values as it is shown in the diagram.

2. Click New Deployment button. Following dialog box will then appear. Select Ranger as the host from the drop down menu. Then fill in the application that you are going to invoke (in this case /bin/echo) and set the proper scratch directory. The scratch directory can be identified by running the following command on Ranger once you are logged into it through your shell.

echo $SCRATCH

3. Then click "HPC Configuration" button. Select job type as serial. Specify the project account assigned to you. Finally is specify the queue as development as your job will be submitted to this queue. You can configure the other fields as well; but if you don't set them, default values will be added for them. Then click "update" button.

4. Click "Add" button.

5. Click "Register" button.

6. If the registration was successful, following message will appear.

7. Click XBaya Menu and select "New Workflow".

8. Give a name for your workflow and hit "Ok".

9. Your application will be listed under "Application Services" in the left menu.

10. Drag it and drop on to the workflow window.

11. Select "input" and "output" components from Component list. Drag them to the workflow window. Then connect the dots using the mouse pointer as it is shown in the below diagram.

12. Then hit the red colored "play" button on the top left corner. Following dialog will appear. Give an input for your workflow and a experiment name for the workflow run. Then hit "Run". Now your job will be launched on Ranger.

[1] - http://www.tacc.utexas.edu/user-services/user-guides/ranger-user-guide
[2] - https://software.xsede.org/security/xsede-certs.tar.gz
[3] - http://airavata.apache.org/documentation/system/airavata-in-10-minutes.html