Wednesday, June 13, 2012

Transfer a local file to Ranger using Airavata Client

This post will describe how to transfer a local file to Ranger [1] using Airavata Client.


NOTE: This post assumes that AiravataClient, GFac and the local_file will be in the same machine.

1. Before starting Airavata server, configure repository.properties file by modifying default fake values for following properties.
trusted.cert.location=path to certificates for ranger
myproxy.server=myproxy.teragrid.org
myproxy.user=username
myproxy.pass=password
myproxy.life=3600

Configure your certificate location and my proxy credentials and start Jackrabbit instance and airavata server with all the services.

2. To create a workflow to run on ranger you need to create Host Description, Application Description and a Service Description. If you  don’t know know how to create them refer airavata 10 minutes article [2] to understand how to create those documents. Once you become familiar with XBaya UI, use following values for fields given below and create documents using XBaya GUI.

Host Description
  • Host Address      - gatekeeper.ranger.tacc.teragrid.org
Click on the check box - Define this host as Globus host, then following two entries will enable to fillup.
  • Globus Gate Keeper Endpoint - gatekeeper.ranger.tacc.teragrid.org:2119/jobmanager-sge
  • Grid FTP Endpoint           - gsiftp://gridftp.ranger.tacc.teragrid.org:2811/
Service Description

Create a Service description as follows.

IO Parameter Name Type
input echo_input URI
output echo_output URI


Application Description
  • Executable path  - /share/home/01437/ogce/airavata-test/dir/file-breed.sh
  • Temporary Directory  -  /scratch/01437/ogce/test
Select the above created service descriptor and host Descriptor.. When you select the above created host descriptor you will see a new button Gram Configuration .. now click on that and fill following values.
  • Job Type  - Single
  • Project Account Number  - You will see this when you login to ranger, put your project Account number in this field
  • Project Description  - Description of the project - not mandetory
  • Queue Type  - development
Click on Update button and save the Application Description.

3. Running the workflow.

i) Using XBaya
Then Compose your workflow using the  service  created in Application Services. Now hit the red colored play button to run the workflow. It will prompt for a input file. Then give the input file path in following format (The file which you want to copy to Ranger).
file:/home/heshan/Dev/testing/airavata/temp.txt

Save the workflow in the registry (Xbaya Menu → Export → To Registry). This will be used when invoking the workflow using the AiravataClient.

ii) Using Airavata Client
public class AiravataClientTest {

    @Test
   public void testInvokeWorkflowString() {
       try {
           AiravataClient airavataClient = new AiravataClient("xbaya.properties");
           List<String> workflowTemplateIds = airavataClient.getWorkflowTemplateIds();
           Iterator<String> iterator = workflowTemplateIds.iterator();
           while(iterator.hasNext()) {
               String next = iterator.next();
               System.out.println("workflow : " + next);

               List<WorkflowInput> workflowInputs = airavataClient.getWorkflowInputs(next);
               Iterator<WorkflowInput> workflowInputIterator = workflowInputs.iterator();
               while(workflowInputIterator.hasNext()) {
                   WorkflowInput input = workflowInputIterator.next();
                   System.out.println("Name            :" + input.getName());
                   System.out.println("Type            :" + input.getType());
                   System.out.println("Value           :" + input.getValue());
                   System.out.println("Default Value   :" + input.getDefaultValue());
                   // System.out.println("input " + input);
               }

           }

           String workflowTemplateId = "GridFtp-local-workflow";
           List<WorkflowInput> workflowInputs = airavataClient.getWorkflowInputs(workflowTemplateId);
           String topicId = airavataClient.runWorkflow(workflowTemplateId, workflowInputs);

           WorkflowExecution workflowExecutionData = airavataClient.getWorkflowExecutionData(topicId);

       } catch (RegistryException e1) {
           e1.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
       } catch (IOException e1) {
           e1.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
       } catch (Exception e) {
           e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
       }
    }
}

[1] - http://www.tacc.utexas.edu/user-services/user-guides/ranger-user-guide
[2] - http://incubator.apache.org/airavata/documentation/system/airavata-in-10-minutes.html

3 comments:

Hasitha Aravinda said...

Hi Heshan,

This is a great tutorial. Thank you for sharing it with us.

But Recently I try this using Current Airavata client 0.4-incubating-SNAPSHOT version. And found that this is not working any more with 0.4-incubating-SNAPSHOT. Can you please give us guidance to how to fix it with new version.

Hasitha

Heshan Suriyaarachchi said...

Hi Hasitha,

Thanks for trying out the sample. Yes, may be with changes in the trunk, this might not work. I did not try it out now.

I think you wanted to try out an Airavata Client sample. If that is the case, Can you follow the updated article on how to use Airavata Client API [1].

https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Client+Quickstart

Heshan Suriyaarachchi said...

Hi Hasitha,

You need to have an account in Ranger inorder for you to run this tutorial. Therefore, you wont be able to execute this workflow.

Heshan.