Showing posts with label Workflow. Show all posts
Showing posts with label Workflow. Show all posts

Friday, April 5, 2013

Run EC2 Jobs with Airavata - Part III

This is a followup to my earlier posts [1] [2]. Here we will execute the application mentioned in [2] programmetically using Airavata.

import org.apache.airavata.commons.gfac.type.*;
import org.apache.airavata.gfac.GFacAPI;
import org.apache.airavata.gfac.GFacConfiguration;
import org.apache.airavata.gfac.GFacException;
import org.apache.airavata.gfac.context.security.AmazonSecurityContext;
import org.apache.airavata.gfac.context.ApplicationContext;
import org.apache.airavata.gfac.context.JobExecutionContext;
import org.apache.airavata.gfac.context.MessageContext;
import org.apache.airavata.schemas.gfac.*;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;

import java.io.File;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;

/**
 * Your Amazon instance should be in a running state before running this test.
 */
public class EC2ProviderTest {
    private JobExecutionContext jobExecutionContext;

    private static final String hostName = "ec2-host";

    private static final String hostAddress = "ec2-address";

    private static final String sequence1 = "RR042383.21413#CTGGCACGGAGTTAGCCGATCCTTATTCATAAAGTACATGCAAACGGGTATCCATA" +
            "CTCGACTTTATTCCTTTATAAAAGAAGTTTACAACCCATAGGGCAGTCATCCTTCACGCTACTTGGCTGGTTCAGGCCTGCGCCCATTGACCAATATTCCTCA" +
            "CTGCTGCCTCCCGTAGGAGTTTGGACCGTGTCTCAGTTCCAATGTGGGGGACCTTCCTCTCAGAACCCCTATCCATCGAAGACTAGGTGGGCCGTTACCCCGC" +
            "CTACTATCTAATGGAACGCATCCCCATCGTCTACCGGAATACCTTTAATCATGTGAACATGCGGACTCATGATGCCATCTTGTATTAATCTTCCTTTCAGAAG" +
            "GCTGTCCAAGAGTAGACGGCAGGTTGGATACGTGTTACTCACCGTGCCGCCGGTCGCCATCAGTCTTAGCAAGCTAAGACCATGCTGCCCCTGACTTGCATGT" +
            "GTTAAGCCTGTAGCTTAGCGTTC";

    private static final String sequence2 = "RR042383.31934#CTGGCACGGAGTTAGCCGATCCTTATTCATAAAGTACATGCAAACGGGTATCCATA" +
            "CCCGACTTTATTCCTTTATAAAAGAAGTTTACAACCCATAGGGCAGTCATCCTTCACGCTACTTGGCTGGTTCAGGCTCTCGCCCATTGACCAATATTCCTCA" +
            "CTGCTGCCTCCCGTAGGAGTTTGGACCGTGTCTCAGTTCCAATGTGGGGGACCTTCCTCTCAGAACCCCTATCCATCGAAGACTAGGTGGGCCGTTACCCCGC" +
            "CTACTATCTAATGGAACGCATCCCCATCGTCTACCGGAATACCTTTAATCATGTGAACATGCGGACTCATGATGCCATCTTGTATTAAATCTTCCTTTCAGAA" +
            "GGCTATCCAAGAGTAGACGGCAGGTTGGATACGTGTTACTCACCGTGCG";

    /* Following variables are needed to be set in-order to run the test. Since these are account specific information,
       I'm not adding the values here. It's the responsibility of the person who's running the test to update
       these variables accordingly.
       */

    /* Username used to log into your ec2 instance eg.ec2-user */
    private String userName = "";

    /* Secret key used to connect to the image */
    private String secretKey = "";

    /* Access key used to connect to the image */
    private String accessKey = "";

    /* Instance id of the running instance of your image */
    private String instanceId = "";

    @Before
    public void setUp() throws Exception {
        URL resource = GramProviderTest.class.getClassLoader().getResource("gfac-config.xml");
        assert resource != null;
        System.out.println(resource.getFile());
        GFacConfiguration gFacConfiguration = GFacConfiguration.create(new File(resource.getPath()), null, null);

        /* EC2 Host */
        HostDescription host = new HostDescription(Ec2HostType.type);
        host.getType().setHostName(hostName);
        host.getType().setHostAddress(hostAddress);

        /* App */
        ApplicationDescription ec2Desc = new ApplicationDescription(Ec2ApplicationDeploymentType.type);
        Ec2ApplicationDeploymentType ec2App = (Ec2ApplicationDeploymentType)ec2Desc.getType();

        String serviceName = "Gnome_distance_calculation_workflow";
        ec2Desc.getType().addNewApplicationName().setStringValue(serviceName);
        ec2App.setJobType(JobTypeType.EC_2);
        ec2App.setExecutable("/home/ec2-user/run.sh");
        ec2App.setExecutableType("sh");

        /* Service */
        ServiceDescription serv = new ServiceDescription();
        serv.getType().setName("GenomeEC2");

        List inputList = new ArrayList();

        InputParameterType input1 = InputParameterType.Factory.newInstance();
        input1.setParameterName("genome_input1");
        input1.setParameterType(StringParameterType.Factory.newInstance());
        inputList.add(input1);

        InputParameterType input2 = InputParameterType.Factory.newInstance();
        input2.setParameterName("genome_input2");
        input2.setParameterType(StringParameterType.Factory.newInstance());
        inputList.add(input2);

        InputParameterType[] inputParamList = inputList.toArray(new InputParameterType[inputList.size()]);

        List outputList = new ArrayList();
        OutputParameterType output = OutputParameterType.Factory.newInstance();
        output.setParameterName("genome_output");
        output.setParameterType(StringParameterType.Factory.newInstance());
        outputList.add(output);

        OutputParameterType[] outputParamList = outputList
                .toArray(new OutputParameterType[outputList.size()]);

        serv.getType().setInputParametersArray(inputParamList);
        serv.getType().setOutputParametersArray(outputParamList);

        jobExecutionContext = new JobExecutionContext(gFacConfiguration,serv.getType().getName());
        ApplicationContext applicationContext = new ApplicationContext();
        jobExecutionContext.setApplicationContext(applicationContext);
        applicationContext.setServiceDescription(serv);
        applicationContext.setApplicationDeploymentDescription(ec2Desc);
        applicationContext.setHostDescription(host);

        AmazonSecurityContext amazonSecurityContext =
                new AmazonSecurityContext(userName, accessKey, secretKey, instanceId);
        jobExecutionContext.addSecurityContext(AmazonSecurityContext.AMAZON_SECURITY_CONTEXT, amazonSecurityContext);

        MessageContext inMessage = new MessageContext();
        ActualParameter genomeInput1 = new ActualParameter();
        ((StringParameterType)genomeInput1.getType()).setValue(sequence1);
        inMessage.addParameter("genome_input1", genomeInput1);

        ActualParameter genomeInput2 = new ActualParameter();
        ((StringParameterType)genomeInput2.getType()).setValue(sequence2);
        inMessage.addParameter("genome_input2", genomeInput2);

        MessageContext outMessage = new MessageContext();
        ActualParameter echo_out = new ActualParameter();
        outMessage.addParameter("distance", echo_out);

        jobExecutionContext.setInMessageContext(inMessage);
        jobExecutionContext.setOutMessageContext(outMessage);
    }

    @Test
    public void testGramProvider() throws GFacException {
        GFacAPI gFacAPI = new GFacAPI();
        gFacAPI.submitJob(jobExecutionContext);
        MessageContext outMessageContext = jobExecutionContext.getOutMessageContext();
        Assert.assertEquals(MappingFactory.
                toString((ActualParameter) outMessageContext.getParameter("genome_output")), "476");
    }
}

References
[1] - http://heshans.blogspot.com/2013/04/run-ec2-jobs-with-airavata-part-i.html
[2] - http://heshans.blogspot.com/2013/04/run-ec2-jobs-with-airavata-part-ii.html 

Run EC2 Jobs with Airavata - Part II

In this post we will look at how to compose a workflow out of an application that is installed in an Amazon Machine Image (AMI). In the earlier post we discussed how to do ec2 instance management using XBaya GUI. This is the followup to that post.

For the Airavata EC2 integration testing, I created an AMI which has an application which does gene sequence alignment using Smith-Waterman algorithm. I will be using that application as a reference to this post. You can use an application of your preference that resides in your AMI.

1. Unzip Airavata server distribution and start the server.
unzip apache-airavata-server-0.7-bin.zip
cd apache-airavata-server-0.7/bin
./airavata-server.sh

2. Unzip Airavata XBaya distribution and start XBaya.
unzip apache-airavata-xbaya-gui-0.7-bin.zip
cd apache-airavata-xbaya-gui-0.7/bin
./xbaya-gui.sh

Then you'll get the XBaya UI.


3. Select "XBaya" Menu and click "Add Host" to register an EC2 Host. Once you add the details, click   "ok".


4. You will then be prompted to enter "Airavata Registry" information. If you are using the default setup, you don't have to do any configuration. Just click "ok".


5. In order to use your application installed in the AMI, you must register it as an application in Airavata system. Select "XBaya" menu and click "Register Application". You will get the following dialog. Add the input parameters expected and the output parameters generated by your application.


6. Then Click the "New deployment" button. You have to then select the EC2Host that you registered earlier as the Application Host. Configure the executable path to your application in your AMI and click "Add".


7. Then click "Register". If the application registration was successful, you will be getting the following message.


8. Now select "Registry" menu and click "Setup Airavata Registry". Click "ok".


9. Select "XBaya" menu and click "New workflow". Then configure it accordingly.


10. Select your registered application from the "Application Services" and drag drop it to the workflow window.


11. Drag an "Instance" component from "Amazon Components" and drop it into workflow window. Then connect it to your application using Control ports.


12. Click on top of the "Instance" components config label. Configure your instance accordingly.


13. Drag and drop two input components and one output component to the workflow from "System Components".


14. Connect the components together accordingly.


15. Now click the red colored "play" button to run your workflow. You will be prompted for the inputs   values (in my case the gene sequences) and experiment id. Then click "Run" to execute your workflow.


16. The execution result will be shown in the XBaya GUI.


References
[1] - http://heshans.blogspot.com/2013/04/run-ec2-jobs-with-airavata-part-i.html

Run EC2 Jobs with Airavata - Part I

This will be the first of  many posts that I will be doing on Apache Airavata EC2 integration. First let's have a look at how you can use Airavata's "XBaya GUI" to manage amazon instances.

Applies to : Airavata 0.7 and above

1. Unzip Airavata server distribution and start the server.
unzip apache-airavata-server-0.7-bin.zip
cd apache-airavata-server-0.7/bin
./airavata-server.sh
2. Unzip Airavata XBaya distribution and start XBaya.
unzip apache-airavata-xbaya-gui-0.7-bin.zip
cd apache-airavata-xbaya-gui-0.7/bin
./xbaya-gui.sh
Then you'll get the XBaya UI.


3. Then Select "Amazon" menu and click "Security Credentials". Specify your secret key and access key in the security credentials dialog box and click "ok".


4. Then Select "Amazon" menu and click "EC2 Instance Management". It will give a glimpse of your running instances.


5. Click the "launch" button to launch new instances and "terminate" button to terminate, running instances.


6. When you launch a new instance, it will be showed in your "Amazon EC2 Management Console".



Friday, March 15, 2013

Airavata Deployment Studio (ADS)


This is an independent study that I have been doing for Apache Airavata [1]. Airavata Deployment Studio or simply ADS, is a platform where an Airavata user can deploy his/her Airavata deployment on a Cloud computing resource on demand. Now let's dive into ADS and what's the actual problem that we are trying the solve here. 


What is Airavata? 


Airavata is a framework which enables a user to build Science Gateways. It is used to compose, manage, execute and monitor distributed applications and workflows on computational resources. These computational resources can range from local resources to computational grids and clouds. Therefore, various users with different backgrounds either contribute or use Airavata in their applications.



Who uses Airavata? 

From the Airavata standpoint, three main users can be identified.


1) End Users


End User is the one who will have a model code to do some scientific application. Sometimes this End User can be a Research Scientist. He/She writes scripts to wrap the applications up and by executing those scripts, they run the scientific workflows in Super Computers. This can be called a scientific experiment.

2) Gateway Developers


The Research Scientist is the one who comes up with requirement of bundling scientific applications together and composing as a workflow. The job of the Gateway Developer is to use Airavata and wrap the above mentioned model code and scripts together. Then, scientific workflows are created out these. In some cases, Scientist might be the Gateway Developer as well.

3) Core Developers


Core Developer is the one who develops and contributes to Airavata framework code-base. The Gateway Developers use the software developed by the Core Developers to create science gateways.

Why ADS?

According to the above description, Airavata is used by different people with different technical backgrounds. Some people will have in depth technical knowledge on their scientific domains; like chemistry, biology, astronomy, etc and may not have in depth knowledge on computer science aspects such as cluster configuration, configuring and trouble-shooting in VMs, etc. 

When it comes to ADS, it's targeted towards the first two types of users as they will be ones who will be running in to configuration issues with Airavata in their respective systems. 

Sometimes we come across instances where a user might run into issues while setting up Airavata on their Systems. These might be attributed to; 
  1. User not following the documented steps properly.
  2. Issues in setting up the user environment. 
  3. User not being able to diagnose the issues at their end on their own.
  4. Sometimes when we try to diagnose their issue remotely, we face difficulties trying to access user's VM remotely due to security policies defined in their System. 
  5. Different security policies at client's firewall.

Due to the above mentioned issues, a first time user might go away with a bad impression due to a System/VM level issue that might not be directly related to Airavata. 

What we are trying to do here is to give a first time user a good first impression as well as ease of configuring the Airavata eco system for production usage. 

How? 

Now you might be wondering how does ADS achieve this? ADS will use FutureGrid [3] as the underlying resource platform for this application. If you are interested in learning about what FutureGrid is, please refer [3] for more information. ADS will ultimately become a plugin to the FutureGrid's CloudMesh [4] environment.

ADS will provide a user with a web interface which a user can use to configure his/her Airavata eco system. Once the configuration options are selected and user hits the submit button, a new VM with the selected configurations will be created. The user will be able to create his/her image with the following properties. 
  • Infrastructure - eg: OpenStack, Eucalyptus, EC2, etc
  • Architecture - eg: 64-bit, 32-bit 
  • Memory - eg: 2GB, 4GB, 8GB, etc
  • Operating System - eg: Ubuntu, CentOS, Fedora, etc
  • Java version - eg: Java 1.6, Java 1.7
  • Tomcat Version - eg: Tomcat6, Tomcat7
  • Airavata Version - eg: Airavata-0.6, Airavata-0.7

Advantages?

  1. One click install. 
  2. No need to interact with the shell to configure an Airavata environment.
  3. Deploying on various Cloud platforms based on user preference.
  4. Ease of use. 
  5. First time user will be able to quickly configure an insatnce of his own and run a sample workflow quickly. 
  6. On demand aspect.

Sneak Peak

Following screenshots show how ADS will look like.









References 


Friday, August 3, 2012

Apache Airavata 0.4-INCUBATING Released


The Apache Airavata (Incubating) team is pleased to announce the immediate availability of the Airavata 0.4-INCUBATING release.

The release can be obtained from the Apache Airavata download page - http://incubator.apache.org/airavata/about/downloads.html


Apache Airavata is a software toolkit currently used to build science gateways but that has a much wider potential use. It provides features to compose, manage, execute, and monitor small to large scale applications and workflows on computational resources ranging from local clusters to national grids and computing clouds. Gadgets interfaces to Airavata back end services can be deployed in open social containers such as Apache Rave and modify them to suit their needs. Airavata builds on general concepts of service oriented computing, distributed messaging, and workflow composition and orchestration.

For general information on Apache Airavata, please visit the project website:

Disclaimer:
 Apache Airavata is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects.  While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.

Thursday, July 5, 2012

Deploying Airavata Server on Tomcat


A shell script named setup_tomcat.sh is shipped with Airavata that will assist you to $subject. Following steps describe how you can do it.

1) Update tomcat.hostname and tomcat.port properties of the airavata-tomcat.properties file. You can keep the defaults if you dont want to change ports. In that case you don't have to edit the airavata-tomcat.properties file. This file can be found in AIRAVATA_HOME/tools/airavata-tomcat.properties.

2) Download following to your local file system.
a) apache-tomcat-7.0.28.zip
b) apache-airavata-0.4-incubating-SNAPSHOT-bin.zip
b) axis2-1.5.1-war.zip (Unzip it. When running the script point to the axis2.war)
d) ackrabbit-webapp-2.4.0.war

3) Run the script (setup_tomcat.sh) by providing the full file paths of the files you downloaded. This script can be found in AIRAVATA_HOME/tools/ directory.
./setup_tomcat.sh --tomcat=/home/heshan/Dev/setup/gw8/apache-tomcat-7.0.28.zip --airavata=/home/heshan/Dev/setup/gw8/apache-airavata-0.4-incubating-SNAPSHOT-bin.zip --axis2=/home/heshan/Dev/setup/gw8/axis2.war --jackrabbit=/home/heshan/Dev/setup/gw8/jackrabbit-webapp-2.4.0.war --properties=/home/heshan/Dev/setup/gw8/airavata-tomcat.properties

4) Start Tomcat server.
eg: ./catalina.sh start

5) Before using airavata go to http://localhost:8090/jackrabbit-webapp-2.4.0 and create a default content repository.

6) Restart Tomcat server.

Tuesday, July 3, 2012

Airavata Programming API


Apache Airavata's Programming API is the API which is exposed to the Gateway Developers. Gateway Developers can use this API to execute and monitor workflows. The API user should keep in mind that a user can not compose an Airavata workflow (.xwf) using the API. Inorder to do that a user can use the XBaya User Interface. Therefore, other than creation of the workflow; Client API supports all other workflow related operations.


The main motivation behind, having a Client API is that to expose the user to an API that will let him/her to a the persistent information stored in the Registry. The information persisted in the Registry can be;
  • Descriptors
  • Workflow information
  • Workflow provenance information
  • Airavata configuration

Following are the high level usecases which uses Airavata Client API.



Client API Usecases



  1. Registry Operations 
    • Retrieve registry information
    • Access registry information
    • Update registry information
    • Delete registry information
    • Search registry information
  2. Execute workflow
    • Run workflow
    • Set inputs
    • Set workflow node IDs 
  3. Monitoring
  4. Provenance
  5. User Management (This is not yet implemented. It's currently in our Road Map and this is added as a place holder.)
    • User roles
    • Administration



Client API Components



The Client API consists of 5 main components.
  1. Airavata API
    • It is an Aggregator API which contains all the base methods for Airavata API.
  2. Airavata Manager
    • This exposes config related information on Airavata. This currently contains Service URLs only.
  3. Application Manager
    • This will handle operations related to descriptors. Namely;
      1. Host description
      2. Service description
      3. Application description
  4. Execution Manager
    • This can be used to run and monitor workflows.
  5. Provenance Manger
    • This provides API to manage provenance related information. ie. Keeps track of inputs, outputs, etc related to a workflow.
  6. User Manger
    • User management related API is exposed through this. Currently, Airavata does not support User management but it is in Airavata roadmap.
  7. Workflow manager
    • Every operation related to workflows is exposed through this. ie:
      1. saving workflow
      2. deleting workflow
      3. retrieving workflow

Wednesday, June 27, 2012

Apache Airavata Stakeholders

Airavata Users
Airavata is a framework which enables a user to build Science Gateways. It is used to compose, manage, execute and monitor distributed applications and workflows on computational resources. These computational resources can range from local resources to computational grids and clouds. Therefore, various users with different backgrounds either contribute or use Airavata in their applications. From the Airavata standpoint, three main users can be identified.
  • Research Scientists (Gateway End Users)
  • Gateway Developers
  • Core Developers
Now let's focus on each user and how they fit into Airavata's big picture.

 

Gateway End Users

Gateway End Users
Gateway End User is the one who will have a model code to do some scientific application. Sometimes this End User can be a Research Scientist. He/She writes scripts to wrap the applications up and by executing those scripts, they run the scientific workflows in Super Computers. This can be called a scientific experiment. Now the Scientist might have a requirement to call multiple of these applications together and compose a workflow. That's where the Gateway Developer comes into the picture.

 

Gateway Developers

Gateway Developers
The Research Scientist is the one who comes up with requirement of bundling scienitific applications together and composing as a workflow.
The job of the Gateway Developer is to use Airavata and wrap the above mentioned model code and scripts together. Then, scientific workflows are created out these.
Above diagram depicts how Gateway Developer fits into the picture.

 

Core Developers

Core Developers

Core Deveoper is the one who develops and contributes to Airavata framework codebase. The Gateway Developers use the software developed by the Core Developers to create science gateways.

Thursday, June 14, 2012

Apache Airavata ForEach construct

This post will introduce the ForEach construct and what are it's uses. Then we'll look at how to ForEach samples shipped in with Apache Airavata [1]. 

Introduction
Airavata supports parametric sweeps through a construct called Foreach. Given an array of inputs of size n and an application service, it is possible to run the application service n times for each of the input values in the input array. There are two modes of ForEach;
  1. Cartesian product of inputs: Inputs of array sizes n1 ,n2 , .. nk will yield:  
    • n1 ∗ n2 ∗ ... ∗ nk invocations.
  2. Dot product of inputs. Inputs of equal size arrays of size n will yield n computations.
Executing a simple workflow which uses ForEach
Apache Airavata is shipped with a workflow named SimpleForEach.xwf. As the name suggests, it contains a simple workflow which demonstrates the use of ForEach construct. Following are the steps to running the sample workflow.

Instructions to run the sample
  1. Download the latest Airavata release pack from downloads link and extract, for future explanation lets assume the path to extracted directory is AIRAVATA_HOME.

    NOTE: The above mentioned workflow samples are committed to the trunk. Therefore, you are better off using a trunk build.

  2. Now Run the following scripts in the given order below to start the components of Airavata.

    AIRAVATA_HOME/bin/jackrabbit-server.sh - This will start Jackrabbit repository on port 8081.

    AIRAVATA_HOME/bin/airavata-server.sh - This will start SimpleAxis2Server on port 8080

    AIRAVATA_HOME/bin/xbaya-gui.sh - This will start XBaya GUI application.
  3. Click the Xbaya tab and open up an Airavata workflow configuration (.xwf) from file system (sample workflows shipped in with Airavata can be found in AIRAVATA_HOME/samples/workflows) eg. Assume that you selected the SimpleForEach workflow
    • Now Click on the run button (red colored play).
    • Then the workflow will get executed.
    • Finally the result of the workflow will get displayed.
The workflow which we ran early looks like following (Figure 1.)  

Figure 1 : SimpleForEach sample
When running the workflow if you tick the "Execute in cross product" condition in the "Execute Workflow" dialog box; the workflow will be run in cross product. Otherwise, by default a given ForeEach will run in dot product. "Execute Workflow" dialog box is shown in Figure 2.

Figure 2 : Execute Workflow Dialog Box
Results
As you might have noticed in Figure 2, we have given two arrays (containing 1,2 and 3,4) as input to the ForEach construct.

If we ran the dot product, there will be 2 resulting outputs. It can be seen in Figure 3.

Figure 3: Dot product result

If we ran the cross product, there will be 4 resulting outputs. It can be seen in Figure 4. 

Figure 4: Cross product result
Apache Airavata contains another sample named ComplexForEach.xwf, which demonstrates the use of multiple ForEach constructs inside one workflow. Have fun running the ComplexForEach sample

Figure 5: ComplexForEach sample



REFERNCES
[1] - airavata.org

Running Airavata sample workflows


The purpose of this post is to give an understanding of how to run sample workflows shipped in with Airavata. If you are a new user and would like to acquire basic understanding of how to use Airavata, please refer the 5 minute tutorial and then refer the 10 minute tutorial. If you are familiar with Airavata and would like to run a sample workflow, this is the right place for you.


Introduction


This post will explain how to run a workflow, using an existing Airavata Workflow Configuration. Airavata currently, ships sample workflow configurations with it's distribution. The Samples included are;


  1. SimpleMath workflow
  2. ComplexMath workflow
  3. LevenshteinDistance workflow

Note: Currently Airavata will work with Linux distributions and Mac and we do not support all the Apache Airavata components to work on Windows.



Running a sample


  • Download the latest Airavata release pack from downloads link and extract, for future explanation lets assume the path to extracted directory is AIRAVATA_HOME.
  • Now Run the following scripts in the given order below to start the components of Airavata.

AIRAVATA_HOME/bin/jackrabbit-server.sh - This will start Jackrabbit repository on port 8081.

AIRAVATA_HOME/bin/airavata-server.sh - This will start SimpleAxis2Server on port 8080

AIRAVATA_HOME/bin/xbaya-gui.sh - This will start XBaya GUI application.
  • Click the Xbaya tab and open up an Airavata workflow configuration (.xwf) from file system (sample workflows shipped in with Airavata can be found in AIRAVATA_HOME/samples/workflows) eg. Assume that you selected the SimpleMath workflow
    • Now Click on the run button (red colored play).
    • Then the workflow will get executed.
    • Finally the result of the workflow will get displayed.
    • Similarly, other workflows can be executed.

Workflow Samples



Basic samples


  1. SimpleMath workflow
    • This workflow will hand over the inputs to 4 nodes. Then the results will be handed over to another 2 nodes which will then hand over the results to another node. The last node will output the result of the operation. All the nodes considered are doing addition operations.
  2. ComplexMath workflow
    • This workflow will hand over the inputs to 4 nodes which are doing addition operations. Then the outputs(results) will be handed over to another 2 nodes which are doing multiplication operation. The results of the multiplications are handed over to another node. The last node will do addition operation on the input data and output the resulting value.

Advanced Samples


XBaya support Parametric Sweeps which can be used to tackle uncertainty of inputs to a workflow. It supports, Cartesian product and Dot product of inputs.

  1. Levenshtein Distance workflow
    • This workflow will use Airavata's ForEach construct to calculate Levenshtein Distance of strings. This workflow will use cross product to calculate distance.             

Friday, March 16, 2012

How to Deploy Apache Airavata on Tomcat

Applies to
  • Airavata Trunk - apache-airavata-0.3-incubating-SNAPSHOT.zip
  • Axis version - Axis2 1.5.1.war
  • Jackrabbit version - jackrabbit-webapp-2.4.0.war
  • Tomcat version - apache-tomcat-7.0.23
Airavata services depends on Apache Axis2 as a Web Service engine. In order to deploy Airavata system on Tomcat, you need to deploy Apache Axis2 on Tomcat first.

1) Copy all the service jar files in AIRAVATA_HOME/standalone-server/repository/services/ directory to Tomcat axis2 services directory(TOMCAT_HOME/webapps/axis2/WEB-INF/services)

2) Copy all the dependent jars (copy all the jars in AIRAVATA_HOME/lib and AIRAVATA_HOME/standalone/lib ) to TOMCAT_HOME/webapps/axis2/WEB-INF/lib.

3) Copy the properties files for messageBox (msgBroker.properties) and messageBroker (msgBroker.properties) to TOMCAT_HOME/webapps/axis2/WEB-INF/classes/conf/. These properties files can be found in AIRAVATA_HOME/standalone-server/conf directory.

4) Copy the properties files for GFac Service (repository.properties) and Interpretor Service (xbaya.properties) to TOMCAT_HOME/webapps/axis2/WEB-INF/lib/ directory. These properties files can be found in AIRAVATA_HOME/standalone-server/conf directory.

i) Based on your setup configure the GSI security credentials and certificate paths in the repository.properties file.

ii) Fill up the following property names values in your xbaya.properties file based on your gsi security credentials.
eg : myproxy.user
myproxy.password

NOTE: keep the same repository.properties file in TOMCAT_HOME/bin directory too. This issue will be fixed in later release.

4) Download and copy a database library (Apache Derby or MySql) to webapps/axis2/WEB-INF/lib directory.
eg: derby-10.7.1.1.jar
derbytools-10.7.1.1.jar

5) Copy directory AIRAVATA_HOME/standalone-server/bin/database_scripts to TOMCAT_HOME/webapps/axis2/WEB-INF/classes/database_scripts directory.

6) Now we have configured a Tomcat instance to run Airavata Services. Next is to deploy Jacrabbit instance on a separate Tomcat instance(I'm referring it's home directory as TOMCAT_HOME2).

7) Deploy Jackrabbit on Tomcat

i) Download jcr-2.0.jar and put it into TOMCAT_HOME2/webapps/jackrabbit-webapp-2.4.0/WEB-INF/lib/.

ii) Get the WAR distribution from the Downloads page and deploy it into Tomcat.

iii) Make sure to change and configure the ports by editing TOMCAT_HOME2/conf/server.xml file.

8) Now startup the Jacrabbit deployed Tomcat instance.

9) Startup the Tomcat instance with Airavata services.

10) Go to service listing page and check the service status. If there are no faulty services, then you have done the setup properly.