Friday, July 6, 2012

How to Submit Patches to Apache Airavata


This post describes how an Airavata user can contribute to the Airavata project by submitting patches. User can follow the steps given below.
  • Identify an issue that you want to fix or improve
  • Search JIRA and the mailing list to see if it’s already been discussed
  • If it’s a bug or a feature request, open a JIRA issue
  • Create a sample that you can use for prototyping the feature or demonstrating the bug. If creating a sample is time consuming, write steps to reproduce the issue.
  • Attach this sample to the JIRA issue if it’s representing a bug report.
  • Setup a svn client in your system.
  • Checkout the source code.
  • Make your changes
  • Create the patch:
    • svn add any_files_you_added
    • svn diff > /tmp/fix-AIRAVATA-NNNN.patch
  • Attach that file (/tmp/fix-AIRAVATA-NNNN.patch) to the JIRA

Thursday, July 5, 2012

Deploying Airavata Server on Tomcat


A shell script named setup_tomcat.sh is shipped with Airavata that will assist you to $subject. Following steps describe how you can do it.

1) Update tomcat.hostname and tomcat.port properties of the airavata-tomcat.properties file. You can keep the defaults if you dont want to change ports. In that case you don't have to edit the airavata-tomcat.properties file. This file can be found in AIRAVATA_HOME/tools/airavata-tomcat.properties.

2) Download following to your local file system.
a) apache-tomcat-7.0.28.zip
b) apache-airavata-0.4-incubating-SNAPSHOT-bin.zip
b) axis2-1.5.1-war.zip (Unzip it. When running the script point to the axis2.war)
d) ackrabbit-webapp-2.4.0.war

3) Run the script (setup_tomcat.sh) by providing the full file paths of the files you downloaded. This script can be found in AIRAVATA_HOME/tools/ directory.
./setup_tomcat.sh --tomcat=/home/heshan/Dev/setup/gw8/apache-tomcat-7.0.28.zip --airavata=/home/heshan/Dev/setup/gw8/apache-airavata-0.4-incubating-SNAPSHOT-bin.zip --axis2=/home/heshan/Dev/setup/gw8/axis2.war --jackrabbit=/home/heshan/Dev/setup/gw8/jackrabbit-webapp-2.4.0.war --properties=/home/heshan/Dev/setup/gw8/airavata-tomcat.properties

4) Start Tomcat server.
eg: ./catalina.sh start

5) Before using airavata go to http://localhost:8090/jackrabbit-webapp-2.4.0 and create a default content repository.

6) Restart Tomcat server.

Wednesday, July 4, 2012

Registering Application Descriptors using Airavata Client API

Following post demonstrates how to programmetically register; 1. Host 2. Application 3. Service descriptors using Apache Airavata Client API

import org.apache.airavata.common.registry.api.exception.RegistryException;
import org.apache.airavata.commons.gfac.type.ApplicationDeploymentDescription;
import org.apache.airavata.commons.gfac.type.HostDescription;
import org.apache.airavata.commons.gfac.type.ServiceDescription;
import org.apache.airavata.migrator.registry.MigrationUtil;
import org.apache.airavata.registry.api.AiravataRegistry;
import org.apache.airavata.schemas.gfac.*;

import java.net.MalformedURLException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class DescriptorRegistrationSample {

    public static void main(String[] args) {
        Map<String, String> config = new HashMap<String, String>();
        config.put(org.apache.airavata.client.airavata.AiravataClient.MSGBOX,"http://localhost:8090/axis2/services/MsgBoxService");
        config.put(org.apache.airavata.client.airavata.AiravataClient.BROKER, "http://localhost:8090/axis2/services/EventingService");
        config.put(org.apache.airavata.client.airavata.AiravataClient.WORKFLOWSERVICEURL, "http://localhost:8090/axis2/services/WorkflowInterpretor?wsdl");
        config.put(org.apache.airavata.client.airavata.AiravataClient.JCR, "http://localhost:8090/jackrabbit-webapp-2.4.0/rmi");
        config.put(org.apache.airavata.client.airavata.AiravataClient.JCR_USERNAME, "admin");
        config.put(org.apache.airavata.client.airavata.AiravataClient.JCR_PASSWORD, "admin");
        config.put(org.apache.airavata.client.airavata.AiravataClient.GFAC, "http://localhost:8090/axis2/services/GFacService");
        config.put(org.apache.airavata.client.airavata.AiravataClient.WITHLISTENER, "false");
        config.put(org.apache.airavata.client.airavata.AiravataClient.TRUSTED_CERT_LOCATION, "/Users/Downloads/certificates");

        org.apache.airavata.client.airavata.AiravataClient airavataClient = null;
        try {
            airavataClient = new org.apache.airavata.client.airavata.AiravataClient(config);
        } catch (MalformedURLException e) {
            e.printStackTrace();
        }

        // Create Host Description
        HostDescription host = new HostDescription();
        host.getType().changeType(GlobusHostType.type);
        host.getType().setHostName("gram");
        host.getType().setHostAddress("gatekeeper2.ranger.tacc.teragrid.org");
        ((GlobusHostType) host.getType()).
                setGridFTPEndPointArray(new String[]{"gsiftp://gridftp.ranger.tacc.teragrid.org:2811/"});
        ((GlobusHostType) host.getType()).
                setGlobusGateKeeperEndPointArray(new String[]{"gatekeeper.ranger.tacc.teragrid.org:2119/jobmanager-sge"});


        // Create Application Description 
        ApplicationDeploymentDescription appDesc = new ApplicationDeploymentDescription(GramApplicationDeploymentType.type);
        GramApplicationDeploymentType app = (GramApplicationDeploymentType) appDesc.getType();
        app.setCpuCount(1);
        app.setNodeCount(1);
        ApplicationDeploymentDescriptionType.ApplicationName name = appDesc.getType().addNewApplicationName();
        name.setStringValue("EchoMPILocal");
        app.setExecutableLocation("/home/path_to_executable");
        app.setScratchWorkingDirectory("/home/path_to_temporary_directory");
        app.setCpuCount(16);
        app.setJobType(MigrationUtil.getJobTypeEnum("MPI"));
        //app.setMinMemory();
        ProjectAccountType projectAccountType = ((GramApplicationDeploymentType) appDesc.getType()).addNewProjectAccount();
        projectAccountType.setProjectAccountNumber("XXXXXXXX");

        // Create Service Description
        ServiceDescription serv = new ServiceDescription();
        serv.getType().setName("MockPwscfMPIService");

        InputParameterType input = InputParameterType.Factory.newInstance();
        input.setParameterName("echo_input_name");
        ParameterType parameterType = input.addNewParameterType();
        parameterType.setType(DataType.Enum.forString("String"));
        parameterType.setName("String");
        List<InputParameterType> inputList = new ArrayList<InputParameterType>();
        inputList.add(input);
        InputParameterType[] inputParamList = inputList.toArray(new InputParameterType[inputList
                .size()]);

        OutputParameterType output = OutputParameterType.Factory.newInstance();
        output.setParameterName("echo_mpi_output");
        ParameterType parameterType1 = output.addNewParameterType();
        parameterType1.setType(DataType.Enum.forString("String"));
        parameterType1.setName("String");
        List<OutputParameterType> outputList = new ArrayList<OutputParameterType>();
        outputList.add(output);
        OutputParameterType[] outputParamList = outputList
                .toArray(new OutputParameterType[outputList.size()]);
        serv.getType().setInputParametersArray(inputParamList);
        serv.getType().setOutputParametersArray(outputParamList);

        // Save to Registry
        if (airavataClient!=null) {
            System.out.println("Saving to Registry");
            AiravataRegistry jcrRegistry = airavataClient.getRegistry();
            try {
                jcrRegistry.saveHostDescription(host);
                jcrRegistry.saveServiceDescription(serv);
                jcrRegistry.saveDeploymentDescription(serv.getType().getName(), host.getType().getHostName(), appDesc);

                jcrRegistry.deployServiceOnHost(serv.getType().getName(), host.getType().getHostName());
            } catch (RegistryException e) {
                e.printStackTrace();
            }
        }

        System.out.println("DONE");
        
    }

}

Tuesday, July 3, 2012

Airavata Programming API


Apache Airavata's Programming API is the API which is exposed to the Gateway Developers. Gateway Developers can use this API to execute and monitor workflows. The API user should keep in mind that a user can not compose an Airavata workflow (.xwf) using the API. Inorder to do that a user can use the XBaya User Interface. Therefore, other than creation of the workflow; Client API supports all other workflow related operations.


The main motivation behind, having a Client API is that to expose the user to an API that will let him/her to a the persistent information stored in the Registry. The information persisted in the Registry can be;
  • Descriptors
  • Workflow information
  • Workflow provenance information
  • Airavata configuration

Following are the high level usecases which uses Airavata Client API.



Client API Usecases



  1. Registry Operations 
    • Retrieve registry information
    • Access registry information
    • Update registry information
    • Delete registry information
    • Search registry information
  2. Execute workflow
    • Run workflow
    • Set inputs
    • Set workflow node IDs 
  3. Monitoring
  4. Provenance
  5. User Management (This is not yet implemented. It's currently in our Road Map and this is added as a place holder.)
    • User roles
    • Administration



Client API Components



The Client API consists of 5 main components.
  1. Airavata API
    • It is an Aggregator API which contains all the base methods for Airavata API.
  2. Airavata Manager
    • This exposes config related information on Airavata. This currently contains Service URLs only.
  3. Application Manager
    • This will handle operations related to descriptors. Namely;
      1. Host description
      2. Service description
      3. Application description
  4. Execution Manager
    • This can be used to run and monitor workflows.
  5. Provenance Manger
    • This provides API to manage provenance related information. ie. Keeps track of inputs, outputs, etc related to a workflow.
  6. User Manger
    • User management related API is exposed through this. Currently, Airavata does not support User management but it is in Airavata roadmap.
  7. Workflow manager
    • Every operation related to workflows is exposed through this. ie:
      1. saving workflow
      2. deleting workflow
      3. retrieving workflow