Configure Apache Rave for SSL

I had to do $subject for the oa4mp integration work that I am currently doing with Rave. I had to do some configuration changes to get SSL working with Rave. Following are the instructions on how to $subject.

Enabling SSL in Tomcat

Following instructions demonstrate how to get Tomcat 6 running over SSL using a self signed certificate.

• Find the reverse DNS (of the IP address )of the machine in which you are going to install.

$ host your-ip-address

•  Then you'll be getting the reverse DNS of the IP address you gave.

 xxx-yy-zzz-hhh.dhcp-bl.xxx.edu

• Generate a self signed certificate that you'll use with Tomcat.

keytool -genkey -alias tomcat -keyalg RSA -validity 365 -storepass changeit -keystore $JAVA_HOME/jre/lib/security/cacerts

What is your first and last name?
  [Unknown]:  xxx-yy-zzz-hhh.dhcp-bl.xxx.edu
What is the name of your organizational unit?
  [Unknown]:  SGG
What is the name of your organization?
  [Unknown]:  IU
What is the name of your City or Locality?
  [Unknown]:  Bloomington
What is the name of your State or Province?
  [Unknown]:  IN
What is the two-letter country code for this unit?
  [Unknown]:  US
Is CN=xxx-yy-zzz-hhh.dhcp-bl.xxx.edu, OU=SGG, O=IU, L=Bloomington, ST=IN, C=US correct?
  [no]:  yes

Enter key password for
        (RETURN if same as keystore password):

• Edit Tomcats server.xml to enable an SSL listener on port 443 using our alternate cacerts file. By default Tomcat looks for a certificate with the alias "tomcat" which is what we used to create our self signed certificate. (uncommented the HTTPS connector and configured it to use our custom cacerts file)

<Connector port="443" protocol="HTTP/1.1" SSLEnabled="true"
           maxThreads="150" scheme="https" secure="true"
           keystoreFile="$JAVA_HOME/jre/lib/security/cacerts" keystorePass="changeit"
           clientAuth="false" sslProtocol="TLS" />

Configure Apache Rave and Shindig to run over SSL.

1. Configure properties files.

• Edit the portal.properties file to configure Apache Rave to use SSL. (updated the following values at the top of the portal.properties config file with)

portal.opensocial_engine.protocol=https
portal.opensocial_engine.root=xxx-yy-zzz-hhh.dhcp-bl.xxx.edu
portal.opensocial_engine.gadget_path=/gadgets
Edit the rave.shindig.properties and  container.js files to configure Shindig to use SSL.

• The changes to container.js are - search and replace of http:// with https://

• Updated the following values at the top of the rave.shindig.properties config file with.

shindig.host= xxx-yy-zzz-hhh.dhcp-bl.xxx.edu
shindig.port=
shindig.contextroot=

2. Update the rave-portal pom.

• Add the following configuration to the cargo plugin. It uses the tomcat server.xml file (configured in the first section) give in the configuration to startup a Tomcat instance.

<configfiles>
    <configfile>
        <file>${project.basedir}/../rave-portal-resources/src/main/dist/conf/tomcat-users.xml</file>
        <todir>conf/</todir>
        <tofile>tomcat-users.xml</tofile>
    </configfile>
    <configfile>
        <file>/home/heshan/Dev/airavata-rave-integration/oauth/rave-0.15-oa4mp-branch/config/server.xml</file>
        <todir>conf/</todir>
        <tofile>server.xml</tofile>
    </configfile>
</configfiles>

• Build raven project.

mvn clean install

• Move to the rave-portal module and start Rave using the Cargo plugin.

cd rave-portal
mvn cargo:start

• Log into the portal using the login page. 

https://156-56-179-232.dhcp-bl.indiana.edu/portal/login

Apache Airavata 0.4-INCUBATING Released

The Apache Airavata (Incubating) team is pleased to announce the immediate availability of the Airavata 0.4-INCUBATING release.

The release can be obtained from the Apache Airavata download page - http://incubator.apache.org/airavata/about/downloads.html

Release notes are available at - https://svn.apache.org/repos/asf/incubator/airavata/tags/airavata-0.4-incubating/RELEASE_NOTES

Apache Airavata is a software toolkit currently used to build science gateways but that has a much wider potential use. It provides features to compose, manage, execute, and monitor small to large scale applications and workflows on computational resources ranging from local clusters to national grids and computing clouds. Gadgets interfaces to Airavata back end services can be deployed in open social containers such as Apache Rave and modify them to suit their needs. Airavata builds on general concepts of service oriented computing, distributed messaging, and workflow composition and orchestration.

For general information on Apache Airavata, please visit the project website:

http://incubator.apache.org/airavata/

Disclaimer:

 Apache Airavata is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects.  While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.

How to Submit Patches to Apache Airavata

This post describes how an Airavata user can contribute to the Airavata project by submitting patches. User can follow the steps given below.

• Identify an issue that you want to fix or improve
• Search JIRA and the mailing list to see if it’s already been discussed
• If it’s a bug or a feature request, open a JIRA issue
• Create a sample that you can use for prototyping the feature or demonstrating the bug. If creating a sample is time consuming, write steps to reproduce the issue.
• Attach this sample to the JIRA issue if it’s representing a bug report.
• Setup a svn client in your system.
• Checkout the source code.
• If it is the source trunk;
• svn co https://svn.apache.org/repos/asf/incubator/airavata/trunk
• If it is the site documentation trunk;
• svn co https://svn.apache.org/repos/asf/incubator/airavata/site/trunk
• Make your changes
• Create the patch:
  • svn add any_files_you_added
  • svn diff > /tmp/fix-AIRAVATA-NNNN.patch
• Attach that file (/tmp/fix-AIRAVATA-NNNN.patch) to the JIRA

Deploying Airavata Server on Tomcat

A shell script named setup_tomcat.sh is shipped with Airavata that will assist you to $subject. Following steps describe how you can do it.

1) Update tomcat.hostname and tomcat.port properties of the airavata-tomcat.properties file. You can keep the defaults if you dont want to change ports. In that case you don't have to edit the airavata-tomcat.properties file. This file can be found in AIRAVATA_HOME/tools/airavata-tomcat.properties.

2) Download following to your local file system.
a) apache-tomcat-7.0.28.zip
b) apache-airavata-0.4-incubating-SNAPSHOT-bin.zip
b) axis2-1.5.1-war.zip (Unzip it. When running the script point to the axis2.war)
d) ackrabbit-webapp-2.4.0.war

3) Run the script (setup_tomcat.sh) by providing the full file paths of the files you downloaded. This script can be found in AIRAVATA_HOME/tools/ directory.

./setup_tomcat.sh --tomcat=/home/heshan/Dev/setup/gw8/apache-tomcat-7.0.28.zip --airavata=/home/heshan/Dev/setup/gw8/apache-airavata-0.4-incubating-SNAPSHOT-bin.zip --axis2=/home/heshan/Dev/setup/gw8/axis2.war --jackrabbit=/home/heshan/Dev/setup/gw8/jackrabbit-webapp-2.4.0.war --properties=/home/heshan/Dev/setup/gw8/airavata-tomcat.properties

4) Start Tomcat server.
eg: ./catalina.sh start

5) Before using airavata go to http://localhost:8090/jackrabbit-webapp-2.4.0 and create a default content repository.

6) Restart Tomcat server.

Registering Application Descriptors using Airavata Client API

Following post demonstrates how to programmetically register; 1. Host 2. Application 3. Service descriptors using Apache Airavata Client API

import org.apache.airavata.common.registry.api.exception.RegistryException;
import org.apache.airavata.commons.gfac.type.ApplicationDeploymentDescription;
import org.apache.airavata.commons.gfac.type.HostDescription;
import org.apache.airavata.commons.gfac.type.ServiceDescription;
import org.apache.airavata.migrator.registry.MigrationUtil;
import org.apache.airavata.registry.api.AiravataRegistry;
import org.apache.airavata.schemas.gfac.*;

import java.net.MalformedURLException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class DescriptorRegistrationSample {

    public static void main(String[] args) {
        Map<String, String> config = new HashMap<String, String>();
        config.put(org.apache.airavata.client.airavata.AiravataClient.MSGBOX,"http://localhost:8090/axis2/services/MsgBoxService");
        config.put(org.apache.airavata.client.airavata.AiravataClient.BROKER, "http://localhost:8090/axis2/services/EventingService");
        config.put(org.apache.airavata.client.airavata.AiravataClient.WORKFLOWSERVICEURL, "http://localhost:8090/axis2/services/WorkflowInterpretor?wsdl");
        config.put(org.apache.airavata.client.airavata.AiravataClient.JCR, "http://localhost:8090/jackrabbit-webapp-2.4.0/rmi");
        config.put(org.apache.airavata.client.airavata.AiravataClient.JCR_USERNAME, "admin");
        config.put(org.apache.airavata.client.airavata.AiravataClient.JCR_PASSWORD, "admin");
        config.put(org.apache.airavata.client.airavata.AiravataClient.GFAC, "http://localhost:8090/axis2/services/GFacService");
        config.put(org.apache.airavata.client.airavata.AiravataClient.WITHLISTENER, "false");
        config.put(org.apache.airavata.client.airavata.AiravataClient.TRUSTED_CERT_LOCATION, "/Users/Downloads/certificates");

        org.apache.airavata.client.airavata.AiravataClient airavataClient = null;
        try {
            airavataClient = new org.apache.airavata.client.airavata.AiravataClient(config);
        } catch (MalformedURLException e) {
            e.printStackTrace();
        }

        // Create Host Description
        HostDescription host = new HostDescription();
        host.getType().changeType(GlobusHostType.type);
        host.getType().setHostName("gram");
        host.getType().setHostAddress("gatekeeper2.ranger.tacc.teragrid.org");
        ((GlobusHostType) host.getType()).
                setGridFTPEndPointArray(new String[]{"gsiftp://gridftp.ranger.tacc.teragrid.org:2811/"});
        ((GlobusHostType) host.getType()).
                setGlobusGateKeeperEndPointArray(new String[]{"gatekeeper.ranger.tacc.teragrid.org:2119/jobmanager-sge"});


        // Create Application Description 
        ApplicationDeploymentDescription appDesc = new ApplicationDeploymentDescription(GramApplicationDeploymentType.type);
        GramApplicationDeploymentType app = (GramApplicationDeploymentType) appDesc.getType();
        app.setCpuCount(1);
        app.setNodeCount(1);
        ApplicationDeploymentDescriptionType.ApplicationName name = appDesc.getType().addNewApplicationName();
        name.setStringValue("EchoMPILocal");
        app.setExecutableLocation("/home/path_to_executable");
        app.setScratchWorkingDirectory("/home/path_to_temporary_directory");
        app.setCpuCount(16);
        app.setJobType(MigrationUtil.getJobTypeEnum("MPI"));
        //app.setMinMemory();
        ProjectAccountType projectAccountType = ((GramApplicationDeploymentType) appDesc.getType()).addNewProjectAccount();
        projectAccountType.setProjectAccountNumber("XXXXXXXX");

        // Create Service Description
        ServiceDescription serv = new ServiceDescription();
        serv.getType().setName("MockPwscfMPIService");

        InputParameterType input = InputParameterType.Factory.newInstance();
        input.setParameterName("echo_input_name");
        ParameterType parameterType = input.addNewParameterType();
        parameterType.setType(DataType.Enum.forString("String"));
        parameterType.setName("String");
        List<InputParameterType> inputList = new ArrayList<InputParameterType>();
        inputList.add(input);
        InputParameterType[] inputParamList = inputList.toArray(new InputParameterType[inputList
                .size()]);

        OutputParameterType output = OutputParameterType.Factory.newInstance();
        output.setParameterName("echo_mpi_output");
        ParameterType parameterType1 = output.addNewParameterType();
        parameterType1.setType(DataType.Enum.forString("String"));
        parameterType1.setName("String");
        List<OutputParameterType> outputList = new ArrayList<OutputParameterType>();
        outputList.add(output);
        OutputParameterType[] outputParamList = outputList
                .toArray(new OutputParameterType[outputList.size()]);
        serv.getType().setInputParametersArray(inputParamList);
        serv.getType().setOutputParametersArray(outputParamList);

        // Save to Registry
        if (airavataClient!=null) {
            System.out.println("Saving to Registry");
            AiravataRegistry jcrRegistry = airavataClient.getRegistry();
            try {
                jcrRegistry.saveHostDescription(host);
                jcrRegistry.saveServiceDescription(serv);
                jcrRegistry.saveDeploymentDescription(serv.getType().getName(), host.getType().getHostName(), appDesc);

                jcrRegistry.deployServiceOnHost(serv.getType().getName(), host.getType().getHostName());
            } catch (RegistryException e) {
                e.printStackTrace();
            }
        }

        System.out.println("DONE");
        
    }

}

Airavata Programming API

Apache Airavata's Programming API is the API which is exposed to the Gateway Developers. Gateway Developers can use this API to execute and monitor workflows. The API user should keep in mind that a user can not compose an Airavata workflow (.xwf) using the API. Inorder to do that a user can use the XBaya User Interface. Therefore, other than creation of the workflow; Client API supports all other workflow related operations.

The main motivation behind, having a Client API is that to expose the user to an API that will let him/her to a the persistent information stored in the Registry. The information persisted in the Registry can be;

• Descriptors
• Workflow information
• Workflow provenance information
• Airavata configuration

Following are the high level usecases which uses Airavata Client API.

Client API Usecases

1. Registry Operations 
   • Retrieve registry information
   • Access registry information
   • Update registry information
   • Delete registry information
   • Search registry information
2. Execute workflow
   • Run workflow
   • Set inputs
   • Set workflow node IDs 
3. Monitoring
4. Provenance
5. User Management (This is not yet implemented. It's currently in our Road Map and this is added as a place holder.)
   • User roles
   • Administration

Client API Components

The Client API consists of 5 main components.

1. Airavata API
   • It is an Aggregator API which contains all the base methods for Airavata API.
2. Airavata Manager
   • This exposes config related information on Airavata. This currently contains Service URLs only.
3. Application Manager
   • This will handle operations related to descriptors. Namely;
     1. Host description
     2. Service description
     3. Application description
4. Execution Manager
   • This can be used to run and monitor workflows.
5. Provenance Manger
   • This provides API to manage provenance related information. ie. Keeps track of inputs, outputs, etc related to a workflow.
6. User Manger
   • User management related API is exposed through this. Currently, Airavata does not support User management but it is in Airavata roadmap.
7. Workflow manager
   • Every operation related to workflows is exposed through this. ie:
     1. saving workflow
     2. deleting workflow
     3. retrieving workflow

Apache Airavata Stakeholders

Airavata is a framework which enables a user to build Science Gateways. It is used to compose, manage, execute and monitor distributed applications and workflows on computational resources. These computational resources can range from local resources to computational grids and clouds. Therefore, various users with different backgrounds either contribute or use Airavata in their applications. From the Airavata standpoint, three main users can be identified.

• Research Scientists (Gateway End Users)
• Gateway Developers
• Core Developers

Now let's focus on each user and how they fit into Airavata's big picture.

 

Gateway End Users

Gateway End User is the one who will have a model code to do some scientific application. Sometimes this End User can be a Research Scientist. He/She writes scripts to wrap the applications up and by executing those scripts, they run the scientific workflows in Super Computers. This can be called a scientific experiment. Now the Scientist might have a requirement to call multiple of these applications together and compose a workflow. That's where the Gateway Developer comes into the picture.

 

Gateway Developers

The Research Scientist is the one who comes up with requirement of bundling scienitific applications together and composing as a workflow.
The job of the Gateway Developer is to use Airavata and wrap the above mentioned model code and scripts together. Then, scientific workflows are created out these.
Above diagram depicts how Gateway Developer fits into the picture.

 

Core Developers

Core Deveoper is the one who develops and contributes to Airavata framework codebase. The Gateway Developers use the software developed by the Core Developers to create science gateways.