Monitor CPU Utilization of a Amazon EC2 instance using Amazon CloudWatch

Following code snippet show how to monitor CPU Utilization of a Amazon EC2 instance using Amazon Cloud Watch. In order to monitor the instance, Cloud Watch Monitoring should be enabled for the running instance.

private double monitorInstance(AWSCredentials credential, String instanceId) {
   try {
       AmazonCloudWatchClient cw = new AmazonCloudWatchClient(credential) ;

       long offsetInMilliseconds = 1000 * 60 * 60 * 24;
       GetMetricStatisticsRequest request = new GetMetricStatisticsRequest()
               .withStartTime(new Date(new Date().getTime() - offsetInMilliseconds))
               .withNamespace("AWS/EC2")
               .withPeriod(60 * 60)
               .withDimensions(new Dimension().withName("InstanceId").withValue(instanceId))
               .withMetricName("CPUUtilization")
               .withStatistics("Average", "Maximum")
               .withEndTime(new Date());
       GetMetricStatisticsResult getMetricStatisticsResult = cw.getMetricStatistics(request);

       double avgCPUUtilization = 0;
       List dataPoint = getMetricStatisticsResult.getDatapoints();
       for (Object aDataPoint : dataPoint) {
           Datapoint dp = (Datapoint) aDataPoint;
           avgCPUUtilization = dp.getAverage();
           log.info(instanceId + " instance's average CPU utilization : " + dp.getAverage());
       }

       return avgCPUUtilization;

   } catch (AmazonServiceException ase) {
       log.severe("Caught an AmazonServiceException, which means the request was made  "
               + "to Amazon EC2, but was rejected with an error response for some reason.");
       log.severe("Error Message:    " + ase.getMessage());
       log.severe("HTTP Status Code: " + ase.getStatusCode());
       log.severe("AWS Error Code:   " + ase.getErrorCode());
       log.severe("Error Type:       " + ase.getErrorType());
       log.severe("Request ID:       " + ase.getRequestId());

   }
   return 0;
}

OGCE XRegistry to Airavata Registry migration guide

This post describes how to migrate OGCE XRegistry to Apache Airavata Registry.

Apache Airavata contains a migration tool that facilitates the migration of XRegistry resouces to Airavata Registry. Following are the steps to be followed for a migration of resources. (If you find any information missing that should be included in this post, please comment on [1])

1) Build the Airavata trunk [2].

2) Unzip the apache-airavata-0.3-incubating-SNAPSHOT.zip.

3) Start Airavata Jackrabbit Server by running the shell script in AIRAVATA_HOME/bin directory.
eg: jackrabbit-server.sh

4) Configure the properties file xregistry.properties file. This properties file consists information of XRegistry and Airavata Registry. (This properties file should be placed in the your classpath)
eg.

registryURL=https://YOUR_HOST:19443/xregistry?wsdl
ssl.trustedCertsFile=/home/heshan/Dev/setup/certificates
ssl.hostcertsKeyFile=/home/heshan/Dev/setup/xregistry/target/dist-bin/conf/ogce_services_key.pem
myproxyServer=myproxy.teragrid.org
myproxyUser=*****
myproxyPass=*****
myproxyLifetime=14400
jcr.url=http://localhost:8081/rmi
jcr.username=****
jcr.password=****

Following is a description of each of the above mentioned properties;

registryURL - URL of the XRegistry instance.
ssl.trustedCertsFile=Directory path Cert file used to connect to XRegistry
ssl.hostcertsKeyFile=Host Certificates key file
myproxyServer - URL for MyProxy credential management service
myproxyUser - User name used for MyProxy credential management service
myproxyPass - Password associated with the myproxy username
myproxyLifetime - Lifetime for myproxy certificate.
jcr.url - URL for JCR Registry
jcr.username - User name used to access the JCR Registry
jcr.password - Password associated with the JCR Registry User name

5) Go to the AIRAVATA_HOME/bin directory. Execute the xregistry-migrator shell script with your properly configured xregistry.properties file.
eg:
./xregistry-migrator.sh /home/heshan/Dev/xregistry.properties

6) Start the Airavata server using the shell script in AIRAVATA_HOME/bin directory.
eg:
./airavata-server.sh

Browsing the Airavata Registry
7) Viewing the migrated descriptors.
i) Start the XBaya GUI by using the shell script in AIRAVATA_HOME/bin directory.
eg:
./xbaya-gui.sh

ii) Click on the Registry Menu and select “Setup Registry” option. Click ok.

iii) Then using the Registry browser you can view the host, service and application descriptors saved in the registry.

8) Now using the saved application services, you can compose scientific workflows.

NOTE:
1) If there are multiple services under the same name (under different namespaces), those will be renamed (appending _NUMBER to the service name) during migration.
2) If there are multiple applications under the same name (under different namespaces), those will be renamed (appending _NUMBER to the application name) during migration.

References

[1] - https://issues.apache.org/jira/browse/AIRAVATA-321

[2] - https://svn.apache.org/repos/asf/incubator/airavata/trunk/

How to Deploy Apache Airavata on Tomcat

Applies to

• Airavata Trunk - apache-airavata-0.3-incubating-SNAPSHOT.zip
• Axis version - Axis2 1.5.1.war
• Jackrabbit version - jackrabbit-webapp-2.4.0.war
• Tomcat version - apache-tomcat-7.0.23

Airavata services depends on Apache Axis2 as a Web Service engine. In order to deploy Airavata system on Tomcat, you need to deploy Apache Axis2 on Tomcat first.

1) Copy all the service jar files in AIRAVATA_HOME/standalone-server/repository/services/ directory to Tomcat axis2 services directory(TOMCAT_HOME/webapps/axis2/WEB-INF/services)

2) Copy all the dependent jars (copy all the jars in AIRAVATA_HOME/lib and AIRAVATA_HOME/standalone/lib ) to TOMCAT_HOME/webapps/axis2/WEB-INF/lib.

3) Copy the properties files for messageBox (msgBroker.properties) and messageBroker (msgBroker.properties) to TOMCAT_HOME/webapps/axis2/WEB-INF/classes/conf/. These properties files can be found in AIRAVATA_HOME/standalone-server/conf directory.

4) Copy the properties files for GFac Service (repository.properties) and Interpretor Service (xbaya.properties) to TOMCAT_HOME/webapps/axis2/WEB-INF/lib/ directory. These properties files can be found in AIRAVATA_HOME/standalone-server/conf directory.

i) Based on your setup configure the GSI security credentials and certificate paths in the repository.properties file.

ii) Fill up the following property names values in your xbaya.properties file based on your gsi security credentials.
eg : myproxy.user
myproxy.password

NOTE: keep the same repository.properties file in TOMCAT_HOME/bin directory too. This issue will be fixed in later release.

4) Download and copy a database library (Apache Derby or MySql) to webapps/axis2/WEB-INF/lib directory.
eg: derby-10.7.1.1.jar
derbytools-10.7.1.1.jar

5) Copy directory AIRAVATA_HOME/standalone-server/bin/database_scripts to TOMCAT_HOME/webapps/axis2/WEB-INF/classes/database_scripts directory.

6) Now we have configured a Tomcat instance to run Airavata Services. Next is to deploy Jacrabbit instance on a separate Tomcat instance(I'm referring it's home directory as TOMCAT_HOME2).

7) Deploy Jackrabbit on Tomcat

i) Download jcr-2.0.jar and put it into TOMCAT_HOME2/webapps/jackrabbit-webapp-2.4.0/WEB-INF/lib/.

ii) Get the WAR distribution from the Downloads page and deploy it into Tomcat.

iii) Make sure to change and configure the ports by editing TOMCAT_HOME2/conf/server.xml file.

8) Now startup the Jacrabbit deployed Tomcat instance.

9) Startup the Tomcat instance with Airavata services.

10) Go to service listing page and check the service status. If there are no faulty services, then you have done the setup properly.