Friday, March 15, 2013

Airavata Deployment Studio (ADS)

This is an independent study that I have been doing for Apache Airavata [1]. Airavata Deployment Studio or simply ADS, is a platform where an Airavata user can deploy his/her Airavata deployment on a Cloud computing resource on demand. Now let's dive into ADS and what's the actual problem that we are trying the solve here. 

What is Airavata? 

Airavata is a framework which enables a user to build Science Gateways. It is used to compose, manage, execute and monitor distributed applications and workflows on computational resources. These computational resources can range from local resources to computational grids and clouds. Therefore, various users with different backgrounds either contribute or use Airavata in their applications.

Who uses Airavata? 

From the Airavata standpoint, three main users can be identified.

1) End Users

End User is the one who will have a model code to do some scientific application. Sometimes this End User can be a Research Scientist. He/She writes scripts to wrap the applications up and by executing those scripts, they run the scientific workflows in Super Computers. This can be called a scientific experiment.

2) Gateway Developers

The Research Scientist is the one who comes up with requirement of bundling scientific applications together and composing as a workflow. The job of the Gateway Developer is to use Airavata and wrap the above mentioned model code and scripts together. Then, scientific workflows are created out these. In some cases, Scientist might be the Gateway Developer as well.

3) Core Developers

Core Developer is the one who develops and contributes to Airavata framework code-base. The Gateway Developers use the software developed by the Core Developers to create science gateways.

Why ADS?

According to the above description, Airavata is used by different people with different technical backgrounds. Some people will have in depth technical knowledge on their scientific domains; like chemistry, biology, astronomy, etc and may not have in depth knowledge on computer science aspects such as cluster configuration, configuring and trouble-shooting in VMs, etc. 

When it comes to ADS, it's targeted towards the first two types of users as they will be ones who will be running in to configuration issues with Airavata in their respective systems. 

Sometimes we come across instances where a user might run into issues while setting up Airavata on their Systems. These might be attributed to; 
  1. User not following the documented steps properly.
  2. Issues in setting up the user environment. 
  3. User not being able to diagnose the issues at their end on their own.
  4. Sometimes when we try to diagnose their issue remotely, we face difficulties trying to access user's VM remotely due to security policies defined in their System. 
  5. Different security policies at client's firewall.

Due to the above mentioned issues, a first time user might go away with a bad impression due to a System/VM level issue that might not be directly related to Airavata. 

What we are trying to do here is to give a first time user a good first impression as well as ease of configuring the Airavata eco system for production usage. 


Now you might be wondering how does ADS achieve this? ADS will use FutureGrid [3] as the underlying resource platform for this application. If you are interested in learning about what FutureGrid is, please refer [3] for more information. ADS will ultimately become a plugin to the FutureGrid's CloudMesh [4] environment.

ADS will provide a user with a web interface which a user can use to configure his/her Airavata eco system. Once the configuration options are selected and user hits the submit button, a new VM with the selected configurations will be created. The user will be able to create his/her image with the following properties. 
  • Infrastructure - eg: OpenStack, Eucalyptus, EC2, etc
  • Architecture - eg: 64-bit, 32-bit 
  • Memory - eg: 2GB, 4GB, 8GB, etc
  • Operating System - eg: Ubuntu, CentOS, Fedora, etc
  • Java version - eg: Java 1.6, Java 1.7
  • Tomcat Version - eg: Tomcat6, Tomcat7
  • Airavata Version - eg: Airavata-0.6, Airavata-0.7


  1. One click install. 
  2. No need to interact with the shell to configure an Airavata environment.
  3. Deploying on various Cloud platforms based on user preference.
  4. Ease of use. 
  5. First time user will be able to quickly configure an insatnce of his own and run a sample workflow quickly. 
  6. On demand aspect.

Sneak Peak

Following screenshots show how ADS will look like.


Installing Moab Web Services on a Unix box

1) Install Tomcat
yum install tomcat6
2) Install 64-bit version of Oracle Java SE6 JRE.
sh jre-6u37-linux-x64-rpm.bin
rm -f /usr/bin/java
ln -s /etc/alternatives/java /usr/bin/java
alternatives --install /usr/bin/java java /usr/java/jre1.6.0_37/bin/java 500
alternatives --set java /usr/java/jre1.6.0_37/bin/java
3) Create mws home directories and sub-directories
mkdir -p /opt/mws/etc /opt/mws/hooks /opt/mws/plugins /opt/mws/log
chown -R tomcat:tomcat /opt/mws # Depending on your OS, the Tomcat username might be
chmod -R 555 /opt/mws
chmod u+w /opt/mws/plugins /opt/mws/log
4) Extract mws tarball to a tempory directory.
mkdir /tmp/mws-install
cd /tmp/mws-install
tar xvzf $HOME/Downloads/mws-.tar.gz
cd /tmp/mws-install/mws-
5) Set up the MWS configuration file.
 i) In the extracted MWS directory are two sample configuration files:
   mws-config-cloud.groovy and mws-config-hpc.groovy
   mws-config-cloud.groovy provides sample configuration for the Moab Cloud Suite
   mws-config-hpc.groovy provides sample configuration for the Moab HPC Suites

ii) Choose the correct file for your suite, rename it to mws-config.groovy, and copy it to /opt/mws/etc.

iii) Give the Tomcat user read access to /opt/mws/etc/mws-config.groovy. 6) Add the following line to the end of /etc/tomcat6/tomcat6.conf.
CATALINA_OPTS="-DMWS_HOME=/opt/mws -Xms256m -Xmx3g -XX:MaxPermSize=384m"
7) Start Tomcat and deploy mws.war.
chkconfig tomcat6 on
service tomcat6 stop
cp /tmp/mws-install/mws-/mws.war /var/lib/tomcat6/webapps
service tomcat6 start
8) Visit http://localhost:8080/mws/ in a web browser to verify that MWS is running. You will see some sample queries and a few other actions.

9) Log into MWS to verify that the MWS credentials are working. The credentials are the values of auth.defaultUser.username and auth.defaultUser.password that you set above.