Heshan's Blog: Synapse

Showing posts with label Synapse. Show all posts

Sunday, January 30, 2011

Became a Committer for Apache Synapse

I earned committership for Apache Synapse ESB project :)

Tuesday, January 11, 2011

Apache Synapse Enterprise Service Bus (ESB) - 2.0.0 Released

Overview

The Apache Synapse ESB is a robust, lightweight and highly scalable and distributed open source Enterprise Service Bus (ESB). It supports SOAP Web services as well as Legacy services over transports such as JMS, Apache VFS File systems, Mail etc, and SOAP, REST/POX, plain text and binary message payloads. (Please see http://synapse.apache.org for more details)

Installation Prerequisites

Apache Synapse requires a J2SE runtime of version 1.5.x or later. Running the samples also requires Apache Ant 1.7.x or later. Although Synapse would run with a JDK > 1.5.x, the Script mediator may not properly function on these JDKs. Building Synapse from source requires JDK 1.5.x or later, and Apache Maven 2.1.0 or later

Quick start
Please see the http://synapse.apache.org/Synapse_Quickstart.html guide

Building the Samples
Please see the documents http://synapse.apache.org/Synapse_Samples.html and http://synapse.apache.org/Synapse_Samples_Setup.html

Synapse configuration language
Please see the document http://synapse.apache.org/Synapse_Configuration_Language.html

Extending Synapse
Please see the document http://synapse.apache.orgs/Synapse_Extending.html

Known Issues and limitations

SYNAPSE-180 Does not support throttling by concurrency within a cluster
SYNAPSE-186 Does not support HTTP some of the REST operations (such as put/delete etc)
SYNAPSE-181 Does not yet support JTA transactions
SYNAPSE-330 Does not yet support load balancing with session affinity using SOAP sessions
SYNAPSE-280 Does not preserve CDATA sections within payloads
SYNAPSE-307 The XSLT mediator will not report errors encountered in a provided stylesheet
The Synapse JMS implementation supports JMS 1.0.2b, however due to licensing issues we includethe JMS 1.1 spec JAR from Apache Geronimo (geronimo-jms_1.1_spec-1.1.jar) instead. If you have any issues with JMS 1.0.x, please download the Sun JMS 1.0.2b JAR and replace supplied JAR from Geronimo.

Frequently asked questions

How can I change the default logging level of Synapse? Edit the lib/log4j.properties and set the line "log4j.category.org.apache.synapse=INFO" to "log4j.category.org.apache.synapse=DEBUG" and restart Synapse.
If you get an error related to WS-Security or when using it, check to ensure that your JDK uses the "Unlimited Strength Jurisdiction Policy Files". These could be downloaded from http://java.sun.com/javase/downloads/index_jdk5.jsp Refer to the associated documentation for instructions on how to install it to your JDK.
If you encounter issues with your JDK related to XML processing, try placing the Xerces jar files xercesImpl-2.8.0.jar and xml-apis-1.3.03.jar in your /jre/lib/endorsed/ directory.

Reporting Problems

Please use the Apache JIRA system (http://issues.apache.org/jira/browse/SYNAPSE) to report issues

You may also join the: synapse-dev mailing list by sending email to dev-subscribe@synapse.apache.org

synapse-user mailing list by sending email to user-subscribe@synapse.apache.org

New features in the Synapse 2.0.0 release

New, fine-grained configuration model
Hot deployment and hot update support for configuration artifacts
Priority based mediation support
Comprehensive eventing capabilities with WS-Eventing support
Secure vault for encrypting passwords in configuration files
File locking support in the VFS transport for concurrent polling
URLRewrite medaitor for fast and simple URL rewriting
Synapse configuration observer API
Multiple identity support in the HTTPS transport
Enhanced JMX monitoring support for the NHTTP transport
Dead letter channel implementation (experimental)
Synapse XAR Maven plugin for generating configuration artifacts

Wednesday, January 5, 2011

SFTP file transer with WSO2 ESB.

The VFS transport implementation is based on Apache Commons VFS implementation. VFS (Virtual File System) transport implementation is a module which belongs to the Apache Synapse project. It has a set of service level parameters that needs to be specified for each service.

VFS service level parameters and their descriptions can be found in [1] and the endpoint formats can be found in [2].

Following is a sftp sample for ESB 3.0.1. It copies a file from one sftp location to another sftp folder. SFTPVFSProxy is the proxy service which copies file from one sftp location to another.

If you need further information or need help troubleshooting this sample refer [5].


<proxy name="SFTPVFSProxy" transports="vfs" startOnLoad="true" xmlns="http://ws.apache.org/ns/synapse">
     <target>
         <inSequence>
             <log level="full"/>
             <property name="File" expression="fn:concat('test-', get-property('transport', 'FILE_PATH'))" scope="default"/>
             <property name="transport.vfs.ReplyFileName" expression="fn:concat(fn:substring-after(get-property('MessageID'), 'urn:uuid:'), '.xml')" scope="transport"/>
             <property name="OUT_ONLY" value="true"/>
             <send>
                 <endpoint name="endpoint_urn_uuid_A1546EFFD75FC9CCED785986339425964585275">
                     <address uri="vfs:sftp://heshan:password@10.101.1.112/home/heshan/out"/>
                 </endpoint>
             </send>
         </inSequence>
     </target>
     <parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
     <parameter name="transport.PollInterval">15</parameter>
     <parameter name="transport.vfs.MoveAfterProcess">vfs:sftp://heshan:password@10.101.1.112/home/heshan/original</parameter>
     <parameter name="transport.vfs.FileURI">vfs:sftp://heshan:password@10.101.1.112/home/heshan/in/test.xml</parameter>
     <parameter name="transport.vfs.MoveAfterFailure">vfs:sftp://heshan:password@10.101.1.112/home/heshan/original</parameter>
     <parameter name="transport.vfs.FileNamePattern">.*.xml</parameter>
     <parameter name="transport.vfs.ContentType">application/xml</parameter>
     <parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
</proxy>

Following is the XML file (test.xml) that is being moved.


<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wsa="http://www.w3.org/2005/08/addressing">
  <soapenv:Body>
      <getQuote xmlns="http://services.samples">
          <request>
              <symbol>IBM</symbol>
          </request>
      </getQuote>
  </soapenv:Body>
</soapenv:Envelope>

If you are interested in trying out more VFS samples, refer [3] and [4].

[1] - http://wso2.org/project/esb/java/3.0.1/docs/transports/transports-catalog.html#VfsTrp
[2] - http://commons.apache.org/vfs/filesystems.html#SFTP
[3] - http://wso2.org/project/esb/java/3.0.1/docs/samples/transport_samples.html#Sample254
[4] - http://wso2.org/project/esb/java/3.0.1/docs/samples/transport_samples.html#Sample255
[5] - http://wso2.org/library/tutorials/2011/01/sftp-file-transer-wso2-esb

Friday, September 10, 2010

Migrate Synapse Configuration from 1.2 to 2.0

The current namespace of the synapse-2.0 trunk is http://synapse.apache.org/ns/2010/04/configuration. The earlier namespace of synapse-1.2 was http://ws.apache.org/ns/synapse. Therefore you may encounter some errors, when running a synapse configuration of a 1.2 release against a trunk build(ie. synapse-2.0.0-SNAPSHOT). Since the Synapse configuration has undergone some configuration changes, merely changing the namespace of the the old synapse configuration wont help you to solve your problem.

That's why Synapse ships a migration tool for migrating a 1.2 configuration to a 2.0 configuration. Ruwan has written the xslt transformation for the above. You can find it in the Synapse trunk.

Recently I had to update the Synapse sample configurations of WSO2 ESB's trunk. This tool made my life a lot easier during the migration process. I modified the sample migrator to suit my purpose. I am sharing the code here, if you need to migrate the synapse configurations in bulk.


import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;

/**
*
*/
public class ConfigurationMigrator {

  private static final String MIGRATOR_XSLT_PATH
          = "modules/migrator/src/main/resources/synapse-configuration-migrator.xslt";

  public static int confgNumber [] = new int []
              {0,1,2,3,4,5,6,7,8,9,10,11,
              50,51,52,53,54,55,56,57,58,
              100,101,102,
              150,151,152,153,154,155,
              200,201,202,
              250,251,252,253,254,255,256,257,258,259,
              260,261,262,264,265,
              300,
              350,351,352,353,354,
              360,361,362,363,
              370,371,372,
              380,381,
              390,391,
              400,420,430,
              500,501,502,503,504,
              550,551,
              600,601,602,603,604,605,606,
              652,654};

  public static void doTransform(String xmlFile, String xslFile, String outFile)
          throws TransformerException, IOException {

      FileReader xslFileReader = new FileReader(xslFile);
      StreamSource xslStreamSource = new StreamSource(xslFileReader);

      FileReader xmlFileReader = new FileReader(xmlFile);
      StreamSource xmlStreamSource = new StreamSource(xmlFileReader);

      FileWriter outFileWriter = new FileWriter(outFile);
      StreamResult outStreamResult = new StreamResult(outFileWriter);

      TransformerFactory transformerFactory = TransformerFactory.newInstance();
      Transformer transformer = transformerFactory.newTransformer(xslStreamSource);

      transformer.transform(xmlStreamSource, outStreamResult);
      outFileWriter.flush();
  }

  public static void migrateAllConfigs(String source, String destination){
      System.out.println("\n\t#######################################################");
      System.out.println("\t#      Apache Synapse - Configuration Migration       #");
      System.out.println("\t#######################################################");

      System.out.println("\n[INFO] Migration STARTED");

      for (int aConfgNumber : confgNumber) {
          try {
              System.out.println("[INFO]Migrating synapse configuration : " + aConfgNumber);
              doTransform(source + aConfgNumber + ".xml", MIGRATOR_XSLT_PATH, destination + aConfgNumber +".xml");

          } catch (TransformerException e) {
              handleException("Migration FAILED\n\t" + e.toString());
          } catch (IOException e) {
              handleException("Migration FAILED\n\t" + e.toString());
          }
      }

      System.out.println("[INFO] Migration COMPLETED");

  }

  public static void main(String args[]) {
      migrateAllConfigs("/home/heshan/Dev/trunk/carbon/products/esb/modules/samples/src/main/conf/synapse/synapse_sample_",
              "/tmp/synapse-config/synapse_sample_");
  }

  private static void handleException(String message) {
      System.out.println("[ERROR] " + message);
  }
}

Tuesday, July 20, 2010

Sending UTF-16 messages through Synapse

When fixing SYNAPSE-662 and SYNAPSE-670, I had to send UTF-16 messages to the SimpleStockQuoteService to verify the fixes. Following are the sample configuration and client code that I used to verify the fixes.

1. Start the Synapse ESB with the following configuration.


<definitions xmlns="http://ws.apache.org/ns/synapse">
   <switch source="//m0:getQuote/m0:request/m0:symbol" xmlns:m0="http://services.samples/xsd">
       <case regex="IBM">
           <!-- the property mediator sets a local property on the *current* message -->
           <property name="symbol" value="Great stock - IBM"/>
       </case>
       <case regex="MSFT">
           <property name="symbol" value="Are you sure? - MSFT"/>
       </case>
       <default>
           <!-- it is possible to assign the result of an XPath expression as well -->
           <property name="symbol"
                 expression="fn:concat('Normal Stock - ', //m0:getQuote/m0:request/m0:symbol)"
                 xmlns:m0="http://services.samples/xsd"/>
       </default>
   </switch>

   <log level="custom">
       <!-- the get-property() XPath extension function allows the lookup of local message properties
           as well as properties from the Axis2 or Transport contexts (i.e. transport headers) -->
       <property name="symbol" expression="get-property('symbol')"/>
       <!-- the get-property() function supports the implicit message headers To/From/Action/FaultTo/ReplyTo -->
       <property name="epr" expression="get-property('To')"/>
   </log>

   <!-- Send the messages where they are destined to (i.e. the 'To' EPR of the message) -->
   <send/>
</definitions>

2. Start the Axis2 server and deploy the SimpleStockQuoteService.

3. Use the following axis2 client (which generates a UTF-16 message) to invoke the service.


package org.wso2.esb;

import org.apache.axiom.om.OMElement;
import org.apache.axiom.om.OMNamespace;
import org.apache.axiom.om.OMFactory;
import org.apache.axiom.om.OMAbstractFactory;
import org.apache.axis2.AxisFault;
import org.apache.axis2.addressing.EndpointReference;
import org.apache.axis2.client.Options;
import org.apache.axis2.client.ServiceClient;
import org.apache.axis2.context.ConfigurationContext;
import org.apache.axis2.context.ConfigurationContextFactory;

import java.lang.*;

public class MyClient {
   public static OMElement createPayload(){
       OMFactory fac = OMAbstractFactory.getOMFactory();
       OMNamespace omNs = fac.createOMNamespace("http://services.samples", "ns");
       OMElement method = fac.createOMElement("getQuote", omNs);
       OMElement value1 = fac.createOMElement("request", omNs);
       OMElement value2 = fac.createOMElement("symbol", omNs);

       value2.addChild(fac.createOMText(value1, "IBM"));
       value1.addChild(value2);
       method.addChild(value1);
       return method;
   }

   public static void main(String args[]){
       OMElement payload = createPayload();
       ServiceClient serviceclient;

       ConfigurationContext configurationContext = null;
       try {        
           configurationContext = ConfigurationContextFactory.
                  createConfigurationContextFromFileSystem("/home/heshan/Dev/trunk/synapse/modules/distribution/target/synapse-2.0.0-SNAPSHOT/samples/axis2Client/client_repo",
                          "/home/heshan/Dev/trunk/synapse/modules/distribution/target/synapse-2.0.0-SNAPSHOT/samples/axis2Client/client_repo/conf/axis2.xml");

          serviceclient = new ServiceClient();
          Options opt = new Options();

          opt.setTo(new EndpointReference ("http://localhost:8281/services/StockQuote"));
          opt.setAction("urn:getQuote");
    opt.setProperty(org.apache.axis2.Constants.Configuration.CHARACTER_SET_ENCODING,"UTF-16");
          opt.setSoapVersionURI(org.apache.axiom.soap.SOAP11Constants.SOAP_ENVELOPE_NAMESPACE_URI);
          serviceclient.setOptions(opt);

          OMElement ome = serviceclient.sendReceive(payload);
          System.out.println("The output is : " + ome);

       } catch (AxisFault axisFault) {
           axisFault.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.

       }

   }
}

Friday, April 30, 2010

Mediator to support Database stored procedure

Following blogpost shows how to use Apache Synapse's dblookup mediator to support a mysql database stored procedure.

Setting up mysql database server
1. Install mysql server.

heshan@heshan-laptop:~$ apt-get install mysql

2. Connect to mysql server.

heshan@heshan-laptop:~$ mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 300
Server version: 5.0.75-0ubuntu10.3 (Ubuntu)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql>

3. Create a sample databasae

mysql>CREATE DATABASE comp;

4. Create a table using the following statement.

mysql>CREATE TABLE company(name varchar(10), id varchar(10), price double);

5. Inserts some data using following statements

mysql>INSERT INTO company VALUES ('IBM','c1',3.7563);
mysql>INSERT INTO company VALUES ('SUN','c2',3.2554);
mysql>INSERT INTO company VALUES ('MSFT','c3',0.0);

6. Create a Stored Procedure.

mysql>CREATE PROCEDURE getCompany(compName VARCHAR(10)) SELECT name, id, price FROM company WHERE name = compName;

7.Add mysql-connector-java-5.1.12-bin.jar to the classpath. This can be done by putting the jar into the Synapse lib directory.

Running the Synapse sample
1. Save the following synapse-configuration as synapse_sample_364.xml to the samples folder.

SYNAPSE_HOME/repository/conf/sample/synapse_sample_364.xml

2. Synapse configuration.

<!-- SYNAPSE_HOME/repository/conf/sample/synapse_sample_364.xml -->
<definitions xmlns="http://synapse.apache.org/ns/2010/04/configuration">

<sequence name="myFaultHandler">
   <makefault response="true">
       <code value="tns:Receiver" xmlns:tns="http://www.w3.org/2003/05/soap-envelope"/>
       <reason expression="get-property('ERROR_MESSAGE')"/>
   </makefault>
   <send/>
   <drop/>
</sequence>

<sequence name="main" onError="myFaultHandler">
   <in>
       <log level="custom">
           <property name="text"
                     value="** Looking up from the Database **"/>
       </log>
       <dblookup>
           <connection>
               <pool>
                   <driver>com.mysql.jdbc.Driver</driver>
                   <url>jdbc:mysql://localhost/comp</url>
                   <user>root</user>
                   <password>admin</password>
               </pool>
           </connection>
           <statement>
               <sql>call getCompany(?)</sql>
               <parameter expression="//m0:getQuote/m0:request/m0:symbol"
                          xmlns:m0="http://services.samples" type="VARCHAR"/>
               <result name="company_id" column="id"/>
           </statement>
       </dblookup>

       <switch source="get-property('company_id')">
           <case regex="c1">
               <log level="custom">
                   <property name="text"
                             expression="fn:concat('Company ID - ',get-property('company_id'))"/>
               </log>
               <send>
                   <endpoint>
                       <address uri="http://localhost:9000/services/SimpleStockQuoteService"/>
                   </endpoint>
               </send>
           </case>
           <case regex="c2">
               <log level="custom">
                   <property name="text"
                             expression="fn:concat('Company ID - ',get-property('company_id'))"/>
               </log>
               <send>
                   <endpoint>
                       <address uri="http://localhost:9000/services/SimpleStockQuoteService"/>
                   </endpoint>
               </send>
           </case>
           <case regex="c3">
               <log level="custom">
                   <property name="text"
                             expression="fn:concat('Company ID - ',get-property('company_id'))"/>
               </log>
               <send>
                   <endpoint>
                       <address uri="http://localhost:9000/services/SimpleStockQuoteService"/>
                   </endpoint>
               </send>
           </case>
           <default>
               <log level="custom">
                   <property name="text" value="** Unrecognized Company ID **"/>
               </log>
               <makefault response="true">
                   <code value="tns:Receiver"
                         xmlns:tns="http://www.w3.org/2003/05/soap-envelope"/>
                   <reason value="** Unrecognized Company ID **"/>
               </makefault>
               <send/>
               <drop/>
           </default>
       </switch>
       <drop/>
   </in>

   <out>
       <send/>
   </out>

</sequence>

</definitions>

3. Start the Synapse configuration numbered 364.

i.e. synapse -sample 364

4. Start the Axis2 server and deploy the SimpleStockQuoteService if not already done.

5. Run the client.

ant stockquote -Daddurl=http://localhost:9000/soap/SimpleStockQuoteService -Dtrpurl=http://localhost:8280/ -Dsymbol=IBM

6. Voila, you have invoked a database stored procedure.

Tuesday, April 27, 2010

Running Synapse ESB's Script Mediator with include option

Apache Synapse ESB's Script Mediator can be used for message mediation. Sample 350 of Synapse samples is demonstrating how this Script Mediator can be used in a real world scenario.

Suppose the script you specified is calling a function of another script. Then the latter script should also be included in the script mediator configuration. It's done using the <include> element. The key attribute of the <include> element can be specified with the script program statements stored in a separate file, which is referenced via the local or remote registry entry. Following example shows how it can be done.

<script key="string" language="string" [function="script-function-name"]>
  <include key="string"/>
</script>

I am using the same example used in Sample 350 to demonstrate this scenario.

1) Replace the stockquoteTransform.js with the following javascript.


// stockquoteTransform.js
function transformRequest(mc) {
transformRequestFunction(mc);
}

function transformResponse(mc) {
transformResponseFunction(mc);
}

2) Add the following sample.js file to SYNAPSE_HOME/repository/samples/resources/script


// sample.js
function transformRequestFunction(mc) {
var symbol = mc.getPayloadXML()..*::Code.toString();
mc.setPayloadXML(
 <m:getquote m="http://services.samples">
    <m:request>
       <m:symbol>{symbol}</m:symbol>
    </m:request>
 </m:getquote>);
}

function transformResponse(mc) {
var symbol = mc.getPayloadXML()..*::symbol.toString();
var price = mc.getPayloadXML()..*::last.toString();
mc.setPayloadXML(
 <m:checkpriceresponse m="http://services.samples/xsd">
<m:code>{symbol}</m:code>
<m:price>{price}</m:price>
 </m:checkpriceresponse>);
}

3) Update the synapse configuration with the following configuration.


<definitions xmlns="http://ws.apache.org/ns/synapse">
<localEntry key="stockquoteScript" src="file:repository/conf/sample/resources/script/stockquoteTransform.js"/>
<localEntry key="sampleScript" src="file:repository/samples/resources/script/sample.js"/>

<in>
 <!-- transform the custom quote request into a standard quote request expected by the service -->
 <script language="js" key="stockquoteScript" function="transformRequest">
 <include key="sampleScript"/>
<script>
 <send>
     <endpoint>
         <address uri="http://localhost:9000/soap/SimpleStockQuoteService"/>
     </endpoint>
 </send>
</in>
<out>
 <!-- transform the standard response back into the custom format the client expects -->
 <script language="js" key="stockquoteScript" function="transformResponse"/>
     <include key="sampleScript"/>
<script>
 <send/>
</out>
</definitions>

4) Start the Synapse ESB with the above configuration. Deploy the SimpleStockQuote service (which is shipped with Synapse) on SimpleAxisServer. Then invoke the client giving the following command.


ant stockquote -Daddurl=http://localhost:9000/services/SimpleStockQuoteService -Dtrpurl=http://localhost:8280/ -Dmode=customquote

5) It will yield the following result.

   [java] Custom :: Stock price = $161.76045110619708

6) For more information.

WSO2 ESB articles - http://wso2.org/library/esb
Apache Synapse - http://synapse.apache.org/