Thursday, July 14, 2011

XSLT transformations with WSO2 ESB

Recently I did some performance comparison on a new mediator that I have developed for WSO2 ESB for XSLT transformations.

Following 6 scenarios were taken into consideration for this benchmark;

1) SCENARIO-1 : Direct Service call to backend service (which is hosted at axis2 server) from the client.

2) SCENARIO-2 : Service Call to the backend service through a Pass through proxy (Using message realy)

3) SCENARIO-3 : Service call to the backend service through the XSLT transformation proxy (with xalan XSLT processor)


3) SCENARIO-3 : Service call to the backend service through the XSLT transformation proxy (with xalan XSLT processor)

4) SCENARIO-4 : Service call to the backend service through the XSLT transformation proxy (with saxon XSLT processor)

5) SCENARIO-5 : Service call to the backend service through the XSLT transformation proxy (with saxon XSLT processor) - with patched XSLT mediator which is doing XML stream transformation (using message relay)

6) SCENARIO-6 : Service call to the backend service through the XSLT transformation proxy (with saxon enterprise edition XSLT processor) - with patched XSLT mediator which is doing XML stream transformation (using message relay)


Following are the configurations of the machines which were used in the performance benchmark. For more information refer the attached document named hardware-configuration-of-machines.txt

Hardware/Software configuration of the machines used

1) Configuration of the machine(32 bit) which was running the client (load testing tool used was java-bench):

OS : Debian GNU/Linux 6.0

CPU : Intel(R) Pentium(R) 4 CPU 2.80GHz

RAM : 1 GB

2) Configuration of the machine(64 bit) which was running the axis2 server which is hosting the service:

OS : Ubuntu 11.04

CPU : Genuine Intel(R) CPU T2400 @ 1.83GHz

RAM : 3 GB

3) Configuration of the machine(64 bit) which was running the WSO2 ESB:

OS : Debian GNU/Linux 6.0

CPU : Pentium(R) Dual-Core CPU E5400 @ 2.70GHz

RAM : 4 GB

Results/Observation

The time per request were measured for the XSLT transformations with the ESB and calculated the latency numbers. The message sizes used were 5k, 10k, 50k, 100k, 200k, 300k, 400k, 500k. The time per request were measured for the number of requests 10000, 11000, 12000, 13000, 14000, 15000. During the benchmarking session, I got the numbers for xalan as well as saxon XSLT processors. Since the primary concern of this benchmark was on latency, the concurrency level used was one.

i) Time per request

Request Size

Direct request (ms)

ESB -Pass through(Message relay) - Time per request(ms)

ESB (xalan) – Time per request (ms)

ESB (saxon) – Time per request (ms)

ESB patched (saxon) -– Time per request (ms)

ESB Patched (Saxon EE)

Timer per request (ms)

5k

2.932

5.087

11.458

9.899

7.486

7.413

10k

4.564

6.332

16.73

15.387

11.145

10.473

50k

17.881

20.984

46.328

42.481

40.834

34.598

100k

34.691

37.339

83.554

79.325

71.151

62.992

200k

67.66

70.771

155.233

146.885

128.672

119.886

300k

100.661

103.945

233.79

213.84

186.985

177.184

400k

133.125

137.898

295.675

283.95

245.029

238.081

500k

166.008

169.460

364.027

356.182

302.779

297.676

ii) Latency

The latency was calculated by the following equation.

Latency = (Time per request through ESB - Direct request)

Request Size

ESB(Message relay) latency (ms)

ESB (xalan) latency (ms)

ESB (saxon) latency (ms)

ESB patched (saxon) latency (ms)

ESB patched (saxon-EE) latency (ms)

5k

2.155

8.526

6.967

4.554

4.481

10k

1.768

12.166

10.823

6.581

5.909

50k

3.103

28.447

24.6

22.953

16.717

100k

2.648

48.863

44.634

36.46

28.301

200k

3.111

87.573

79.225

61.012

52.226

300k

3.284

133.129

113.179

86.324

76.523

400k

4.773

162.55

150.825

111.904

104.956

500k

3.452

198.019

190.174

136.771

131.668

iii) Overhead

The overhead percentage introduced by the ESB is calculated by the following equation.

ESB Overhead = (Time per request through ESB - Direct request time)/(Direct request time) * 100 %

Request Size

ESB(Message relay) overhead(%)

ESB (xalan) overhead percentage (%)

ESB (saxon) overhead percentage (%)

ESB patched (saxon) overhead percentage (%)

ESB patched (saxon-EE) overhead percentage (%)

5k

73.49931787176

290.7912687585

237.619372442

155.3206002729

152.830832196453

10k

38.737949167397

266.5644171779

237.1384750219

144.1936897458

129.469763365469

50k

17.353615569599

159.0906548851

137.5761981992

128.3653039539

93.490296963257

100k

7.6331036868352

140.8520942031

128.661612522

105.0993052953

81.580236949065

200k

4.5979899497488

129.4309784215

117.0928170263

90.1744014189

77.188885604493

300k

3.2624353026495

132.2547957998

112.4357993662

85.7571452698

76.020504465483

400k

3.585352112676

122.103286385

113.2957746479

84.059342723


78.840187793427

500k

2.0794178593803

119.2828056479

114.5571297769

82.3881981591

79.314249915667

Observation

In case of scenarios 3 & 4, the only difference is the underlying XSLT processor used by the ESB is changed from the xalan to saxon. When comparing the performance numbers of xalan and saxon; saxon is showing a little performance improvement. Therefore, there is no significant gain with the use of saxon as opposed to xalan (which is the default XSLT processor shipped with the ESB).

In case of scenario 5, I developed a new mediator to do XSLT transformation by passing in the stream directly to the XSLT processor with the use of Message Relay.

Compared to scenarios 3 & 4, scenario 5 is showing a significant performance improvement.

The difference between scenario 5 and 6 is that the underlying saxon XSLT processor is changed from the community edition to enterprise edition. Compared to scenario 5, scenario 6 is showing a performance improvement.

Throughout this post I have talked about a transformation mediator which is supporting streaming. This will be available with the next release of the WSO2 ESB. This meadiator's name will be relayTransformer.

I will write a new post on this new mediator on some other time.

NOTE: This performance figures were calculated, inorder for me to get an idea about the performance gain of this new mediator. These figures were not calculated in an ideal environment. Therefore, if you try to compare these numbers with a performance benchmark that you might have done, it might not be comparable.

References

[1] - http://wso2.org/products/download/esb/java/4.0.0/wso2esb-4.0.0.zip

[2] - http://heshans.blogspot.com/search/label/ESB

[3] - http://wso2.org/project/esb/java/4.0.0/docs/samples/message_mediation_samples.html#Sample8


No comments: