Digital Earth Weblog

A client has been developing a SOAP web service. I got involved with specifying the web service interface and testing compliance with the specification. I ended up using JMeter and XPath assertions. It doesn’t have the addictive unit test “green bar”, but it worked.

The requirement was to perform the tests on the SOAP messages that passed across the wire. At least one client application will be POSTing raw XML SOAP requests to, and pulling apart SOAP responses from, this service rather than using using a SOAP toolkit. We needed to ensure that the web service interface behaved as expected, and that business logic changes behind the interface had no impact on the client applications.

Prior to setting up the tests in JMeter a quick Google didn’t reveal anything suitable for a quick and dirty test. The tests could have been written unit and executed using JUnit, but a declarative solution that could be emailed to the client was preferred. JMeter did the job, but it did have a few issues.

The Recipe

1. Download and install JMeter. (Activation.jar and mail.jar are required for SOAP testing. See installation notes.)
2. Create a new Test Plan.
3. Add a Thread Group to the Test Plan. Check that the number of threads is set to 1 and the number of loops is 1.
4. For each test, add a Web Service (SOAP) Request samplers as a child of the Thread Group. Name them with a numeric identitifier for ease of reference. e.g. “1. Invalid Foo, Valid Bar”. You will need to supply the URL to the WSDL file and a fully formed SOAP request XML document.
5. For each sampler, add as many XPath Assertions as you need to validate the result. XPath expressions can be used to check pretty much everything about the content and structure of XML documents.
6. Add a View Results Tree listener to capture and display the requests and responses. Any tests that fail will be shown in red.
7. Save your Test Plan.
8. Run it.
9. View results by clicking on the View Results Tree.
10. Clear results before running the test again. Results are stored cumulatively, not replaced by each test.

Here is an example test plan:

The Results

In this example, the fifth test “Invalid Values” did not return the expected results.

The View Results Tree listener also lets you view the HTTP headers, the XML request, and the XML response (not shown here). This is helpful for debugging, and you can also ask JMeter to write that data to file.

Minor Issues

1. No green bar, but then JMeter is not intended to be a unit test framework.
2. Duplication. XPath assertions are repeated in multiple tests. If you need to modify a repeated assertion you need to do it manually. That said, the JMeter project file (.jmx) is an XML document so you can (carefully) do a global search and replace. JMeter also makes it easy to copy and paste any element of the test plan so is not a major drama.

The JMeter project that was created became useful tool. The web service developer can now run the tests at any time, particularly before deploying a new version of the web service. Simple declarative regression testing.

[tags]JMeter, JUnit, unit testing, SOAP, web services[/tags]