SOAP Story
Recently I was asked to assist in debugging a SOAP integration issue. A large desktop application was being enhanced to obtain data from a web service created by my client. They were not interoperating as advertised. This post looks at what happens when SOAP interop goes bad.
My sweepingly general theory is that developers like SOAP because it’s easy for them to use. Just create some classes and let the SOAP tools take care of generating WSDL and providing serialisation to/from the XML wire format. Same on the client end. Use the SOAP tools to read the WSDL and generate the required stub classes.
Developers never have to see what travels across the wire, or even understand it, until it doesn’t work. Then they suddenly need to know a lot about XML – SOAP, WSDL, namespaces, and XML Schema. It’s a whole new ballgame.
In this particular case the web service was created using Microsoft Visual Studio and ASP.Net. The desktop application was developed by a company in another country. It was written in C++ so the Microsoft SOAP Toolkit was used as the SOAP client. The challenge was to get the two to work together.
Requests to the web service were quite simple, just a few parameters, but for some reason the request was failing due to a data type error. The error message was not very helpful so the first step was to work out where the data type problem was occurring – on the client or on the server.
We tried two free SOAP monitors without any success. No SOAP traffic was showing up. Ethereal to the rescue. By monitoring HTTP traffic we discovered that the test client application was requesting, and receiving, the WSDL from the web service. But, it was failing before it could issue a SOAP request. Suspected problem at the client end.
A quick test with one of my XML editors (<oXygen/>) helped prove that the web service was working as expected. <oXygen/> has a SOAP client that reads a WSDL file and generates a SOAP request document. We filled in some test values and fired off a request. The SOAP response contained all the right values in the right elements. Problem definitely at the client end.
To cut a long story short, it turned out the test client application was getting the order of the request parameters mixed up and the SOAP Toolkit was complaining about data type errors. I thought this would be difficult to do if you used classes generated from the WSDL, but it turned out that the developer was using low level functions in the SOAP Toolkit, not generated classes. It took quite a few iterations, over the course of several weeks, to get it all working.
In the middle of all this the web service developer and myself had written our own test clients (VB.Net and Delphi) just to prove to ourselves that it was possible to get this working in a few hours. Apache Axis was also used to generate C++ classes from the WSDL.
For better or worse, SOAP has become the industry standard for web services. For me, the take away from this saga was not to assume that every developer out there knows what SOAP and XML are, or how to leverage the benefits that SOAP provides. The tools make it pretty easy to use, but SOAP is only useful if the humans at both ends really understand how to exploit it.
What do you think about REST (representational state transfer) services?
I have seen a lot of large companies using REST (Yahoo API, 80% of Amazon web services are REST based, and others for example), but I have not seen the GIS industry make large use of it. The most notable example would be WMS and Google Earth (Network links are REST based). Actually ArcIMS, at its base communication level, is REST based (XML request to the servlet connector via port 80 which leads to an xml response from ArcIMS).
What about ArcGIS server or ArcWeb services? They seem to be only SOAP based.
For more information on REST:
http://webservices.xml.com/pub/a/ws/2002/02/06/rest.html
http://www.xml.com/pub/a/2002/05/08/deviant.html?page=last&x-showcontent=off
Cheers,
Jeremy
Jeremy — 21 September 2005, 10:55
I'm a REST fan, and try to use REST principles in my web applications where I can. I would have prefered that a simple REST service (XML over HTTP) be used instead of SOAP for the web service I mentioned in the post above. It would have been a lot more transparent than dealing with SOAP stack black boxes.
From my observation, there are two factors that are driving the adoption of SOAP by individual developers (I'm ignoring the situation where architecture, tools and standards are dictated by a central authority). 1) IDE's like Visual Studio .Net make it easy to create SOAP web services without the developer having to understand what is happening under the covers, and 2) a lot of developers don't want to deal with XML, but they are happy to deal with classes. Unfortunately the result is distributed objects, yet again. Not enough thought goes into the design of the service interface.
ESRI Australia are promoting the use of SOAP, and in doing so are claiming that ESRI products are complying with industry standards. Maybe, but by mentioning UDDI, and the way they demonstrated the use of SOAP with ArcGIS Server (binding directly to ArcOjects), didn't gain them any credibility in my eyes. That said, there is nothing stopping a developer from writing a REST style interface on top of ArcGIS Server. An ArcXML-like approach would have been fine for stateless (pooled SOC) ArcGIS Server applications.
I've been meaning to have a look at ArcWeb Services but haven't got around to it yet.
Andrew
Andrew Hallam — 21 September 2005, 18:54
I have never quite understood why some sophisticated developers don't like to deal with XML.
We have just started messing with ArcGIS server and dealing with the binding of services directly to ArcObjects is a major ordeal. It seems almost impossible to debug (thanks for pointing out the editor, it has been very useful for debuging SOAP services in AGS).
I am hopeful for the next version of ArcGIS server. I have heard about the ability to publish geoprocessing models as SOAP services through AGS. I am also assuming that if ArcIMS gets merged with AGS (I hope that is many years down the road) that someone at ESRI will build an ArcXML service that sits on top of ArcObjects. This could be SOAP based or (even better) REST based.
I have used the ArcWeb public geocoding SOAP service and it works quite well.
I have been using both Google's API (SOAP based) and Yahoo's API (REST based) for the last year or so. Initially Google's SOAP service worked flawlessly, but recently it has been timing out and throwing SOAP errors. I have basically switched to the Yahoo service--no timeouts, easy error tracking.
Anyway good post.
Cheers,
Jeremy
Jeremy — 22 September 2005, 22:14
Tom
Tom — 28 September 2005, 19:45
That's excellent news. I look forward to seeing the what has been done. Now, I just need to find an excuse to play with ArcWweb Services.
Andrew
Andrew Hallam — 28 September 2005, 19:50
Jeremy
Jeremy — 1 October 2005, 00:41