Digital Earth Weblog

John Udell has written a really interesting article on Service Oriented Architecture (SOA), and the state of web services, over at InfoWorld. Well worth a read if you are implementing web services.

The article points out that for most point-to-point web services plain old XML over HTTP is all you need. This is the approach taken by the Web Map Service (WMS) and Web Feature Service (WFS) standards.

When the services get more complex then the WS-* stack becomes relevant, but the technical complexity and opportunity for vendor lock-in suddenly increase dramatically. However, most of us can get the job done by moving XML documents around the Web.

Thanks to Tim Bray for the heads up.

Recently I was asked to assist in debugging a SOAP integration issue. A large desktop application was being enhanced to obtain data from a web service created by my client. They were not interoperating as advertised. This post looks at what happens when SOAP interop goes bad.

My sweepingly general theory is that developers like SOAP because it’s easy for them to use. Just create some classes and let the SOAP tools take care of generating WSDL and providing serialisation to/from the XML wire format. Same on the client end. Use the SOAP tools to read the WSDL and generate the required stub classes.

Developers never have to see what travels across the wire, or even understand it, until it doesn’t work. Then they suddenly need to know a lot about XML – SOAP, WSDL, namespaces, and XML Schema. It’s a whole new ballgame.

In this particular case the web service was created using Microsoft Visual Studio and ASP.Net. The desktop application was developed by a company in another country. It was written in C++ so the Microsoft SOAP Toolkit was used as the SOAP client. The challenge was to get the two to work together.

Requests to the web service were quite simple, just a few parameters, but for some reason the request was failing due to a data type error. The error message was not very helpful so the first step was to work out where the data type problem was occurring – on the client or on the server.

We tried two free SOAP monitors without any success. No SOAP traffic was showing up. Ethereal to the rescue. By monitoring HTTP traffic we discovered that the test client application was requesting, and receiving, the WSDL from the web service. But, it was failing before it could issue a SOAP request. Suspected problem at the client end.

A quick test with one of my XML editors (<oXygen/>) helped prove that the web service was working as expected. <oXygen/> has a SOAP client that reads a WSDL file and generates a SOAP request document. We filled in some test values and fired off a request. The SOAP response contained all the right values in the right elements. Problem definitely at the client end.

To cut a long story short, it turned out the test client application was getting the order of the request parameters mixed up and the SOAP Toolkit was complaining about data type errors. I thought this would be difficult to do if you used classes generated from the WSDL, but it turned out that the developer was using low level functions in the SOAP Toolkit, not generated classes. It took quite a few iterations, over the course of several weeks, to get it all working.

In the middle of all this the web service developer and myself had written our own test clients (VB.Net and Delphi) just to prove to ourselves that it was possible to get this working in a few hours. Apache Axis was also used to generate C++ classes from the WSDL.

For better or worse, SOAP has become the industry standard for web services. For me, the take away from this saga was not to assume that every developer out there knows what SOAP and XML are, or how to leverage the benefits that SOAP provides. The tools make it pretty easy to use, but SOAP is only useful if the humans at both ends really understand how to exploit it.

The publicity around Google Earth and World Wind has generated a lot of interest in spatial data. It is significant that a lot of that interest is coming from the managerial ranks. I know of several cases where management are asking if existing spatial data assets can be displayed in Google Earth.

Yesterday (paraphrased): “Is it possible to get XYZ data displayed in Google Earth? If so, what is involved and how long will it take?”

Using the free version of Google Earth I set up a simple proof of concept. It used a single map image as a static overlay. There was a slight projection mismatch, and if you zoomed in the overlay image got all chunky and pixelated. However, it was good enough for a quick demo.

During my afternoon commute I got to thinking: ArcXML is XML, KML is XML (such perception!). Hmm…maybe that XSL stuff might actually be able to transform one to the other. I had some sample ArcXML data on my notebook. Work commenced, and by journey’s end I had an XSL stylesheet that converted an ArcXML IMAGE response into KML, and a bullet point functional specification on what the application needed to do.

Some hacking during the morning commute, and a bit of testing at the client site, resulted in a working baby “KML Connector” servlet for ArcIMS 9. It can only expose one ArcIMS Image Service as KML so there is a lot of core functionality missing. I haven’t even begun to think about producing KML vectors. Still, not bad for about 150 lines of Java code and comments, and a small XSL stylesheet.

The next step is to make it a bit more flexible and to limit the extent to which it will respond with a map image. No point generating a map image if I’m viewing the opposite side of the planet. After that, I’ll see if I can find the time to add some new features.

Aside: ESRI have announced that they are working on supporting Google Earth as a client.

Update, 2006-05-12: A new version of the KML Adapter software is now available as open source.

The Open Web Application Security Project has released version 2.0 of the OWASP Guide to Building Secure Web Applications. If you build web applications, or someone is building one for you, this document is worth a read.

ArcGIS Server is well suited to being the engine behind web services that have a spatial aspect. Tools like Microsoft Visual Studio .Net and Apache Axis (Java) make it easy to expose objects in your project as SOAP web services. There are numerous issues raised by the use of SOAP, but in this post I’ll focus on service design in the ArcGIS Server/ArcObjects environment.

A while ago I witnessed a demonstration that showed how easy it was to create a SOAP web service based on ArcGIS Server. It was a great example of how easy it is to create a SOAP binding using Visual Studio. It was an equally great example of how not to create a well designed web service. Granted, it was just a demo, but there are plenty of people in the GIS community who haven’t yet had a lot to do with web services. For them, it was a very poor introduction.

The person doing the demonstration picked a coarse-grained ArcObject class and created a SOAP interface directly from it. Sure, you can do that, but that’s using web services to do distributed objects. That’s bad form because you are exposing every property and method in that ArcObject class, and every object that it contains, to the outside world. You are also tightly coupling your web service, and the applications that use it, to the underlying ArcObjects implementation.

Web services are about interfaces. The applications that use the web service should not have to know anything about how the web service is implemented. i.e. They don’t care whether it was created using ArcGIS Server, MapXtreme or 100% custom code. All they need to know is how to call the service (the request) and what they get back (the response). That’s the interface.

If you are creating web services that works with ArcGIS Server I recommend that you:

1. Design the interface first.
2. Create facade classes that support the interface, and only then
3. Connect the facade classes to your Application Developer Framework implementation classes.

This approach ensures that your interface does exactly what you need, and is separate from your implementation.

Just a thought: When using a data-centric web application the web page is perceived to be the document. When using a map-centric web application the map is perceived to be the document. Therefore, when browsing the map it is reasonable for the user to expect to be able to pan and zoom around the the map without the web page refreshing.

Sean McGrath points to a post by Martin Fowler who suggests that the term Service Oriented Architecture (SOA) is no longer useful because it is too ambiguous.

“It’s beyond saving – so the concrete ideas that do have some substance need to get an independent life.”

Sean McGrath offers some thoughts on what is worth saving.

JavaServer Faces (JSF), and ASP.Net, are relatively new event driven web application frameworks. They as designed to enable developers, who are familiar with tools like Visual Basic and Delphi, to easily create web applications. The developers drag and drop components onto a canvas (representing a web page), and then write event handler code to glue it all together.

Leaving aside the relative merits of drag-n-drop development, I’d like to focus on the use of event driven web application frameworks for creating map-centric web applications. By “map-centric” I’m referring to web applications where the user spends a lot of time zooming, panning, switching layers on/off, and querying spatial features.

Both JSF and ASP.Net are component-based web frameworks. In short, this means that actions performed by the user in their web browser are sent back to the server where those actions trigger events. Examples include submitting a form, and clicking on a navigation link.

This approach works fine for data-centric business applications. These types of applications are built around the concept of submitting chunks of data to a server for processing. Put another way, data-centric application are “transactional”, both from the user’s point of view and the developer’s point of view.

So what’s the problem? Well, map-centric web applications are not data entry forms. They are a richer environment because, well, they contain maps. Maps on the Web are a continuous resource. They change when you pan (X, Y), and when you zoom (Z), but to the user it’s still the same map. Maps are not perceived as “transactional”.

The user is not expecting the map to disappear, or the entire web page to reload, every time they interact with the map. That’s a poor user experience. Yet, this is what happens when you build a web mapping application using the Java (JSF) and .Net Application Developer Frameworks that ship with ArcGIS Server 9.0. By default, every time the user pans, zooms, turns a layer on or off, or queries a spatial feature, the entire web page reloads.

Hey, it gets the job done, doesn’t it? Well yes, but it’s clumsy. Click-n-wait web mapping is not productive for the end user, and I prefer to do better for my clients. After all, they’ve all seen Google Maps by now.

So what are your choices?

1. Subvert the Framework

If you have no choice but to use a web application framework like JSF or ASP.Net then you will have to write some code to work around the limitations of the framework. This is not ideal. Frameworks are supposed to make developers more productive by letting them focus on the application rather than workarounds.

2. Do It Yourself

Building it yourself is likely to be time consuming, but you will get exactly what you need. Yet, it’s possibly better to…

3. Use Another Framework

Use a presentation framework that will let you develop the web applications that suit your needs. This can also save considerable time and money, especially when used for multiple projects. There are several commercial and open source options, including:

• Internet Mapping Framework (IMF) from Moxi Media (commercial)
• Community Map Builder (open source)

Of course, it helps if your map server support well known interfaces, like WMS or ArcXML. ArcGIS Server doesn’t provide such interfaces, but in its defence, it was not designed to be just a map publishing platform.

4. Use a Thicker Client

For highly interactive web mapping applications the use of a thicker client is well worth considering. i.e. A browser plug-in or Java applet. A lot of people scoff at this idea because it’s not a pure browser client, but it does have its merits.

Data can be transmitted across the network in highly compressed formats, and cached in the browser. Vectors can be live selectable objects in the browser, rather than just pixels in a dumb image. This lets you do things like select features by region or attribute, create buffers, etc, without having to send a request to the server.

Tools like iDelve, Image Web Server, and MapGuide have been providing these capabilities for years.

There are plenty of options. However, I’m not convinced that the relatively new component-based web application frameworks like JSF and ASP.Net are the right way to go for map-centric web applications. The game has already moved on.

Not to be outdone, Yahoo! have Yahoo! Maps Web Services. No Australian data there either. It wouldn’t have anything to do with the price of data in this country, would it?

Via All Points Blog.

Google have published the Google Maps API. Now, Google, how about some Australian data…please?