A Simple Netkernel Spatial Application

Andrew Hallam | | 7 May 2007, 03:21

The simple Netkernel application that I had previously put together was quite basic, and not that interesting from a real world perspective. Here’s a more practical example that also uses PostGIS to perform some spatial processing.

Requirement: Given an X/Y coordinate, find me the nearest piece of equipment.

Request:
http://hostname/webdb/nearest-equip?x=9555555&y=4555555

Application:

(Apology: My syntax highlighter has trouble with XML namespaces, so I’ve had to present the plain text version.)

<idoc>
  <seq>
    <instr>
      <type>SQLEscapeXML</type>
      <operand>this:param:param</operand>
      <target>var:param</target>
    </instr>
    <instr>
      <type>xslt</type>
      <operand>var:param</operand>
      <operator>
        <xsl:stylesheet 
          xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
          version="1.0">
          <xsl:template match="/nvp">
            <sql>
              SELECT equip.equipment_id, 
              Distance(
              GeomFromText(
              'POINT(<xsl:value-of select="x"/>
              <xsl:text> </xsl>
              <xsl:value-of select="y"/>
              )'
              ), equip.the_geom
              ) as distance
              FROM equip
              ORDER BY distance ASC
              LIMIT 1;
            </sql>
          </xsl:template>
          </xsl>
      </operator>
      <target>var:sql</target>
    </instr>
    <instr>
      <type>sqlQuery</type>
      <operand>var:sql</operand>
      <target>var:queryResult</target>
    </instr>
    <instr>
      <type>xslt</type>
      <operand>var:queryResult</operand>
      <operator>
        <xsl:stylesheet 
          xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
          version="1.0">
          <xsl:output indent="no" method="xml"/>
          <xsl:template match="/results/row">
            <equipment>
              <xsl:copy-of select="./*"/>
            </equipment>
          </xsl:template>
          </xsl>
      </operator>
      <target>this:response</target>
    </instr>
  </seq>
</idoc>

Normally I’d store the XSL in separate files, but have included it here inline so you can see what’s going on.

The result is a small XML document that looks like:

<?xml version="1.0" encoding="UTF-8"?>
<equipment> 
  <equipment_id>123456</equipment_id>
  <distance>78349.5745430395</distance>
</equipment>

Of course, all the cool cats will want JSON output rather than “boring old XML”, so here’s how to do it. The new <instr>uction at the bottom of the DPML document adds a whopping 20% to the amount of code in the application. :-)

<idoc>
  <seq>
    <instr>
      <type>SQLEscapeXML</type>
      <operand>this:param:param</operand>
      <target>var:param</target>
    </instr>
    <instr>
      <type>xslt</type>
      <operand>var:param</operand>
      <operator>
        <xsl:stylesheet 
          xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
          version="1.0">
          <xsl:template match="/nvp">
            <sql>
              SELECT equip.equipment_id, 
              Distance(
              GeomFromText(
              'POINT(<xsl:value-of select="x"/>
              <xsl:text> </xsl:text>
              <xsl:value-of select="y"/>
              )'
              ), equip.the_geom
              ) as distance
              FROM  equip
              ORDER BY distance ASC 
              LIMIT 1;
            </sql>
          </xsl:template>
          </xsl>
      </operator>
      <target>var:sql</target>
    </instr>
    <instr>
      <type>sqlQuery</type>
      <operand>var:sql</operand>
      <target>var:queryResult</target>
    </instr>
    <instr>
      <type>xslt</type>
      <operand>var:queryResult</operand>
      <operator>
        <xsl:stylesheet 
          xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
          version="1.0">
          <xsl:output indent="no" method="xml"/>
          <xsl:template match="/results/row">
            <equipment>
              <xsl :copy-of select="./*"/>
            </equipment>
          </xsl:template>
          </xsl>
      </operator>
      <target>var:xmlResult</target>
    </instr>
    <instr>
      <type>JSONFromXML</type>
      <operand>var:xmlResult</operand>
      <target>this:response</target>
    </instr>
  </seq>
</idoc>

Spatially enabled database. 100% declarative code. Love it!

Update: The above spatial query takes 80ms to execute on my 2Ghz laptop, although the equip table only contains 270 records.

A Simple Netkernel Application

Andrew Hallam | | 27 April 2007, 19:08

I’ve been interested in Netkernel for a while. It offers a decarative development approach, like using SQL to query a database instead of writing the code that performs the actual query processing. Netkernel also gets extra bonus points for embracing REST (but I digress).

Netkernal lets you focus on logical resources that are loosely coupled. You can use declarative and imperative languages where they make sense.

As a simple example, I set out to build the simplest possible application that could accept an HTTP GET request, query a database, and return the results of the query as an XML document. Here’s the core application, minus the configuration of the module and database connection:

<idoc>
  <seq>
    <instr>
      <type>sqlQuery</type>
      <operand>
        <sql>
          SELECT id, name 
          FROM items;
        </sql>
      </operand>
      <target>var:queryResult</target>
    </instr>
    <instr>
      <type>xslt</type>
      <operand>var:queryResult</operand>
      <operator>ffcpl:/resources/style/get-item-list.xsl</operator>
      <target>this:response</target>
    </instr>
  </seq>
</idoc>

This is a Declarative Process Markup Language (DPML) document. It contains a <seq>uence of three two <instr>uctions. The first instruction queries the database and stores the result in the variable “queryResult”. If you view an XML representation of the query result it looks like:

<results>
  <row>
    <id>1</id>
    <name>Foo</name>
  </row>
  <row>
    <id>2</id>
    <name>Bar</name>
  </row>
</results>

This could be returned as the response document, but elements named results/row don’t exactly describe the sematics of the data. Therefore, the next instruction uses a small XSL stylesheet to change the document structure to:

<itemlist>
  <item>
    <id>1</id>
    <name>Foo</name>
  </item>
  <item>
    <id>2</id>
    <name>Bar</name>
  </item>
</itemlist>
<pre>

The third instruction “casts” the result to a MIME type of text/xml. As Peter pointed out in the comments, if I use the stylesheet to define the output method as XML, using <xsl:output method=“xml”/>, I don’t need the third instruction, which was:

<instr>
  <type>cast</type>
  <operand>var:serviceResult</operand>
  <operator>
    <cast>
      <mimetype>text/xml</mimetype>
    </cast>
  </operator>
  <target>this:response</target>
</instr>

Notice that I haven’t had to worry about object types. Netkernel takes care of that for me.

Production Ready

The use of a declarative development approach means that if it works as desired it’s ready for production. There is no imperative code in this application so there isn’t a whole lot to test. The only optimisation would be to use Netkernel’s built in caching to minimise the load of the database.

This example plays to Netkernel’s strengths, but contrast it with the imperative code required to build this application in Java or C#.Net.

The Backstory

A while ago I ran some code metrics on a Java servlet Web application that I’d written. I was surprised at just how much code was involved. 5,500 non-comment source statements (NCSS) for an application that had about 20 data entry forms. That seemed like a lot.

Stripes had been used for the Web framework and Hibernate was used for the database layer. These two tools had reduced the amount of Java code that had to be be written, and about 500 NCSS were to manage an archaic interface with another system. That leaves 5,000 NCSS for user interface logic and domain model.

This exercise prompted me to start looking at alternative ways of developing applications. What’s that? Ruby on Rails, Django, and friends. Sure, they are options, but I was looking for something different. I wanted something declarative, not imperative. Netkernel fits that requirement.

Update: A simple Netkernel Spatial application

HTTP in Adobe's Flex

Andrew Hallam | | 8 April 2007, 04:15

Flex’s HTTPService class supports “GET, POST, HEAD, OPTIONS, PUT, TRACE and DELETE”. Mmm…interesting.

The REST community has long bemoaned the lack of native browser support for HTTP verbs other than GET and POST. The XMLHttpRequest object has provided one option for AJAX applications. Flex provides similar capabilities for Flash “rich Internet applications”.

[Note: First post after upgrade to WordPress 2.1.3.]

SOAP is Dead?

Andrew Hallam | | 20 November 2006, 02:29

Mmm…may be a little premature to make that call, but it’s good to see the alternatives getting a run. Following on from my last post, here are some more links:

Nelson Minar tells us Why SOAP Sucks, based on his experience helping Google implement SOAP APIs.

Duncan Cragg’s is writing a 9 part series titled “The REST Dialogues”:

Part 1: Getting Data.
Part 2: Setting Data.

SOAP was Simple

Andrew Hallam | | 17 November 2006, 06:14

Peter Lacey’s take on SOAP, The S stands for Simple, is funny and contains more than a little truth.

Via Mark Baker.

WMS Practicalities and REST

Andrew Hallam | | 22 October 2006, 01:53

Sean Gillies asked Why Does WFS Dislike the Web?. I haven’t played with WFS in years, but I’d like to explore a point raised by Allan Doyle in a the comment on Sean’s post.

Allan mentioned that adoption of a technology depends on how easy it is to get something working quickly. He was comparing KML with OGC specifications.

In the case of WMS, if it had been designed to follow REST principles I don’t think it would have been as successful because it would have been more difficult to use.

REST says that a URI identifies an abstract resource. HTTP headers are used in the request to convey metadata about the desired resource representation (e.g. Accepts), and in the response to describe the actual resource representation (e.g. Content-Type).

If you look through the WMS query string parameters some of them are metadata that describe the request, the user agent’s desired resource representation, and MIME type to be used for exceptions. Not all are used to identify the abstract resource itself:

  • VERSION
  • WIDTH
  • HEIGHT
  • FORMAT
  • BGCOLOR
  • EXCEPTIONS

In a sense, CRS is also a candidate for being metadata. It’s the same abstract resource regardless of the CRS used for the resource representation (but the CRS is tied to BBOX which is used to describe the resource, so it’s not clear cut).

Anyway, if all those name/value pairs had to be passed through as HTTP headers you would have needed a client other than just a plain old browser to send a WMS request. Not the best way to encourage adoption.

The value of being able to put a URI in your browser’s address bar and see what comes back cannot be understated. Instant gratification counts on the path to getting a quick result.

Update, 2006-10-22: The conversation continues at import cartography.

[tags]WMS, REST, HTTP[/tags]

REST Abuse

Andrew Hallam | | 4 September 2006, 19:04

Public APIs for mapping services are common. Lately, the trend seems to be to call these APIs “RESTful”, because REST (REpresentational State Transfer) has become a bit of a buzzword. However, I’ve yet to see one that actually follows REST principles.

REST is an architectural style. REST is not a toolkit, a push button option in your Integrated Development Environment (IDE), or an API. In fact, if you see parameter name like operation (op), action (actn), or request in the URI query string it is usually an indicator that the REST style is not being used.

You have to consciously build your applications to follow REST principles. Resources, resource collections, links, content types, and state transfer are key ingredients. Paul James does a great job of explaining the issues.

Most mapping services are just HTTP GET requests that return an image. They are “Web-style” and Web friendly. They are useful. But, they are not REST, and calling them such is misleading.

[tags]rest, web-style, map service[/tags]

Web Services Humour - A Parable

Andrew Hallam | | 13 May 2006, 02:52

Here’s some Friday afternoon humour from Elliotte Rusty Harold.

REST vs. WS-*: A Parable

My view on this is that if you really do need the enterprise middleware/complexity required by WS-* then go for it. Just keep in mind that there are many cases where that level of complexity is not needed, and you can do a lot using plain old HTTP and XML. Horses for courses.

ZoomIn In Wordpress Working

Andrew Hallam | | 6 May 2006, 07:34

Thanks to a pointer from James Fee to the Text Control plug-in I now have a ZoomIn map embedded in a Wordpress weblog post.

Note: If you are reading this in an RSS aggregator chances are the scripting has been stripped or ignored. Please try this page.




To get this to work I had to:

  1. Use Text Control to turn off all of Wordpress’ default text formatting, and
  2. Fix a minor bug in the sample code. There was an extra parenthesis on the end of GPoint(151.215,-33.857), 4);

[code]
[/code]

Update: Added a marker for the Sydney Opera House that displays an information window.

ZoomIn in Wordpress

Andrew Hallam | | 5 May 2006, 17:55

In my last post I said it was going to see how easy it was to embed a ZoomIn map in a blog post. Tried. Failed.

When I enter the the <script> … </script> element in the Wordpress editor it escapes some of the characters to HTML entities and Unicode characters and inserts <br /> elements (not good inside a JavaScript block). The <script> elements also get wrapped in <p> elements.

Several attempts at workarounds also failed, including putting all the JavaScript on one line, and wrapping it with <![CDATA[ … ]]> or HTML comment tags.

« Previous |

Powered by Textpattern | Tranquility White made TXP-ready by Textpattern Templates