Skylab Mobilesystems Crawls the Web for Web Map Services

The number of Web Map Services (WMS) and Web Feature Services (WFS) that implement OpenGIS(R) interfaces on the Internet rises each week as more organizations realize the power of using the open standards. At the same time, the number of WMS and WFS clients - designed for use in a browser, or on the desktop or on a mobile device - are growing. There's a lot of excitement around these developments and one big question comes up time and again: How do I find the different services that are available to use with these standards-based clients? Several different organizations have tackled that question in recent months, but most ultimately depend on organizations "signing up" or registered their WMS or WFS sites. One addition, from Boris Boege at Skylab Mobilesystems in Germany, works differently. "The WMS-Crawler" he explains, "works pretty much like any search engine. It crawls for keywords which might indicate a link to a WMS and tries to parse it with a WMS Capabilities parser. He shared that in its early days, back in April of 2005, the list created by the Crawler was updated irregularly. Now, it gets updated once a week.

Some of the many servers found by WMS Crawler.

 

The Crawler finds a wide variety of services, which means the company must offer this caveat: "This list is generated by an automatic application and lists servers which can be accessed publicly. That doesn't necessarily mean that those servers are open for public use. Please refer to the server's usage policy before using it!" The links in the long list are to the "getcapabilities" statements of servers. Each XML document details what the service can do. And, if fed into a WMS client, they allow access to the data for rendering and querying. By exploring the URLs in detail it's sometimes possible to tease out what software is behind the service, though of course, it does not really matter! ArcIMS, MapServer, ER Mapper Image Web Server and other technologies stood behind services that appeared on the list recently.

Data from a British Columbia data found on the WMS list served by Autodesk MapGuide. It’s rendered by Intergraph’s WMSViewer, an online WMS viewer.

 

Skylab Mobilesystems had a good reason to create the Crawler and host the regularly updated list. "All of our mobile GPS/GIS products (including 'Spot' and 'Spot for Blackberry') integrate a WMS client. We created the list to let our customer take fully advantage of the WMS integration." But, besides the benefit for his business, Boege wants to "encourage other GIS users and developers to make use of interoperable services by showing what is already available now." Boege cites visible growth in the list during the recent months. As we go to press, the list includes 994 servers and 339254 layers of data. Mobilesystems has been upgrading the Crawler and shares some upcoming plans: "WFS will be supported soon and we plan to make the list queryable, including queries for keywords, layer names and later also spatial queries."