Category: Thoughts

Manage Your Content – Integrate Your Content

I talked a little bit on the WeoGeo blog last week about how we were very focused on content management. Yea, very sexy stuff… But what I think it highlights is the way that we can share our data with each other. Getting your data “into the Cloud”, whatever that means these days, and then using it as much as possible gives you the best return on your investment. One thing that did come up at the FedUC in February was the huge adoption of ESRI’s ArcGIS Online Map Services among users. Using these great free services as backdrops to your mapping content gives you a great starting point. But how you can integrate your geo-content into them is critical. Making it as easy and simple to do is how you’ll be able to leverage your data.

The “Map Sandwich”

ESRI’s cartography blog of all places has the key to showing where data vendors (and just all around geo-Joes) can leverage their datasets in this new ecosystem of free web services. Now the blog post focuses on the cartography aspects of this mashup, but the huge takeaway here is that you can easily integrate your data into these free services in ways that your users/customers can leverage easily. What users want is to quickly integrate your data into their ecosystem. This means they want to consume them on their terms, not yours. Companies that successfully integrate with the Google, the Microsoft and large GIS vendors such as the ESRI, will see great consumption of their data. Those that create their own private data sharing sites or web services will see their fortunes decline, like Paul says.

A Window into the Future

I’ll tell you right now who I see fitting perfectly into this “Map Sandwich” world, Brian Flood’s Arc2Earth. Take a look at his demo app, “Tax Parcel Search – Westfield, NJ”. What is really cool about this demo app is that Brian has a link to the API that makes it happen. He based this API on some draft standards so it should be really easy to integrate into just about any application out there, but the chocolate syrup and cherry on top is his ArcGIS Server REST API Compatibility. Now, he still hasn’t released this publicly (at least that I can see), but it means that any data you create using his Arc2Earth API will be easily consumable in the ESRI ecosystem natively. No wacky WMS or WFS for ESRI users, this stuff will be copy and paste stupid easy.

Customers want this integration and it just isn’t ESRI users. We can sit on our high horse all we want and talk about open standards, but in the trenches people with money don’t have time for the OGC and others to get their act together (and even then they could care less).

“Give me a web service that integrates into my ArcCatalog natively and I’ll buy”

That simple.

“I’m using the ESRI Flex API and want to use your dataset.”

Here is the ESRI REST URL, have at it.

Give Me Some of that Good Data-as-a-Feature

We should be looking at these Data-as-a-Feature services as opportunities to get our data into the hands of those creating applications. A quick look at the ESRI Mashup Challenge shows that there are tons of very useful apps screaming to integrate your data layers into them. You just need to make sure you provide the meat and let ESRI handle the bread and lettuce (maybe a little bacon too).

http://www.flickr.com/photos/roboppy/ / CC BY-NC-ND 2.0

Bottom Line

So the above example is “ESRI Centric”, but that doesn’t mean it isn’t valid for EVERYONE. If your online geo-content isn’t in formats or services that can easily be integrated into popular mapping APIs and libraries, your data is not going be easily used. For the consultant, if you can’t deliver you data to your clients quickly and easily, they’ll look elsewhere for services (integrating with Drupal, SharePoint, whatever). For data providers, if I can’t grab your data and throw it into my OpenLayers mapping application or Silverlight API app by cutting and pasting lines of code, I’ll probably not use your data at all. You’d better start thinking this way because the landscape has changed, work with web services or be out of work. The writing is clearly on the wall, pay attention.

It looks like the powers that be at the ESRI Developer Summit have decided a .NET ESRI/Microsoft SIG is a good idea after all. Fears (possibly all mine) that the DevSummit was changing and wouldn’t be able community anymore are tempered somewhat by this announcement. So if you work the ESRI/Microsoft angle to develop GIS applications, you can now share this fact with your brothers and sisters in code. Viva Palm Springs!

A late edition to the ESRI Developer Summit is the GeoDesign Idea Lab. This is going to be a set of lightning talks by developers showcasing how they’ve been using the concept of GeoDesign in their applications. I’ll be moderating the session with Eric Wittner of ESRI. If you’ve been telling everyone you’ve “been doing GeoDesign for years”, now is you chance to get up and show everyone how your stuff is teh sexay.

Most of what we’ve seen out of this GeoDesign has been with researchers and university types talking about concepts. Now is the time to show how developers have been in the trenches integrating disparate disciplines and bringing the results to the decision makers and the public.

Email your Lightning Talk (10 minutes maximum) abstract to geodesign_devsum@esri.com to be considered. I’ll be blogging the session in detail so this could be a great opportunity to get your GeoDesign chops out in the open. Plus since it is going on at the same time as the Business Partner Conference expect some of the marketing geeks to drop by and see what is up.

GeoDesign Idea Lab Part 1
GeoDesign Idea Lab Part 2

T.S. Elliott was doing GeoDesign before anyone else!

Google’s Public Data Explorer continues their push to integrate their simply awesome visualization APIs with searchable datasets. Think big though:

If you’re a data provider interested in becoming a part of the Public Data Explorer, contact us.

Oh S#*t!

Why is it companies want to be so disruptive when all you want to do is make maps? Says John Herlihy of Google

In three years time, desktops will be irrelevant.

Bam, game over. We’ll be working our GIS magic on Androids and iPhones. I know, I know… We’ve heard this all before from Messrs. McNealy, Ellison, but now that Google is saying it (again?) we’d be get our act together or we’ll be irrelevant. Heck old Larry bought himself a trophy on the backs of those making desktop irrelevant.

Devil is in the Details

OK, so what does this talk really mean. We all know damn well we’ll be using our desktops to perform GIS analysis for years to come. Consuming data is what Google is talking about. Unlike Sun or Oracle, isn’t focused on content creation, but enabling people to discover and use it. The operative word is mining for data. Google is talking about the world (who most of the time is searching for information on their mobile devices) performing a search and then visualizing that information. Yep, you are thinking what I’m thinking, that is GIS in a nutshell, visualizing information. Content creation will still be performed on desktops around the world, but it will remain a niche marketplace. The “magic” we’ve been taking about will happen on mobile clients.

Cloud/Servers/SaaS

So right, ESRI is “cloud ready”. Everyone is either developing a cloud plan of action or executing it. We want to host our data with providers who scale as the need arises. We want to pay monthly (or hourly) for these services and not have those annoying maintenance agreements. But wait, where are we down Google’s path of mobile nirvana?

We’ve got some efforts into web visualization from Geocommons (who is probably at the forefront of web browser vizualization), but alas their client is flash based so it isn’t really designed for mobile apps. You can bet your last Amazon gift certificate that they’ve been working damn hard on one. ESRI has also moved into this space with two feed. Their Map a Map service is right up that Geocommons alley. I keep waiting for the the announcement that they’ll link Make a Map to their ArcGIS Online service and bam, ESRI users have a way to visualize their data online. But like Geocommons, their client is Flash (see a trend here) and isn’t usable on most mobile clients. Change is coming though. We’ll see their effort at the ESRI Developer Summit realized with their ArcGIS for iPhone API. Right to the front of the line goes ESRI with consuming geo-content on mobile devices.

Of course there are hundreds of others competing in this space. Cloudmade (I think this month they are mobile), SimpleGeo, Twitter (with their GeoAPI), and countless others.

Back to the Future

Yea so go about your business. Desktop Geo isn’t going anywhere. But clearly if you don’t provide mobile versions of your web apps you are going to be as irrelevant as every stupid MapGuide 6.5 Active X junk mapping site out there. All yawls standardizing on ESRI’s Flex API better be planning because Google is spot on, we’ll be visiting your website working with your data on our mobile devices. Good data not accessible on mobile is going to be devalued completely.

It’s dead Jim!

Platial Moving Forward

OK, so the news is grim. Platial is now essentially finished, at least how we knew it before. They’ve turned off their service and now everything is essentially a download link. This means if you’ve been using Platial (or at least used them in a previous life) and want to get your data, you need to act now. Platial has “donated” your data to Geocommons where it now resides under a creative commons license. If you want to keep track of Platial user “poopypants” contributions, thankfully Geocommons has archived it.

Ay, ay, ay, no me gusta KML

Widgets

I found it interesting that Di-Ann followed up with this:

If you’re looking for a new map widget, Google MyMaps offers one. It is not two-way, meaning your users cannot YET contribute but it is a very easy way to map and share maps.

The Google is supreme in this space. And users have embraced Google’s My Maps over other competing services. Google has innovated so quickly in this space and that you can perform simple spatial queries shows that their speed of innovation is going to take out a ton more start ups. Why share your data with someone’s free data portal when you can do the same with Google’s My Maps, visualize with their APIs and oh by the way, indexed by Google’s spiders.

In the Noise

I went back and searched though my blog to see if I ever posted on Platial before. I found it interesting in that I never mentioned them before. It wasn’t because I didn’t know who they were or because I had never seen them. I guess they just didn’t go far enough down the “professional GIS” hold that I’ve got myself into. I wonder if that was the same for many others. Platial covered quite a scope and in the end maybe their efforts were spread too thin, money dried up and well Google sat on them.

Selling ads on top of spatial data does not make a workable business model. I think that either you are too niche and can’t get enough eyeballs or you are too far down the food chain and the real revenue is going to Google, Apple and Microsoft. The pyramid scheme ensures that the last person gets nothing and those developing these websites are funneling so much back up the food chain that they can’t survive.

The Good News

So… depressing isn’t it? Nah, it is just how the world works. Those who started early sometimes never make it to the finish line. Consider yourself lucky that you don’t have millions to lend to these companies. 😉

On a serious note though, what Platial was trying to do, share your data with widget, is a live and well. You see thousands of Google My Maps on so many websites without even thinking about it anymore. Heck, when Platial started, did you think it would be possible embedding an ESRI map on your blog?

Cloud Ready

Well I’m sure you’ve all heard the news. ESRI is now an Amazon Independent Software Vendor.

This means of course that we’ll see some ArcGIS Server in the Amazon Cloud very soon. In fact if you are an ESRI ELA user, you can take advantage of this right now using one of the pre-built AWS AMIs. Licensing still hasn’t been outlined by ESRI, which is probably why the ELA is required, but it seems like we could be close to hourly ArcGIS Server instances by next year. The AMI isn’t anything special, just a Windows Server AMI with ArcGIS Server at this point.

The WeoGeo booth was right next to Amazon (or maybe Amazon was right next to WeoGeo, hmmm) and there seemed to be some traffic and lots of questions. Answers weren’t that concrete from what I heard and Amazon looked rushed into being there, but it did appear people made an effort to seek them out and talk about GIS in AWS. At this point ESRI and Amazon is so early in the public relationship that we’ll have to wait for the BPC/DevSummit or most likely the International UC to get the real details.

The Plenary

OK so ESRI in the cloud didn’t knock your socks off, the inevitability of the whole thing at this point seemed to make many feel like it was anticlimactic. Tough world we live in.

Jack’s plenary talk was as always razor sharp on what ESRI is doing for their Federal customers and as always sets the stage for the year. As I alluded to early, the phrase “Cloud Ready” is something we’ll be hearing a ton about with ArcGIS 10. This means a couple things, first off it integrates with other cloud services with the REST API (something many have already been doing for years), second they’ve got this Amazon AWS AMI which you can license to run a full ArcGIS Server (without any scaling of course) in Amazon’s cloud and third I think it means that ESRI’s web services are going to essentially make even private or internal clouds “GIS Ready” (that’s my term in the spirit of Cloud Ready).

I think the Plenary was well received by the crowd, but they seemed quiet. I’d probably feel the same way if Mother Nature dumped a ton of snow on me for a couple weeks. Some interesting take-aways from the talk is ESRI’s focus on private clouds, which I think aligns very well with the FedUC crowd. Their focus on mobile was very apparent and I think at this point every reference to a Windows Mobile device has been removed from Jack’s slides and replaced with an iPhone. ESRI’s focus on web services means that they can transition to mobile devices with their mobile APIs (Ah, here is the iPhone API ready to work).

Jack focused on the large picture architecture of ArcGIS 10 and then it came for others on the ESRI team to come out and demo. We saw a good overview of ESRI’s ArcGIS Online map services. This world Topo map ESRI has been working on is really special. The cartography just catches your eye and that it goes down to 1:1000k 1:1k (off by a little scale factor there) scale in large cities really makes me want to use it instead of street map services. ArcGIS Explorer Online is a really slick Silverlight app that seems to emulate much of the ArcGIS Explorer (except 3D of course), which might be a good general GIS web services browser for ESRI users. They keep hiding the URL so so I can’t share it, but it was something like http://explorerweb.arcgis.com or similar. We’ll see it soon enough I guess.

It’s About Servers

Then the most surreal part of the whole FedUC occurred. John Calkins ran over his overview of ArcGIS Desktop 10 as he always does. If you’ve never seen John give this talk, you can view one here. John as always did a really good job and some of the refinement of ArcGIS Desktop 10 is simply amazing. The editing environment, threaded geoprocessing and symbology improvements really puts ArcGIS Desktop way beyond anything any other GIS vendor is doing. But what caught me off guard was the crowd’s reaction to it. As I said earlier, the crowd seemed tired and not into things, but during the Desktop demo I heard some things that really amazed me.

• “Why isn’t this demo in Flash (or Silverlight)?”
• “Why isn’t he using an iPhone to do this?”
• “Do people still use ArcMap?”

Here was a crowd that I thought would eat Desktop alive because they spend all day in it and many just didn’t care anymore. (Note: I don’t have super hearing so I could only listen to those in front or behind me) Could we finally be at a big shift in mentality where we are breaking out of these large legacy desktop clients and toward lightweight mobile and web clients for analysis? Are users finally listening to our “web is where the magic happens” talks and taking it to heart? Not sure, but it was interesting.

Now before everyone declares desktop GIS dead, lets be realistic here. Content creation tools are still not developed on mobile devices or web clients to the point were you can get the accuracy you need so for many users Desktop is still a required element and will probably be for decades to come. But I do think that average users of GIS, even those institutionalized in the federal service, are ready for this mobile, crowdsourcing future that we are just about to enjoy.

Crowdsourcing? ESRI?

Yep, Jack talked quite a bit about VGI (Volunteered Geographic Information) which of course is a term used to describe crowdsourcing/neogeogrpahy/participatory GIS or whatever else is the term of the hour. Dave Smith did a really good job of summarizing crowdsouricng and ESRI on his blog so I’d like to point you there from some reading. ESRI has put thought into ArcGIS’ place in VGI and how users will want to get information in and out. I think as ArcGIS 10 progresses we’ll see much more on this and how ESRI users can edit things such as OpenStreetMap directly from their ArcMap clients. I think the International UC should show us much more detail on how this is going to all work.

On the Floor of the Expo

We at WeoGeo of course were on the floor showing what we are doing with ArcGIS and the cloud but so were many others. Amazon was there of course as I said. GeoEye was but DigitalGlobe wasn’t. NAVTEQ and DeLorme were, but TeleAtlas wasn’t. I saw friends at VoyagerGIS and Arc2Earth (who was at the New Light Technologies booth) were there showing their latest products. Ran into Stu Rich at PenBayMedia showing off some of their very impressive building interior modeling and of course everyone else from SAIC to lone GIS professionals who stopped by to say hello.

2010 in the ESRI Community

So as always the FedUC kicks off the ESRI year. We’ll see much more at the BPC and DevSummit next month, but the message is simple. ArcGIS 10 will interact with “the cloud” no matter what that term means to you. The more I see with ArcGIS 10, the more I can see why they named it 10 rather than 9.4. It really is a break from what ESRI was doing in the past on both the Desktop and the Server. ArcGIS 10 should arrive early Summer (not to jinx anything of course), probably before the International UC so we can all give it a test run before we show up in San Diego.

I hadn’t been to a FedUC in more years than I can recall. It was really great to see how much this conference has grown and how many more people are interested in geospatial technology as well as how people have embraced the concept of web services, web clients and mobile GIS as more than just a display tool. Should be a very exciting year.

Well I’m leaving warm sunny Arizona for some crazy reason to head off to the 2010 ESRI Federal User Conference in “balmy” Washington D.C. I haven’t been to the FedUC in more years than I care to share, but I’m excited to go this year. I’ll of course be hanging at the WeoGeo booth most of the day showing the cool ArcGIS Desktop integration we’ve been working on and going to as many talks as I can squeeze in.

I’m also going to see what is going on with this #geoglobaldomination stuff the Mid-Atlantic folks seem so keen on repeating every tweet. As with most things, east coasters seem to make a big deal about everything so this is their chance to impress me. Heck, they even schedule #geoglobaldomination, so it must be good.

“Shall we play a game?”

Metadata

When you see the word metadata I’m sure you begin to sweat. You get that lump in your throat and suppressed memories bubble to the surface (none of which are good).

Now it isn’t hard to think about why, metadata as we’ve been exposed to is just not human readable and thus barely human usable. Working in the government sector as a consultant exposed me to the worst two words that any DoD consultant can be exposed to; “metadata required”.

We deal with four letter acronyms all the time right? FGDC Even the website is built on Plone which of course feels more like Ivy League research project than the traditional SharePoint website we’d all expect from a government website. One should be scared navigating it and trying to find information. Anyway what about metadata as we’ve been utilizing it (FGDC or ISO) is just so painful?

Machine Readable vs. Human Readable

So FGDC or ISO metadata is complex, but there could be good reasons for this. They both try and address every conceivable possibility that might need describing in geo-data. If both were primarily designed for allowing servers to talk with each other, I’m not sure any of us would have any problem with it (nor would we really be looking for it). But servers rarely read and write metadata on their own without human interaction. Thus the reality of the situation is we poor humans have to ingest and parse metadata regularly.

Yikes

Well this brings me to what I see as the biggest problem with metadata. It is almost always in XML format. Now don’t get me wrong, XML does have its purpose. In fact I could list probably thousands of times that XML is the right answer. Sometimes it works and works well, other times you end up with a whole bunch of brackets and text that blends together. With a good eye you can parse out what you need, but there is so much noise there that it almost feels like a “Where’s Waldo” exercise. But XML does do a good job of organizing data for machines, but it doesn’t do it in ways that are easily readable.

What Human Readable Metadata Should Focus on

So some person sends you a dataset for a project you are working on. There are some questions you want answered before you commit to using the dataset:

1. Who is responsible for the dataset?
2. What is the dataset representing?
3. When was it created?
4. Where are its extents (projection, datum, etc)?
5. How was it created?
6. Why was it created?

The problem with metadata today is those questions are hard to parse out of metadata. If you know what to search for you might be able to find it relatively quickly, but the simple fact is that if I want to see the those answers above for a dataset, they should be exposed to me first.

Metadata Style Sheets

One way people have tried to make FGDC metadata (and ISO to some extent) more readable is through the use of style sheets. Many ESRI users are exposed to this inside their ArcCatalog. That drop-down list that lets you choose different ways of viewing the metadata is a style sheet selector. This means that you can take that ugly XML metadata and parse it out in ways that are easier to read. I’ve not seen much in the way of usability improvements on this front. At WeoGeo we offer human readable metadata on our dataset information pages. Others are doing it as well, but there is really no standard as to how this should be organized.

So Who Cares About FGDC/ISO?

Honestly you really shouldn’t care. You should care though about getting information describing the data you are working with. I think most of the issue with both metadata standards is that they are just too hard to input data into and too hard to get out the relevant information. Committee designed standards such as these always end up being way too much for real world use. We need to make sure we get the who, what, when, where, how and why of the dataset and to do this we need to look at the geo-data creation tools and how they help us input metadata. Data creators should have an easy time filling out those 6 things about their data. The issues are in the weeds of the metadata standards. But out on the fringes of the metadata requires, creation tools (such ArcCatalog) can help us manage things. Databases should be tracking who created the data (their name/address/etc), when it was last modified, any look up tables, aliases for field names, links to additional information and anything else that is being used for that dataset. Not having to track all that down gives the creator of the data enough focus to make the who, what, when, where, how and why so much better than they would if they had to enter everything.

And on the display end of things, I’d like to see UI experts work at creating better human readable metadata style sheets that hide the details that you don’t need to see at first glance and expose what we as uses of data need at first glance. It is easy enough to expand the details “below the fold” of a metadata page.

What Now?

It is up to all of us. We are stuck with the metadata standards so changing them at this point isn’t feasible. At WeoGeo we’re committed to working on bringing complex/detailed FGDC/ISO metadata to users in easy to digest methods. What I’d like to hear though is from others trying to crack this same nut and see if we can collaborate on this more and in this age of NSDIs still have usable metadata for people to make decisions.

Yes FINALLY! I can’t tell you how frustrating it has been for me since the day Google Maps arrived. I always wanted to hold down the shift key (like every other modern mapping API) and draw a box to zoom in. With Google you had to use your mouse wheel and really who has a mouse wheel anymore with our notebooks and touch mice. Something had to be done.

Enter Google Maps Labs. You should now see that little green beaker in the upper right hand corner of your Google Maps screen.

Clicking on that icon you are presented with some new features:

A two of note:

Drag ‘n’ Zoom

Now this was the one I was most excited about until I saw its implementation.

See that little square below the zoom bar? You are supposed to click on that if you want to zoom in. You can’t do what is completely obvious to everyone, hold down the shift key. I wouldn’t mind if they had both, but not adding the shift key to enable is totally baffling. But even worse, you can’t use the escape key to get out of the Drag ‘n’ Zoom. You have to move your mouse all the way back over to the left and turn it off.

Aerial Imagery

I don’t agree with what Google calls this because I’m sure there is “Aerial Imagery” in their “Satellite” images, but they’ve got 50 Billion in cash and I’m under water on my mortgage. So what do I know? Anyway this is the Google oblique imagery we’ve read about. It is only available in some small areas, but we can now see them outside of the Google Maps API. When you zoom to an area that has supported oblique imagery, you’ll see the new oblique aerials button that turns it on. You can use the Drag ‘n’ Zoom to quickly get into an area you wish.

The Others

The rest aren’t in my opinion that newsworthy but address probably small needs of users. I think this is a good way for Google to get some new features into Maps quicker than their normal release schedule. I just wish they’d get on board with existing UI and naming conventions.

via GigaOM