Trying to unlock transit data

August 24, 2009 17 Comments

Something that has been bothering me somewhat these last few months is transit data and how we as a community can have access to it.  Here in Phoenix, we are unlucky enough not to have access to Google Transit, Valley Metro is locked in to the 1990′s with their system and you should see how bad the mobile version is.  So many of us here have been trying to free the Valley Metro data to get it incorporated into OpenStreetMap or even Google Transit.  Since I’m not blogging about how successful that effort has been, you can guess that we are stuck back in the last decade still.

But that isn’t the whole story right?  Just getting transit data into Google Transit is great for users of Google services, but not good for the community at large.  Google has been really good at getting data locked up behind government bureaucracy, but they’ve done nothing to help free this data beyond using their own APIs.  But at the same time, is this Google’s responsibility?  Google seems to say they try to get people to share it, but it isn’t their job.  I think I tend to agree with them.  If an organization is willing to share data with Google, why aren’t they willing to share it with the public?  That is where these organizations fall down and it would appear where public transit is a huge problem.

Public transit organizations are looking at protecting their data from being used by others.  It isn’t everyone, but it does send a message that if you as a member of the riding public want to access public data, you need to do so under an application approved by that transit organization.  I find it amazing that DMCA can be applied to public data and it should send chills to anyone who wants to use data they as a taxpayer are funding.  Google is able to negotiate deals because they have the money and the eyeballs that transit organizations want.  I just don’t like the path that we seem headed down where if I want to access public data that I helped create… I need to do so using a commercial API and possibly have to pay a vendor (paying twice is a tax on a tax) the right to use what should be free.

Stay away from that slope, you wont like where it will lead!

Keep away from that slope, you won't like where it will lead!

Filed under GIS Tagged with , , , , ,

Sharing Cartography

April 21, 2009 62 Comments

There are tons of ways to store spatial data, but sharing cartography is much harder. ESRI seems to be using their new Layer Package “format” to keep the ESRI layer file with the datasets. Outside of that I don’t really see much in the way for cartography support with data layers (other than the odd ESRI LYR file offered up on FTP sites). In a way, there is much to like about ESRI’s decision to use essentially a sidecar file for storing symbology and maybe there is a more open method of sharing this information.

Most of the time when you download data, you just get the raw file format (usually shapefiles). You then drag your new downloaded dataset to your viewer (possibly ArcView, maybe uDig) and then you proceed to try and figure out how to color the map up (of course there is no metadata so you can’t figure out what field you should symbolize on) in an attempt to match that PDF of a scanned map that someone gave you.  Wouldn’t it be better if there was included symbology that allowed you to view that data layer the way the author intended?

Well as I said, ESRI’s Layer Package format does this.  The Layer file is of course a problem because you need ArcObjects to read it.  But in an ESRI world, that isn’t too much of a problem.  Outside of an ESRI world, maybe the MapServer Mapfile might be the best way to get symbology information in (heck you can write out a Mapfile from ESRI, let alone all those open source clients).  Some have said KML is that format.  I disagree because it doesn’t lend itself to storing attribute information (maybe there are ways, but I still question if KML should be used this way).  At least KML does store simple symbology with the dataset.

Ron Lake offers up SLD/SE as an option, but acknowleges that it really has no traction.  I really haven’t seen SLD used much beyond some GeoServer documentation and I’ve not seen anyone try and use SLD for storing cartography in desktop clients (NOTE: According to comments uDig does support SLD, but not be default which is a good starting point).  Sean Gillies thinks CSS is the best option but again we run into the issues of desktop clients not supporting it and probably won’t (though Sean gets GeoMonkey cred for continuing to use the term “GeoWeb” every chance he gets).

What about Mapnik?  It does a superb job at rendering great maps and Andrew Turner says there is a project that can convert CSS into Mapnik.  Now that might be a workflow.  CSS is very easy to edit and and work with and  Mapnik is great at rendering cartography (and could easily be supported by desktop clients).  CSS gets stored as a sidecar file to the dataset and the rich cartography can be read with Mapnik to render the maps as the author wishes the maps to look.  Maybe Sean is on to something after all…

If we are going to go out on the GeoWeb wall of death, well need a sidecar.

If we are going to go out on the GeoWeb wall of death, we'll need a sidecar.

Filed under GIS Tagged with , , , , , , , , , , , , , , , ,

Don’t Give Away the Farm!

May 19, 2008 38 Comments

So Google and ESRI will allow indexing of ArcGIS Server services by Google (and anyone who crawls the web).  So what does that mean moving forward.  It really isn’t big news if you think about it because this “feature” (service description) is already enabled in ArcGIS Server 9.3.  The problem with this is no one has really been thinking about what this means for everyone.  If you expose these metadata pages to the Google Bot, you’ll be opening up your services to the world. 

Now don’t get me wrong, this is a great thing.  As a user of data, I’m always wishing that I could search datasets using Google rather than the haphazard way we do it today (luck has more to do with it than anything), but data providers will lose control of their datasets.  Plus how do you monetize your information in such a world?

There are two types of organizations on the Internet, those who want to work with Google and those who don’t.  A great example of a company that isn’t allowing Google to index their pages (well beyond the Whitehouse) is Facebook.  You never see Facebook results on the web and that is probably why they have been so successful.  Giving away your data to Google can be dangerous to your business model. 


Make sure you hoard your geospatial data

That said, I’d like for everyone to expose all their data on the Google so I can perform my job much easier. Maybe I’ll be surprised and there will be millions of new datasets available from ESRI servers by the end of the year, but I’m not holding my breath.

Filed under GIS Tagged with , ,

The GIS Interchange File

May 5, 2008 36 Comments

All too often we have to request people resend datasets to each other because they get blocked by email, one important file gets left off or systems just don’t recognize a file type. I’ve run into a problem today where a company FTP site is rejecting a shapefile because it doesn’t recognize the .shp, .shx, .dbf extensions. I thought I could get around by zipping the data, but it appears to scan the zip file for extension types. So the “solution” was to zip the shapefile, change its extension to .doc and tell the recipient that they need to change the extension back to .zip.

This kind of stuff happens way too often. Personal Geodatabases have the problem of the .mdb extension that is rejected outright by most email systems and other formats aren’t readily usable by folks systems. The “old days” were easy because we all used coverages and shared them via the .e00 format that was almost always acceptable by everyone. Amazing how we take such steps back over time and you’d think data sharing would be easier than it was in 1995.

How do you folks share data? KML, GML, Etch A Sketch, e00, zip, web services, etc?

Update: Jason Birch has some ideas about using SQLite as an interchange format. Well worth the read.

Filed under GIS Tagged with , , , ,