If there is one book I’d recommend anyone to get in our industry this is it. Way back in 2009 I wrote about the first edition:
Looking at the table of contents reveals that this should be the book for learning how to use PostGIS in your GIS applications. I’m really intersted in Chapter 13, “First look at WKT raster”.
Of course that book was on my desk for years and eventually it was updated in 2013. But that was over 7 years ago, technology changes and so has PostGIS. You can now get the long awaited 3rd Edition in the Early Access Program. I’ve started reviewing it and there is so much in there as this is going to be a significant update. PostGIS 3.x should be a big deal in it as well as PostgreSQL 12.
Well that might be a big title for this post, but I was talking with some folks over the weekend about software you’ve used or software that has really influenced your life. I think many people say Google Earth has changed how they view data, but for me it really wasn’t that impressive since Google Earth is more of a validation of what we’ve done over the years than a life changer
I thought it would be fun to look at how things have changed since then. My job is very different, I can’t remember the last time I created a map or changed cartography in a mapping product. I think one can look at that 2006 list as how I got to the point that I lived the rest of my life. So here is the updated list:
HyperCard – I just can’t stress enough how much this changed my life. The concept of a database and visualization. The scripting language on the backend, and everything that eventually become the web (buttons, forms, etc) on the front. I’d like to think that I would have learned to program a different way, but teaching myself Hypercard is exactly how I go to where I am today.
BBEdit – to this day I still use BBEdit. I think I purchased my first copy back in about 1994 and I’ve used it probably every day since then. I’m sure I’ve used every text editor. Today I use BBEdit, VS Code and of course Nano, yet I find myself in BBEdit more than anything else. I taught myself Grep using BBEdit and probably after a hypertext markup language, Grep has done more for me than just about anything. From JSON to Python, from CSS to GeoJSON, from JavaScript to Perl, I write it all right here.
Perl – I was going to put JavaScript here. I probably should have put JavaScript here. But I have to be honest, the scripting language that got me thinking about scripting was Perl. I rarely use it anymore, other than pulling some script out of a folder and running it one off. I use Python more for my scripting or JavaScript. But from the time I bought the first edition of Programming Perl I was hooked.
PostGIS – So another one I thought about. Elastic? MS Access? DBF? SQL Server? I mean what database should be the one that changed my life. It has to be PostGIS. Without it I would probably have put MySQL right here. But no, it’s PostGIS. The reason this blog was created was to learn more about PostGIS and how to get that damn thing installed on Windows Server. Some day on my newsletter I’ll write about the impact of Simple Features for SQL. From the moment in 2005 when I got PostGIS working until today, I’ve always had PostGIS running somewhere near me.
Safe FME – Sadly I don’t use FME anymore. But let me be crystal clear here. There is no better tool out there to help you manage data. I probably should find myself a copy of it and run it again. At WeoGeo we used it for everything. I’ve used it while at Architecture firms, Engineering firms, startups and in between. Data is agnostic and using a tool that is helps keep the integrity of data. Before FME I spent so much time trying to keep all the data in one format and in one projection (I was young, let me be), but when I was able to drag a reader on to a workspace, throw up a transformer and then connect that to a writer, I was hooked. FME should be standard issue for any true Geospatial data user.
Some other software that didn’t make the list but could have and I didn’t mention above? GDAL/OGR, Tippecanoe, ArcGIS, Excel, Google Earth and Photoshop. Such a personal list and one I think changes over time. I think the core of what makes me who I am is up there, but it is also up in that 2006 list too. For fun you can look at the Way Back Machine and see the comments on that blog post. I see Sean Gillies, Morten Nielsen, Brian Timoney, Steve Pousty, Bill Dollins, and others in that list.
Don’t forget to subscribe to my newsletter SpatialTau, the first edition goes out tomorrow morning. Every week on Wednesdays moving forward! Sign up below!
Bill and I have a podcast that we do almost once a month. Podcasts are a lot of fun because you can talk about things easier than writing about them. There is a free flow of ideas (or maybe garbage) out of your head and on to a mp3. One topic we talked about months ago was GIS clients. We talked about tools we use but I just happened to be listening to it last night and I realized maybe I wasn’t as honest with myself as I should have been.
GIS users, if you need a friend, get a dog.
I’m not a GIS user…
Fair point though, what is a GIS user? I think of it as someone who uses GIS software. But even that it is somewhat of a mess because one person’s GIS software is another person’s toolkit. Ignoring that issue for a second, what do I use for GIS?
GDAL/OGR
Turf.js
Elastic
PostGIS
I think that pretty much covers it. I mean there is some Shapely and some other libraries, but that short list is all I use anymore. That of course has a lot to do with my job, if I was GIS Manager at the City of Townsville I might need other tools, but that list above is pretty much it. I can’t help but think of these things as Toolkits rather than GIS software. They are all part of a deeper workflow that I use when I need to use it. The end result is never QGIS, ArcGIS, uDIG or whatever madness you use in your daily life. It is either GeoJSON or “database” (where database could mean a lot of things).
God made men. Men made proprietary software systems
This blog is about to have it’s 15th year anniversary and I can’t think of a better example of how things have changed since that moment. I also think GIS lends itself for this workflow orientated environment anyway. Ignoring the crazy ArcGIS Desktop years with wizard based GIS, mostly GIS has been scripting workflows to accomplish your needs. Fortran, AML, Python, you name it. We use these methods to not only get results but document them. In the end I think all the tools we use for GIS are Toolkits and not software. Yes, one must put a name on something, but GIS has always been about toolkits, even in proprietary workflows, and will always be this. Maybe when we check in right before I retire in 2035 we can see how we are doing with this.
Some of our goals with this project are to extend the Node.js to work with SQL Server and possibly Oracle1. Possibly even write spatial objects to PostGIS from the app2. We used Google Maps for this demo but I think the sweet spot is TileMill generated background tiles with Leaflet.js.
Now that we have our PostGIS/PostgreSQL running just about perfectly on our RedHat server, it is time to move forward with UMN Mapserver. I’m excited to see how much we can do with it and I think it will open up so much more to our products than .NET and ArcIMS ever did. As I said earlier, I want to make a front end that looks the same to the end users, whether we use ArcIMS/ArcSDE on the back end or Mapserver/PostGIS. In my previous post, I said that the GUI just wasn’t there for PostGIS, but letting it sit on a Linux server hosting our data should be great.
This should be a great week as we start playing with Mapserver/PostGIS and seeing what we can do with it.
We’ve loaded up PostgreSQL and PostGIS up on our Linux server to start playing around with it and I’ll post some thoughts I have of things so far.
Getting PostgreSQL installed wasn’t too much trouble, but PostGIS was a pain. It has to be compiled before installing. My database programmer got it working after a couple hours, but after using ArcSDE for so many years it was an eye opener. I’m sure they will get a compiled version up, but for now we had to do it ourselves. My next thought was to see if ArcCatalog could connect to it. We tried an ODBC driver and an OleDb driver but had no luck. Databases are not my strong point and while we were able to get them to connect to PostgreSQL, we couldn’t seem to connect to PostGIS. There must be something in how PostGIS handles the spatial data that these drivers can’t handle. This is somewhat of a big deal for us as most of our data is in either ArcSDE or Personal Geodatabases and Post GIS only allows loading of data via shapefiles. I was hoping to use ArcCatalog to perform the loading, I guess it is export to shapefiles and then use the shp2pgsql command. It appears that the ArcGIS Data Interoperability Extension supports PostGIS, but if you have to spend over two grand on an extension, what is the point of going to PostGIS. I’m sure we can script something, but I would have rather had the ArcCatalog option open to everyone.
So what does this mean to our development? Probably not too much except it is a strike against open source GIS. If PostGIS had a windows driver that allowed ArcCatalog access, things might be different and we could recommend it to our clients, command line isn’t a user friendly proposition. Open source GIS seems to mimic open source in general. Its getting better, but you still need command line experience to truly get value from it. Since ArcGIS 8, ESRI has really pushed the GUI for GIS giving even the most greenhorn GIS specialist commands that 10 years ago where run by very experienced GIS Analysts on UNIX.
It is very easy to criticize ESRI for their products but they have really taken the GUI to places where open source is at least 5 if not 10 years away from being. I still think that open source GIS has a place on the server side, but we need to figure out ways to get data loaded from industry standard programs such as ArcCatalog before it will start to take off.
