GDAL 1.9 Brings Esri FGDB and Google Fusion Tables Support

Good news for users of GDAL/OGR:

The GDAL/OGR team is pleased to announce the release of GDAL/OGR 1.9.0.
This is a major new release including the following major new features:

* New GDAL drivers: ACE2, CTG, E00GRID, ECRGTOC, GRASSASCIIGrid,
GTA, NGSGEOID, SNODAS, WebP, ZMap
* New OGR drivers: ARCGEN, CouchDB, DWG, EDIGEO, ESRI FileGDB, Geomedia,
Google Fusion Tables, IDRISI, MDB, SEGUKOOA, SEGY, SVG, XLS
* Significantly improved drivers: NetCDF
* Encoding support for shapefile/dbf (#882)
* RFC 35: Delete, reorder and alter field definitions of OGR layers
* RFC 37: Add mechanism to provide user data to CPLErrorHandler (#4295)
* gdalsrsinfo: new supported utility to report SRS in various form
(supercedes testepsg)

Some nice new formats in there.  How does it all work?  Paolo Corti takes a look and says, “Brilliant!”.

GeoDesign Summit 2012 — A New Direction

In all fairness to my previous post, I want to share some GeoDesign links. The feedback I’m getting from those who attended is that it has become an education type initiative, rather than working toward changing how we actually do work. I guess bottom up change works sometimes, but these kids graduating with “GeoDesign emphasis” have no chance at changing how established companies are doing business. So here you go if you want to try to figure out what was discussed:

The GeoDesign Little Red Book is ready to teach a generation of students what won't work in the commercial sector.

Let’s Call Esri’s ArcGIS Online What It Is — A Spade

So the Esri GeoDesign Summit is in full session, or at least we know it is because at least two people showed up.  Matt (one of the lucky two) showcases the latest Esri initiatives being demonstrated at Esri’s shindig.

Bernie Szukalski, product strategist and technical evangelist at Esri, spoke today at the GeoDesign Summit about ArcGIS Online initiatives and coming capabilities. Web mapping has morphed from sharing maps and geospatial information to a geospatial content management system that supports collaboration. The new iteration allows for the publication and sharing with others, as well as the access to rich global base data through cloud services.

So yea, da cloud roxxorz!  We’ve all tried to figure out what ArcGIS Online is (beyond the kitchen sink of Esri’s cloud), but this week it’s content management.

Maps can be shared with others by making them publicly available, sharing a link or embedding in a website or blog. Additional content can be found via the gallery where maps are vetted by the community, with ratings and comments. These maps contain documentation with details regarding the source, providing metadata that allows you to understand how they were created. There is also flexibility in how these intelligent maps can be accessed, with options to open in your desktop, open in Explorer Online, and in the map viewer.

If you dropped off the turnip truck, you probably like most of the paragraph.  Sounds like a nice open shared world, where everyone wins.  But the last sentence leads to the truth about ArcGIS Online and their vision of content management.  Flexibility in the sense that if you’ve standardized on Esri’s suite of ArcGIS products, you can share their proprietary formats.  Things like ArcGIS Server, File Geodatabases, Layer files and the rest are not formats we can share with the world.

C'mon people now, Smile on your brother; Ev'rybody get together, Try and love Esri right now

Lets look at it this way, SharePoint is Microsoft’s Content Management System.  Imagine if Microsoft only allowed you to upload Microsoft file formats.  Crazy right?  But that is what ArcGIS Online is.  Sure you can upload shapefiles, but those really are so limited you can’t really store data in them.  There is a reason why nobody uses DBF anymore.  No, you have to use Layer Packages, MXDs, etc to get any “value” out of ArcGIS Online.  It is a little better on the server-side, Esri supports WMS (probably so they can check off OGC support in some contracts), but nothing else.  ArcGIS Online is what it is, an Esri Content Management System that lets you share Esri files with other Esri users.

I don’t fault Esri for creating such a product, they feel there is money to be made doing this.  But let’s not pretend it is a GIS content management system because it just doesn’t support open standards let alone other formats such as TAB, DWG or whatever Intergraph is doing these days.  It is an Esri Content Management System.

But what does that really mean?  Basically Esri’s ArcGIS Online is Google My Maps, but with $10,000 client software.  Creating a map to share with Esri’s online APIs doesn’t make it content management.  There is no geneology of data, no lifecycle to the product.  Just some simple polygons or pushpins on a map that at its core isn’t what customers want.  The biggest reason why Esri is pushing ArcGIS Online so much is that Google Earth Builder is a direct play toward some vision that Esri has to where GIS may go in 2012/2013.

The problem with both solutions is that they don’t actually manage your data that goes into your products (the pushpin maps you share during GIS Day 2012).  The important data is still strewn across hard drives and servers in your organization just hoping that it will never get lost.   That doesn’t sound like progress to me and the focus is not on workflows but some mythical federal contract that the big boys are fighting over.

Cutting edge maps have pushpins on them.

 

UPDATE: Looks like I missed Sean McGinnis’ take on ArcGIS.com a couple weeks ago.

Licensing on the GeoWeb

The GeoWeb is easy, right?

The GeoWeb is easy, right?

OK, love or hate the term GeoWeb, it does reflect the reality of our workflows today.  We are taking our applications off of the desktop and running them on the web, we are combining data silos right inside the browser and we are giving tools to that used to be reserved for technicians on UNIX workstations to users via their mouse.  It is truly a new way of working, but I see it running directly in a wall.

Licensing

Here we are at the tip of the iceberg changing the world (maybe a self-serving statement), but we’ve got a chain around our necks limiting our potential.  Google and Microsoft’s (among others) APIs are sold the same way IBM sold software before there was the world wide web, large companies can cut great deals, smaller users are left paying full price because we don’t matter.  ESRI’s ArcGIS Desktop and Server licensing doesn’t reflect how users are using the applications in the real world (sure, allowing editing on ArcGIS Server Standard instead of Advanced is a step, but it is just one in a long list of problems with the licensing model).  Don’t even get me started on Oracle’s licensing.  Arbitrary levels of licensing that have no real world basis are killing innovation and requiring consumers of the services to look elsewhere or limit what they can do with technology.  I’m not advocating abandoning any of these companies here because there are great business cases to use their software, but their customers are not able to continue business as usual.

Wont someone please think of the users?

Won't someone please think of the users?

Just last Friday, I was talking to a client who was paying for ArcInfo because one hour a month he needed a function that it handled.  The rest of the time ArcView was good enough for him.  This isn’t just limited to ESRI, most software companies sell software this way and users are expect to pick up the cost just to get some functionality that some committee, in a dark room in their software design center, determined that only “power users” would need is crazy.  Sure there are ways to get around most of these limitations using other software, but all these companies are doing is pushing their customers away.

So what do we need here?  Google offers their products as SaaS and so does Microsoft.  Why does this make sense for “Office” applications and not Geospatial software?  Now these efforts of course don’t replace Microsoft Office and that isn’t their mission (well at least Microsoft’s).  But what do they do is allow users to extend their collaboration further than the office conference room.  Geospatial software is well set up to take advantage of services.  Pay for what you use and spend the savings on tools that benefit the end users and not tools that you’d never need.

So I’m not exactly expecting a revolution here in the next year, but unless companies start thinking about the realities of how users are using their software or APIs, we are going to have to look elsewhere or create our own tools.  Given what the licensing costs these days, there is money to get that done.  Personally I’d rather just use an off the shelf solution and pass those savings on to my clients and get back to building great applications for them

The GeoMonkey only wants what is right...

The GeoMonkey only wants what is right

2009 ESRI Developer Summit Plenary Session

The wireless connection here is less that ideal, but I’m going to try and live blog the DevSummit Plenary.  Just refresh the page for updates (but keep in mind I’m not the fastest typer in the world).  You can also follow some discussion at my twitter account:  http://twitter.com/cageyjames

8:32 AM: Video starts showing the languages ESRI supports.  Keeping my eye out for Avenue.  Nope looks like Avenue gets no love.

8:33 AM: Jack opens the DevSummit with a welcome.

8:37 AM: Jim McKinney takes over and explains the purpose of the Developer Summit and why everyone is here.  There are over 1,100 developers at the DevSummit this year from 41 countries (much larger than the BPC).  Tech Sessions will be recorded and placed on the ESRI Resource Centers.

8:43 AM: Jim talks about the road from 9.3 to 9.4.  9.3.1 ships “May”.  9.4 will go beta around the 2009 UC.

8:45 AM: Scott Morehouse takes the stage and talks about the philosophy behind ArcGIS.  Very high level stuff.

9:05 AM: Jim gets back on stage and the demos are about to begin.

9:07 AM: Demo on how 9.3.1 adds functionality to tune map services automatically.  No longer do you have to figure out what layers are slowing down your maps.  Run the analyze tool and get errors and warnings to show what is slowing your services down.  The results lists tells you what you have to do to fix the service, some as easy as right clicking and saying fixing.  After you fix the errors, you get can preview the service inside ArcMap and get see the speed of the service after fixing.  Thus you no longer have to author in ArcMap and publish in ArcCatalog.  You can do this directly in ArcMap.  In fact the tool can be used in just desktop applications to improve map display performance.  All very slick and will be well received.

9:14 AM: Next demo is for the Flex API.  Amazing that 1 year ago it was barely featured at the DevSummit and now it is probably the default API for ESRI.  We drop into some Flex Developer IDE demo and code.  Seems all simple, but I’m no Flex developer.  Demo shows importing data (xls file) directly into the client side application.  I think this gets into unhooking these APIs from ArcGIS Server.  Adobe is here at the DevSummit and is having a get together.  Quite a change from the past.

9:20 AM: Oh my, an actual Java demo.  Talk about Java and excitement at ESRI.  I used to understand how Java Devs worked, but this demo is just way over my head.  Lets just say that ESRI continues to support Java and leave it at that.

9:33 AM: ArcGIS Online is next up.  ESRI seems to refer to this as the cloud so if you here that mentioned here, think ArcGIS Online.  Sud and Jeremy are showing the ArcGIS Online 9.3.1 demo.  Remember now you can upload those Layer Package files (zipped layer files and data).  It seems that ESRI wants you to package your data using the layer package format and then upload it to their system.  I can’t but help think of the Geography Network when I see this, but maybe users are willing to use this much simpler interface.  One thing that it does do is you can limit your content to certain users, rather than the whole cloud Internet.  These users are members of groups so collaboration can occur on ArcGIS Online.  I can’t figure out the business model here for ESRI, but I suppose it ties back into Desktop and Explorer.  Not only can you add data to ArcGIS Online, but you can create hosted maps from your uploaded data.  Reading between the lines here, we see ESRI uncoupling ArcGIS Server from workflows.  You can share data and maps with the world without having to purchase an ArcGIS Server license.  The demo showed the JavaScript API, but I’m sure any ESRI REST API SDKs can be used with ArcGIS Online and without an ESRI ArcGIS Server license.  ArcGIS Online goes public beta after ArcGIS 9.3.1 arrives in May.

9:47 AM: Break

10:17 AM: Back and video showing ESRI mapping implementations

10:18 AM: Jim starts off with the ESRI Resource Centers.  Jim Barry takes the stage to talk about ESRI’s community efforts.  The Resource Centers were updated a couple months ago so most of this stuff on stage is old hat.  The story is that unlike the old EDN pages is ESRI is investing time and effort into the website.  Still no mention of a true Wiki, but maybe one day.  The new template galleries are interesting and hopefully they’ll do much better than the Geoprocessing and Data Model pages on the support site.

10:29 AM: Art Haddad introduces ArcGIS Silverlight API for Microsoft Silverlight/WPF.  Art turns the stage sliver (get it Silverlight?).  He demos the ArcGIS Resource Center Silverlight page.  The demos really show off the power of Silverlight.  We can argue about Silverlight’s install base vs Flash, but it really is compelling.  The getting started demo of Art’s shows how quickly you can get started with Silverlight.  Art’s demo is by far the smoothest we’ve seen today and really highlights where ESRI is going (Microsoft Integration).  The cluster feature method works well and you get some really great maps without much code.  Also slick is the ability to use the same code with WPF.  Silverlight API is not coupled to the ArcGIS release schedule (like the other APIs) so get involved with the beta.

10:43 AM: ArcGIS Explorer 900 is next up.  The new Office ribbon interface makes the application fit in very well with other windows applications.  Bern shows the reading of layer packages being loaded into AGX.  So now you get the fancy cartography generated in ArcMap, inside AGX.  2D Maps are now part of AGX.  3D globes are great for some mapping, but nothing beats 2D for getting the message across.  Microsoft Virtual Earth is now part of AGX so you get much better looking base maps than before.  Yet another new feature, there is a new presentation mode to allow you to use AGX for presentations.  This is hard to describe, but think about using AGX for your presentations rather than PowerPoint.  You can navigate back and forth through the “slides” and then interact with the maps.  Larry Young went into some customization of AGX with an application configuration file.  This means you can streamline AGX to meet the needs of your users, dropping out the tools that users might not need.  This is done with an Application Configure tool that is included with the AGX 900 download.

10:55 AM:  Bill Moreland talks about Python and ArcGIS.    The Python demo uses pulp-or with ESRI’s Geoprocessing model.  In the past, Python hasn’t been getting as much love as it should at the DevSummit so seeing this on the Plenary stage is really a change.  It is good to see the Geoprocessing Resource Center is getting some traction with shareing scripts.

11:05 AM: ArcGIS Desktop at 9.4 is next up.  Some key goals were to simplify common tasks, streamline workflows and improve ability to share work.    Catalog is now integrated into ArcMap (like Toolbox), much faster map drawing, search from ArcMap, better cartography (isn’t that always there?), charting and reporting, 3D GIS (3D editing) and Asynchronous Geoprocessing (no longer locking up Desktop why your analysis runs).  New ADFs at 9.4, extensions can now be developed with .NET and Java (drop in extensions).  Side by side deployments at 9.4, thus you can run 9.3.1 and 9.4 at the same time on the same machine.  Map automation can now be performed with Python in ArcGIS Desktop.  Say goodbye to DS Map Book because now you can create PDF export using Python scripts, much like we did in the old days with ArcPlot.  With a few lines of code you can replicate all the functionality of DS Map Book.  You can now add geoprocessing tools to any of the toolbars (including your Python scripts).  Python is also now integrated into Python.  Yep and interactive Python window right inside ArcMap.  So intellisense GIS analysis with Python.  Brilliant!

11:17 AM:  Editing Demo – Sorry I can’t bring my self to blog about editing.  If you do lots of editing with ArcGIS Desktop, you’ll want to learn more because the editing tools are really leaping over where they are at 9.3.x.

11:30 AM: Jim is back with the news that you can now edit in all ArcGIS Server ADFs and APIs (including the RESTful ones).  Now on to ArcGIS Mobile, the story here is that it doesn’t seem to integrate with what Art Haddad is doing with WPF and Silverlight.  It is a shame that ArcGIS Mobile is still treated as a different product than the ArcGIS API for Microsoft Silverlight/WPF.  I don’t like the idea that the ArcGIS Mobile team is doing WPF work and the ArcGIS Server team is doing it as well.  I want to write code once and deploy on Server or Mobile, not maintain two different apps.

11:45 AM:  Break for lunch.

ArcGIS 9.2 Service Pack 5 is available

For all those who can’t resist installing the latest service pack for ArcGIS, here it is:

ArcGIS 9.2 Service Pack 5
ArcSDE 9.2 Service Pack 5
ArcIMS 9.2 Service Pack 5
ArcGIS Image Server Service Pack 5

In case you can’t remember if you care or not, check out the Service Pack 5 announcement.


Freakin’ sweet!

The Closing Q&A

Yikes – I just saw this draft from yesterday that I didn’t get posted so pardon the untimeliness of it.

The closing Q&A session was over lunch and was pretty calm. A couple issues were raised that I thought I would mention.

VB6 will not be supported at ArcGIS 9.4 and you should really move to .NET.

The File Geodatabase SDK (or whatever it will be called) has been sidelined and they still need to figure out what it will be. The simple solution everyone would like is for them to just work with the GDAL team and get something written into GDAL/OGR (they did at least mention GDAL on stage). I’m just not sure what the problem is on this and why they don’t want greater use of the File Geodatabase (could be why I’m a consultant and they make money selling software).

Dave Bouwman asked them to stop overselling ArcGIS Server. His comment was well received by everyone out there and those on stage all agreed they need to do a better job giving a realistic picture of what it can do and how quickly you can deploy applications.

There was some worry about the future of ArcIMS. Of course the Web ADF works with IMS, but the picture is pretty clear that you need to start seriously working at migrating to ArcGIS Server. There were quite a bit of ArcIMS folks looking at the RESTful API and the JavaScript API. There is a natural progression from AXL to JSON I believe and being able to see the JSON will better help them understand how it all works. The point is though that you probably need to move on as no new features will be incorporated into ArcIMS.


It was the last chance to enjoy an ESRI Squishee

So wait, what was that Flex API for ArcGIS Server?

While the buzz at the DevSummit was for the REST API and the JavaScript API, the Flex API sure got noticed. Folks seemed disappointed that they missed the session on the Flex API. Well some bloggers have already been posting their thoughts on the Flex API and it is well received.

I’ve been telling folks I’ve got serious reservations about using any development environment created by “Adobe”, but this Flex API stuff is really compelling and worth a look.


ArcGIS Chefs have another service to cook with

Reflection on the 2008 ESRI Developer Summit

Well I think most would agree, the 2008 DevSummit was one of the best.  There was tons of new stuff to learn about, much more attendees, more ESRI staff, better layout of the conference (the Community Center was particularly good) and better session (and more of them).  So what did I take away from the conference?

ArcGIS Platform

So underneath it all, what has changed.  Well first ESRI has really focused on bug fixes.  I know we’ve all heard this before, but I think the new crash reporting dialog will give better feedback to ESRI and internally they’ve caught many bugs that might not have been caught without the crash reporter.  In addition ESRI is using Coverity to help uncover hidden bugs in the code (read some of these case studies, very interesting stuff).  I was told that they found stuff that has been hidden for years in the code that would have caused problems, but for one reason or another never was discovered.  I think it is safe to say the 9.3 code base will be as bug free as anything they’ve ever released (hold for joke) and given how short this beta period is I think they are confident that they’ve delivered on this.

The focus at 9.3 is stability, performance and security.  Those are 3 areas I know have been a great concern for most ESRI users/developers and the examples that we were given between 9.2 and 9.3 showed great performance increases (I can’t comment on stability until I’ve worked with 9.3 for a while).  The new security improvements aren’t revolutionary, but address the specific concerns users have had with the product (specifically check out the security presentation on EDN from the DevSummit for the details).

ArcGIS Server

Well there are tons of new "exciting" features with 9.3 as we’ve all heard.  First of the REST API is the real deal.  The REST API can server up tiles to Google Maps and Virtual Earth (assuming you use the "web Mercator" projection) and the ability to use ArcGIS Server with Yahoo! pipes really opens the possibilities of taking ArcGIS Server and moving it into areas that we’ve not been working in.  Providing results from Geoprocessing is as easy as appending f=kmz to the URL (don’t you just love RESTful services?).  The JavaScript API is based on Dojo so you’ve got some power in there to make some really interesting JavaScript applications.  The Google Maps and Virtual Earth (2D and 3D support) extenders allow you to bring your ArcGIS Server services right into consumer mapping products.  The JavaScript API is hosted by ESRI and in the Akamai cloud so it should be very stable anywhere in the world.  We’ll be seeing a ton of new applications out there based on the JavaScript API in the next year, that I’m sure of. 

Now don’t forget about the .NET Web ADF (didn’t hit any Java sessions this year).  It is now what ESRI is calling a "hybrid" model meaning that there is both server and client side stuff going on (rather than the total server side stuff at 9.2).  The key new feature is ASP.NET AJAX, but it is still very compatible with 9.2 projects (usually just change a line or two of code and your old projects should still work).  The core controls are now scriptable with the ASP.NET AJAX libraries so you can do a ton more on the client than you did before.  The JavaScript API in the Web ADF is totally different than the other one (the Server JavaScript API for use with RESTful) so your code may have to be customized between the two versions if you jump between the REST API and the Web ADF.  The Task Framework is much improved and you can now build them using User Contols.  In addition they are releasing tasks into the code gallery on the ESRI Resource Center for Server (I’ll talk more about the Resource Centers later) so you’ll be able to see what ESRI has done and create your own modifications.  The documentation in the Resource Center is so much better than what was available in 9.3.  The examples are great and the explanations are detailed and well written.  The performance of the Web ADF at 9.3 has increased at least 100% if not more.  If you ever blended two data sources (one tiled, one dynamic) you know that it takes the dynamic one time to match the tile scheme that the tile scheme already existing.  Now each resource has its own tiling scheme and the layers load much faster.  The Web ADF (and obviously the JavaScript API above) will be "uncoupled" from the ArcGIS Server release schedule.  This means that you won’t have to wait years for new features to be implemented.  Silverlight 2.0 support will probably happen way before 9.4 arrives which would have us all complaining down the road I’m sure.  Dave Bouwman has a great write-up on the details of the .NET session so head over to his blog to read up on his thoughts.

ArcGIS Explorer

I’ve already posted on the new features in the 480 release due in May and the 600/700 release due by the end of the year in my Plenary session post, but I’ll list some of the new features in Explorer that caught my eye.  First off 480 will increase performance (multi-threaded), direct connect to SDE, GPX support, GeoRSS support and improved task frameworks and popups (the bubbles).  Build 600 has the new Microsoft "ribbon" interface and looks great.  From a usability standpoint, the information you are working with gets presented right to you and not hidden by interfaces.  You will also be able to finally view the maps in 2D mode.  I think this will be a boon to organizations who are using AGX as a decision making tool.  Ease of use goes a long way.  The "enhanced" ArcGIS Explorer SDK will allow you to embed AGX inside your applications.  I asked how ESRI would charge for this SDK and they are still thinking about it (will the SDK be free and the deployments cost, will the SDK cost and deployments be free, or will everything be free).

ESRI Resource Centers

New at 9.3 is the ESRI Resource Centers.  You’ve already been looking at the first one for quite some time (the ArcGIS Explorer Resource Center) and the ones for ArcGIS Server, Desktop, Engine, Image Server, Mobile, IMS and Geodatabase are currently available for those in the 9.3 beta program.  These are help centers where you can get support, online help, code samples, interactive SDKs and other resources that you can use with developing (or even using) the ArcGIS Platform.  The forums are due to be re-launched based on the Beta forums (which means you’ll be able to subscribe to a forum topic via RSS).  There will be many new blogs available from teams that haven’t blogged yet and there *might* be community aspects introduced as well.  How this all interacts with the EDN site I have no idea. 

Issues?

The one thing that scares me and Dave Bouwman did bring it up at the closing session is overselling what you can do with the new REST API and JavaScript API.  Have sales staff running around that you can create "rich" JavaScript applications "consuming" ArcGIS Server services using only 12 lines of code is going to put many of us in a bind.  JavaScript is easy to pick up, but that doesn’t mean you’ll be adding complex geoprocessing to your Google Maps mashup with one line.  The speed that you can develop has increased, but the complexity will still be there.   The JavaScript API will increase your productivity, no doubt.  But telling everyone all you need is 12 lines of code will result in disappointment. 

What now?

Well I’ve got both ArcGIS Server and ArcGIS Desktop installed on my laptop and they seem very stable.  Moving forward I think we’ll jump with both feet into the RESTful API and the JavaScript APIs.  I think users will want to get their services published via the REST API as soon as possible so Google can start indexing them.  What a great way for organizations who want to share their data with the community, just publish and let Google index your services.  The ArcGIS Services Explorer is going to be a great tool to learn what is available out there.  I had quite a few ArcIMS developers say that they can finally feel comfortable working with ArcGIS Server.  The .NET and Java Web ADFs were too much for them and they were usually used to working with simple HTML pages.  Compare the speed of JSON vs the speed of sending XML (AXL) requests to the server and see how fast you get a response.  It really does highlight why the community at large has really moved to JSON.

So go get on the 9.3 beta, but you’ve got to hurry as 9.3 RTM could happen as early as "June".

ArcGIS 9.3 Beta

I don’t think there is anything wrong swinging by the office on the way home from the Developer Summit to pick up the 9.3 Beta disks to install tonight while I watch UCLA destroy Mississippi Valley State. My wife just doesn’t understand me but I’m happy with who I am.


Lets see, I have my .NET Sombrero beer hat and cigar. I’m ready to install the ArcGIS Beta