Memorial Day 2010

As with most other Americans, I’m off today sitting in my pool, drinking beer and eating hot dogs. I hope everyone is enjoying the day and remembering all those men and women who gave their lives in support of freedom.

ArcGIS Server 10 and the Compact Cache Storage Format

Is there anything harder than moving a tile cache from one computer to another? It is hard enough when you have fiber running between two servers, but when you think about copying it up to the cloud? Forget about it!

So now you get a choice when creating your cache in ArcGIS Server 10 – exploded or compact.

Exploded is the same format you worked with in earlier versions of ArcGIS Server, in which each tile is stored as a single file.

Compact is the new format. It actually does not zip or compress the tiles in any way; rather, it groups the tiles together in large files called bundles. A single bundle can hold up to about 16,000 tiles. The result is a cache with dozens or hundreds of files, instead of thousands or millions. This speeds copying immensely, and is especially useful in workflows where you use a staging server to create tiles, then move the tiles to a production server.

One thing to remember, if you are reading the ArcGIS Server tile cache outside of ESRI clients (the web APIs and ArcGIS clients) you’ll want to make sure you use the exploded tile cache, otherwise you won’t be able to read the bundles. I haven’t had any opportunity to test performance of these compact tile caches, but there is no reason to suspect that it will be noticeable either way. The big deal will be on uploading them from your local data store to the server/cloud where you want to consume them.

I’m very torn on this, the benefits to the compact tile cache are clear – uploading tile caches takes forever. But yet another proprietary tile service seems a bit much. I suppose you can use ESRI’s APIs to read them but sometimes that just isn’t an option (or a want).

5 Years of Spatially Adjusted

Well how about that? It was just pointed out to me that I missed my 5 year anniversary with this blog.

1,688 posts and 13,450 comments later it is amazing to think about how far we all have come. Thanks for reading and lets see what amazing stuff happens in the next 5 years. I can’t think of a better way to celebrate than having Eduard Khil sing his epic Soviet song, “I’m so happy to be finally back home”.

ArcGIS Explorer Online Beta is Released

So hopefully not lost in the ArcGIS.com release is ESRI’s latest GIS tool, ArcGIS Explorer. The one sentence explanation from ESRI about what it is:

ArcGIS Explorer Online is similar in appearance to the desktop version of ArcGIS Explorer, and has some of the same capabilities, but it?s a lighter weight version that works in a browser, and is built using Microsoft Silverlight.

ArcGIS Explorer Online is a Silverlight based browser application

So what we have here is a cross platform GIS analysis platform. I’ve liked what I’ve seen from ArcGIS Explorer Online and I think it is a solid start to platform agnostic client applications. It requires Silverlight to run, but I’m able to use it flawlessly on my MacBook Pro laptop. It mimics the Microsoft UI standards with the ribbon, but I’ll be frank that I’m beginning to like the ribbon interface. Of course if you hate it, you’ll probably hate the app.

Presentations are important to ESRI and ArcGIS Explorer Online

When you start working with the app, the first thing that will catch your eye is what ESRI has stuck in the upper left of the toolbar. If I had never seen ArcGIS Explorer Online before, I’d assume that ESRI would have put “Add Data” or “Search” or even “Basemap” in that prime spot. But as you can see to the left, Presentation gets the first place everyone looks. I’m not sure if this is by design but ESRI thinks we’ll be making lots of presentations with ArcGIS Online. I had see Bernie Szukalski use the presentation mode with great success at the ESRI Developer and Business Partner conferences so I get the power. I’d just assume presentations would be a ribbon on its own and not on the primary one.

The basemap button doesn’t disappoint. If there is one thing ESRI has gotten right over the years, it is the freely available basemaps they’ve offered up through ArcGIS Online ArcGIS.com. These are all available here including the Bing layers, ESRI’s Imagery and Streets, the wonderful ESRI Topographic map and of course now the OpenStreetMap layer.

There are tons of basemaps available including OpenStreetMap

Adding content is where I think ArcGIS Explorer Online will eventually shine, but for right now it is limited to only web services that are available on ArcGIS.com or ArcGIS Server services. I’ve been told OGC support will be coming soon, but as of today you can only add ESRI web services or consume services from ArcGIS.com.

As you’d expect, you can save your maps to ArcGIS.com and choose to save them privately, share with a group or share with the world. There is no ability to share a permalink as you might with Google Maps, but saving to ArcGIS.com and then sharing that URL is probably the method for now. ESRI also has provided some “Featured Maps” (I’m assuming this comes from featured maps on ArcGIS.com) that you can get started with.

You can grab existing “Featured Maps” that ESRI provides to get started.

I like the start of ArcGIS Explorer Online. Building it on Silverlight seems like a smart move as it runs flawlessly on any computer I used. The biggest limitation I see right now is the lack of OGC support (WMS, WFS, CSW and of course KML) but ESRI has told me that is coming down the road. I also would have liked to see a permalink feature to share quickly with friends maps I create, but I’m guessing ArcGIS.com is the driving force here so I might as well get used to sharing ArcGIS.com links. I’m not sure how I could share my ArcGIS Server services with ArcGIS Explorer Online (embed or link that I can put in a blog post), but hopefully that part of the story will be set by the ESRI UC.

Oracle Spatial Locks Your Data Away

That is the only conclusion I can come to out of this wacky ruling. A huge warning to everyone, if you put your data in Oracle Spatial, you may never get it back out because it ceases to be data and turns into software.

On a side note, clearly you don’t want me to live in your town our county. I grew up in Orange County and now live in Tempe. Possibly it is I who causes data to be locked away.

ArcGIS.com Beta Goes Live

Over the weekend ESRI pushed out a public release of the ArcGIS.com website (still in beta). They’ve brought along the data from ArcGIS Online (what there was) and have wrapped it around an absolutely beautiful website. It is a little light on content, but I’m sure at ArcGIS 10, ESRI expects all of us to start sharing our data as Layer Packages on ArcGIS.com.

ArcGIS.com is now live

The Gallery page is fairly well laid out, but I’m not sure if it will scale when the billions of ESRI users start integrating this into their workflows.

The Gallery page of ArcGIS.com

You’ve got 3 basic sections of the Gallery; Maps, Web Apps and Mobile Apps. When you mouse over a map in the Maps section, you get an overview of the map and some very basic metadata on it. Pages appear at the bottom and while this works very well for the 2 pages that exist now, I’m sure it will get messy very quickly. But don’t worry, ESRI has a search!

Searching ArcGIS.com

“Gulf Oil” results in a ton of results across the whole ArcGIS.com. The search works well as long as there is never an oil spill in another gulf. There is no way I can see to say I only want results in the Gulf of Mexico and not the Persian Gulf. It’s the typical Google (though with pretty screenshots) search method that I’m just not sure works very well with spatial data. All these maps on ESRI’s ArcGIS.com have extents, why not use that for a search?

The next section of Gallery is Web Apps where users can register their own ArcGIS Server APIs websites. Before we get there there is a huge problem with the navigation of ArcGIS.com. You can’t generate URLs for many of the sections. Thus if you want to see the Web Apps section of ArcGIS.com, you’ll need to go to ArcGIS.com and navigate yourself. Enjoy…

Web Apps section of ArcGIS.com

Some of these you’ve seen for years, others are now. You can either roll your own on your own servers, or create a map on ArcGIS.com (which I’ll get into soon). Again, its got the same issues as the Maps section. Will it scale when the method of discovery doesn’t lend itself to maps? We’ll see.

Lastly is the Mobile Apps section which is essentially the same as the Web Apps, but with only Mobile Apps. It will be interesting to see if ESRI will allow Web and Mobile Apps that use ESRI Web Services or ArcGIS Server, but not their visualization APIs (for example using OpenLayers) into ArcGIS.com.

Map

Map allows you to visualize content in ArcGIS.com or on your own map web services. It isn’t ArcGIS Explorer Web, but appears to be a JavaScript subset of it.

ArcGIS.com Maps using OpenStreetMap basemap

Now I do like ArcGIS.com Maps very much. I think the folks at ESRI did a very good job with it and it is generally very intuitive. There is a huge problem with though though, zero support for open standards. Want to add an OGC web services (even one running on ArcGIS Server)? Can’t do it. I have to assume they’ll add this in soon, but until it happens ArcGIS.com Maps is just half baked. (this is also a problem with ArcGIS Explorer Web)

Open Standards? Not around here…

So anyway, assuming you live in a total ESRI world (I guess we do don’t we?) author you map in ArcGIS.com Map and save it. Then you can choose do you want to share with the world (using that Gallery stuff above) or with a small group (or keep it private). This brings us to the last section of ArcGIS.com

Groups

The groups section is where groups (get it?) can collaborate on their maps together. I’ve not set up a group yet, but browsing through the public ones I can see lots of them called “test”. As I’ve stated before above, you can see how the navigation becomes unusable after a couple pages have been added. No one in their right mind will browse these maps via this interface making discovery very difficult. Long tail need not apply at ArcGIS.com.

Yikes, 90 pages. Time to futz with the search engine.

Conclusions

So what do I think? ArcGIS.com is a very good start. It really looks and feels modern. Runs snappy and is quite intuitive. My problems with it are few, but to me their are just killers. No support for any open standards. As long as you use ArcGIS Server Services and Layer Packages, you can share. If you don’t have ArcGIS then this just isn’t going to be the place for you. Even if you use ArcGIS Server and only share WMS services, you can’t partake in ESRI’s ArcGIS.com.

The other issue I have with it is that I don’t think the interface scales well. If we are all going to be resorting to using search to find anything, it makes it very difficult to just discover things. A perfect example is how I see people using Google Earth with the Wikipedia layer on. They just navigate the globe discovering content on a map.

Lastly I have one huge plea to ESRI. **PLEASE ADD AN UPTIME INDICATOR TO WEB AND MOBILE APPS! **We can’t use any of these services if we don’t know how often they are down. Star ratings are useless so feel free to drop that right now and do something like FGDC does. Reliability lends credibility and I can’t imagine ESRI spending all this money on ArcGIS.com only to see it fail like GeographyNetwork.

That said, ArcGIS.com is a good start and could become the premier method of visualizing geo-content on the web. I’m not sure the sharing aspect will gain much traction since it doesn’t support open standards, but ArcGIS.com Maps will be well used by just about everyone.

ArcGIS as “Cloud Bundle” on Amazon’s Cloud

ESRI has finally come out with a cloud based ArcGIS Server. You can lease ArcGIS Server on Amazon’s EC2 for one year with ArcGIS Server, SQL Server 2008 Workgroup, ArcGIS Desktop (but only to administer ArcGIS Server) and one year of support. Prices aren’t listed, but I’ve heard it isn’t that competitive to your existing licensing. Plus don’t forget with that Large Windows Instance of EC2, you’ll spend $5,000 hosting the thing for that year minimum.

For those who already have ArcGIS Server licenses, here is your package. For this you get everything listed on that webpage plus:

The ArcGIS for Amazon Jumpstart package is a consulting service. It includes one week of up to four (4) days of onsite consulting.

Hmm, guessing that most will still just roll your own rather than pay ESRI for consulting. Nice option for those who need help, but onsite ESRI consulting won’t be cheap.

Back to the Cloud Bundle. What is good about this? First off I’m glad to see ESRI finally start to publicly address demand for ArcGIS Server on Amazon EC2. They’ve broken their traditional maintenance based approached to licensing which is something I think we are all very happy about and they’ve automated the process with an AMI ready to go.

ESRI sales team ready to hit the road and sell the cloud.

What is still lacking? While this is a step into ArcGIS as SaaS, it still requires you to go through your local ESRI office. This will mean that large ESRI customers will get great breaks and those who are smaller or new will pay list prices. The cloud is supposed to bring equity, but the traditional sales model of ESRI plays favorites. Windows only instance of this AMI is also problematic. The cost of a large windows instance of EC2 is going to offset all the benefits of the a 1 year license. Of course ESRI doesn’t support Fedora or CentOS so until they do most are probably going to not scale up ArcGIS in Amazon.

I see nothing about ESRI helping with backing up these Ec2 instances and how that is going to work. These EC2 instances can crash (hello ArcGIS!) and just disappear. If that happens you lose EVERYTHING. Basically this is a GIS infrastructure play and it is up to the user of their AMI to handle this. That said, one ArcGIS license isn’t enough to do redundancy (though maybe the terms allow this). Basically you are paying to use a single, slow (compared to typical servers) hardware with no methods to back up your services. YIKES!

That said, we were all lead to expect nothing on this until later this year so this means ESRI is addressing users concerns. There is still lots of darkness on this such as does it include updates (would a 10.o users automatically get 10.1 when it comes out?) and service packs (I guess one could just apply them, but that puts the maintenance of the AMI on the user not ESRI). I’m looking into this because I really want to run an ArcGIS Server on the Amazon cloud, but not pay for a typical license. Plus actually having ESRI support for EC2 is probably worth it in the long run either way (this service or roll your own). The ESRI UC should be very cloudy (yea I used a cloud pun).

Mix a little ESRI and cloud and who knows what you’ll get…