AWS Start-Up Challenge and a Look Back at WeoGeo

I saw this week that the 2011 Amazon AWS Start-Up Challenge is running again.  It is hard to believe that it has been 4 years since WeoGeo was a finalist in the 2007 AWS Start-Up Challenge.  We finished runner up to Ooyala with Justin.tv.

I blogged about the challenge back then (I wasn’t working for WeoGeo yet) and “we”[ref]GIS Professionals[/ref] were worried about things like the Esri Web ADF, not cloud based computing.  As WeoGeo gets ready for a busy fall, I’m glad the decisions that were made by the team back in 2006 are able to support the amazing places we are going to be going in 2011.  Click on that link above to see a video of Paul Bissett showcasing WeoGeo to the voters.

The business model evolves, but that the fundamental desire to put people and their place together in a single, accessible, framework is still the driving goal of WeoGeo. Yeah we up up up through the clouds….

ArcGIS as “Cloud Bundle” on Amazon’s Cloud

ESRI has finally come out with a cloud based ArcGIS Server.  You can lease ArcGIS Server on Amazon’s EC2 for one year with ArcGIS Server, SQL Server 2008 Workgroup, ArcGIS Desktop (but only to administer ArcGIS Server) and one year of support.  Prices aren’t listed, but I’ve heard it isn’t that competitive to your existing licensing.  Plus don’t forget with that Large Windows Instance of EC2, you’ll spend $5,000 hosting the thing for that year minimum.

For those who already have ArcGIS Server licenses, here is your package.  For this you get everything listed on that webpage plus:

The ArcGIS for Amazon Jumpstart package is a consulting service. It includes one week of up to four (4) days of onsite consulting.

Hmm, guessing that most will still just roll your own rather than pay ESRI for consulting.  Nice option for those who need help, but onsite ESRI consulting won’t be cheap.

Back to the Cloud Bundle.  What is good about this?  First off I’m glad to see ESRI finally start to publicly address demand for ArcGIS Server on Amazon EC2.  They’ve broken their traditional maintenance based approached to licensing which is something I think we are all very happy about and they’ve automated the process with an AMI ready to go.

ESRI sales team ready to hit the road and sell the cloud.

What is still lacking?  While this is a step into ArcGIS as SaaS, it still requires you to go through your local ESRI office.  This will mean that large ESRI customers will get great breaks and those who are smaller or new will pay list prices.  The cloud is supposed to bring equity, but the traditional sales model of ESRI plays favorites.  Windows only instance of this AMI is also problematic.  The cost of a large windows instance of EC2 is going to offset all the benefits of the a 1 year license.  Of course ESRI doesn’t support Fedora or CentOS so until they do most are probably going to not scale up ArcGIS in Amazon.

I see nothing about ESRI helping with backing up these Ec2 instances and how that is going to work.  These EC2 instances can crash (hello ArcGIS!) and just disappear.  If that happens you lose EVERYTHING.  Basically this is a GIS infrastructure play and it is up to the user of their AMI to handle this.  That said, one ArcGIS license isn’t enough to do redundancy (though maybe the terms allow this).  Basically you are paying to use a single, slow (compared to typical servers) hardware with no methods to back up your services.  YIKES!

That said, we were all lead to expect nothing on this until later this year so this means ESRI is addressing users concerns.  There is still lots of darkness on this such as does it include updates (would a 10.o users automatically get 10.1 when it comes out?) and service packs (I guess one could just apply them, but that puts the maintenance of the AMI on the user not ESRI).  I’m looking into this because I really want to run an ArcGIS Server on the Amazon cloud, but not pay for a typical license.  Plus actually having ESRI support for EC2 is probably worth it in the long run either way (this service or roll your own).  The ESRI UC should be very cloudy (yea I used a cloud pun).

Mix a little ESRI and cloud and who knows what you’ll get

Here comes the anti-database “movement”

I’ve been seeing more and more articles like this one from Computerworld about abandoning SQL databases.

The meet-up in San Francisco last month had a whiff of revolution about it, like a latter-day techie version of the American Patriots planning the Boston Tea Party. The inaugural get-together of the burgeoning NoSQL community crammed 150 attendees into a meeting room at CBS Interactive. Like the Patriots, who rebelled against Britain’s heavy taxes, NoSQLers came to share how they had overthrown the tyranny of slow, expensive relational databases in favor of more efficient and cheaper ways of managing data.

NoSQLers?  Oh boy are we going to be in for it when they hear how critical databases are for the geospatial industry.  To me this “revolution” sounds more like a backlash against the traditional SQL DBA who doesn’t want to change in the face of “Web 2.0″.  Of course it is very easy to move to a new data storage platform when you either have a ton of money or no product yet.  While I do think technology such as Google’s BigTable and Amazon’s SimpleDB as an inevitable course for many web applications, wholesale abandonment of SQL and databases such as Oracle/SQL Server/PostgreSQL is absurd.

No-SQL Patriots dump RDBMS without a care to the implications...

No-SQL "Patriots" dump RDBMS without a care to the implications...

ArcGIS Map Server Cache in Amazon S3

Amazon S3 Tile Cache loads quicker for everyone

Amazon S3 Tile Cache loads quicker for everyone

We’ve been loading our tile caches in Amazon S3 for quite some time now and it looks like others are trying to take advantage of the service. I’ve come to the conclusion that using S3 for your tile cache makes a ton of sense for performance and reliability issues. Our S3 tile caches are more reliable than our file servers in serving up the tiles and do it so much faster. Is anyone else noticing the benefits of S3 or has is been problematic for you?

Where is Metered Pricing for Internal Mapping Applications?

So I’ve got a (theoretical) simple internal website for a client that basically puts pushpins for their locations on a map. Since this is on their intranet we’ll have to pay approximately $10,000 to Google or Microsoft to license their products internally. This small company doesn’t have the kind of money (especially in this economy) to put down on such an application. Essentially spending $5-$10K before any coding has taken place is not going to get anywhere and the project is dropped. 

Why is it that both Google and Microsoft are stuck in archaic licensing agreements? The logical way to price these services is a metered service. Much like Amazon AWS works, you’ll pay for what you use (with per-session costs decreasing as usage goes up). It would be monitored so I could see what kind of traffic I’m generating with the service and I could be billed monthly via credit card. We’ve been using Amazon S3 and EC2 with great results and it is very easy to justify the low initial costs and still be able to scale to larger applications if warranted.

How do you get Scrooge McDuck to part with money to pay API licensing costs to Google or Microsoft?

How do you convince Scrooge McDuck to part with money to pay API licensing costs to Google or Microsoft?

The other problem is that if I’ve got to spend that kind of money, I’m going to be developing larger internal applications than I would externally. You can’t do a simple internal mashup if you have to pay large enterprise licensing costs. I was told by one potential client who wanted a Virtual Earth application that “For that price we might as well buy another ArcGIS Server license”. For someone to say that it is cheaper to roll out ArcGIS Server for a simple mashup really puts it all into perspective and if you are going to develop with ArcGIS Server, you’ll probably have a larger application than a “simple” Virtual Earth map.

So I’m pleading with Microsoft and Google to revisit their map API licensing and move toward more of an Amazon AWS type service that will help get their mapping tools into more places than today. Everyone wants a Google or Microsoft map on their intranet website, but the current licensing is killing projects before they can start.

The GeoMonkey is a huge supporter of Amazon Web Services

The GeoMonkey is a huge supporter of Amazon Web Services

Amazon brings Windows (and SQL Server) to the cloud

The Amazon Web Services Blog says that Amazon will be bringing Microsoft Windows to EC2 this fall.

The 32 and 64 bit versions of Windows Server will be available and will be able to use all existing EC2 features such as Elastic IP Addresses, Availability Zones, and the Elastic Block Store. You’ll be able to call any of the other Amazon Web Services from your application. You will, for example, be able to use the Amazon Simple Queue Service to glue cross-platform applications together.

This is on the heels of the Oracle/Amazon EC2 release from a couple weeks ago.  Now that the tools are here, we’ll have to see how well they are adopted by corporate IT administrators who aren’t always open to giving up control of their servers to others.

Mr. Gates saw the value of the cloud early on

Mr. Gates saw the value of the cloud early on

Oracle enters the cloud (MySQL Enterprise too)

Oracle and Amazon today announced that Oracle would be offering some of their products inside Amazon’s EC2 cloud.

The Oracle Database 11gOracle Fusion Middleware, and Oracle Enterprise Manager can now be licensed to run in the cloud on Amazon EC2. Customers can even use their existing software licenses with no additional license fees. 

While I see nothing specifically about Oracle Spatial, I assume is can be licensed as well on the cloud.  The benefit to everyone is outside of licensing costs, the ability to launch the Oracle AMIs on EC2 and be up and running in no time.  That plus the scaleability of EC2 (and thus Oracle) means that you don’t have to worry about hardware limitations with your applications.  RSP Architects uses SQL Server as our database of choice, and while I would have been able to run Oracle in a virtual server, I no longer have to worry about hardware constraints to our development.  Just license (which of course I realize is a problem for some people) and start loading the database.  I’m anxious to see how ArcGIS connects to Oracle Spatial on EC2 and what kind of performance I can expect.

Cloudzilla carries Oracle onshore

Cloudzilla could be unbeatable with Oracle in his hands

Now for those who want to avoid Oracle, MySQL Enterprise as well in the Amazon Cloud.

Amazon Web Services launches SimpleDB Beta

Amazon SimpleDB is on its way and it may just change how you use databases with your web applications (or even desktop apps). SimpleDB is a web service that leverages Amazon S3 and Amazon EC2 to store, process and query datasets. Currently most if not all of us use a RDBMS such as SQL Server, Oracle, MySQL or PostgreSQL to store our data, but that requires hardware and most of the time a DBA to administer database.

Can you imagine a spatial component to Amazon SimpleDB and how you’d integrate it with your workflows? My spine is tingling just thinking about the possibilities.

You can learn more about the details of SimpleDB here.

Update: The more I think about this, the more I realize how disruptive SimpleDB will be. It was designed to be used with web applications and will be able to scale with them easily. You just can’t do that on your own. I was writing about AWS back in June and how its ability to scale could help users provide services that only the largest companies can afford.

Vote for WeoGeo on the Amazon Start-Up Challenge

Time for all us geospatial folks to get together and vote for one of our own. Take a look at the videos for all the Amazon Start-Up Challenge and vote for your favorite (which I assume for most of us will be WeoGeo). Vote early and vote often. It would be great to see a geospatial entry get some good press.

Read more of this post