WNYC 2010 Census Maps

John Keefe shows how he did it.  Spoiler alert!

Using Google Fusion Tables made it super easy to manage, map and serve up a lot of data. And the FT feedback team was fantastic about responding to questions and glitches I encountered along the way.

Yup, love it!  I can even embed it here.  Great job!

http://project.wnyc.org/census-maps/2010pop.html?lat=40.7739&lon=-73.9456&zoom=12&sel=6

I’ll be blogging the opening plenary this morning. Just refresh this page as the morning goes on and get the latest news.

8:30AM: Jack takes the stage and welcomes everyone to the 2010 ESRI UC, the 30th. Jack seems quite excited and wanting to get the message out. Jack goes into his highlighting of maps created with ESRI software. I’d personally love to see some historic maps that were highlighted at ESRI UCs past and show how far we’ve come.

8:35AM: Looks like the WiFi just crashed, so much for live blogging Some idiot is probably streaming video.

8:40AM: Abu Dhabi won the “Making a Difference Award” for building a geo-infrastructure using ESRI software. They view infrastructure as similar to the human body. Not too much detail there, but it is an interesting way to look at things, plus they seem quite successful.

8:50AM: The theme of the conference is “Vision”. Jack dropped into the concept of Computational Geography (which of course became GIS). Jack ties into the convergence of technology with geography at the middle. Jack views the key drivers as mobile, crowd-sourcing and other LBS technology to create a web-based geospatial platform.

9:00AM: Jack introduced a company called CityCourced which “mobilizes citizen involvement”. So its the idea where you and your GPS mobile phone are able to feed that data back to your “geodatabase”. Nothing really new here in concept, but it appears to be in the wild. Nice that they got a great speaking slot to introduce their product. Lets see if governments jump on this before saying it will change the world. I don’t see much here about cities that are getting involved with this product (if I’m filing a report, I want it to go to my city, not just sit on CitySourced servers). How they integrate into these cities will probably make or break the service. API is here.

9:05AM: Jack goes into crowd sourcing and social media. I always wonder how the geodatabase works with both? As long as you use their APIs.

9:07AM: ArcGIS 10 is outlined; Jack says it is easy, powerful and everywhere. ArcGIS is Desktop, Mobile or Web on Cloud, Enterprise or local. ESRI is focused on what Jack called “intelligent maps”. These are basically all the aspects of GIS and loaded into a map where you can interact, query or edit the map.

9:11AM: ArcGIS 10 has hundreds of improvements which ESRI says will improve productivity. Jack says ArcGIS is open; standards based and then open API’s.

Big news, ESRI is publishing their REST Interface as an Open Standard like they did the Shapefile.

9:14AM: ArcGIS Mobile is finally beginning to take off. iPhone, Windows Phone (LMAO), Android and of course the old ArcPad. This all ties back into Jack’s hope that citizens will use ESRI technology. Putting ArcGIS in the hands of everyone.

9:16AM: At ArcGIS 10, ArcGIS imagery support is very improved. From what I hear this could be the biggest raster release that ESRI has ever done. I’m not involved with imagery anymore, but it all looks pretty sweet.

9:20AM: Content is key and ArcGIS 10 brings ArcGIS.com into the desktop. The community basemap program got its own video. We’ve been talking about a national map, and it appears it took ESRI and their investment in the technology to make it happen.

Bing Maps is also free to all ESRI users including ArcGIS Server.

9:25AM: Bernie Szukalski gives his demo of the ArcGIS.com data using ArcGIS Explorer Online (or whatever they call it these days). The community basemap is just beautiful. I wonder if the City of Tempe takes part? The World Imagery basemap has really taken off. I’ve been using it instead of the Bing Aerial map because I think it is higher quality and it is free for anyone to use. ArcGIS.com has lots of great content including OpenStreetMap.

9:30AM: Break time – back at 10am for a look at ArcGIS 10.

9:30AM: Here comes the cloud. Jack says cloud computing will change the way we work, on-demand (of course ESRI licensing isn’t on-demand, but that is a story for another day). Really the story is extending existing deployments right now in 2010.

9:33AM: “The Next Big Step” — ArcGIS.com ESRI’s cloud storage solution today. What is interesting is that hosting is “coming soon”. ArcGIS.com will host your data simply and probably extremely cheaply. Glad I’m not a GIS hosting company. ArcGIS.com is a network of distributed services — as apps or maps. What is nice about ArcGIS.com is that the services are accessible via their APIs.

9:37AM: Bernie gets working again and shows the ArcGIS.com. The UI is nice, but it uses the weird star rating system. I think ESRI should just put a thumbs up button there. How does one rate a map with 5 choices? They say it is either one star or five stars. Just put a thumbs up button there and move on. Making a map with ArcGIS.com is simple and easy. Tags are weird as well, does tag searching work? Just use a search box to searchthe description.

Basically you discover data in ArcGIS.com and then add it to your basemap. You can add content via ArcGIS Online, the web or any ArcGIS Server (no WMS yet). Basically you end up creating a mashup you can share with others. These maps are also easily consumed in ArcGIS 10. ArcGIS Online is built into ArcGIS 10, so you can consume web services without leaving ArcGIS 10.

Bernie also demo’d the ArcGIS for iOS on the iPad. As I said earlier, the UI is just wonderful. Take note devs, this is how maps should be consumed on mobile devices.

9:50AM: Jack says ArcGIS opens GIS for everyone – clearly ESRI is focused on getting as many people using ArcGIS 10 as possible and easy to use clients is how it will be done. Will ArcGIS 10 transform GIS? We’ll see, but it sure will affect how everyone works with ArcGIS. He asks everyone to go to ideas.arcgis.com to help direct where ESRI goes with their software. I’m sure the site is therapeutic for many users

10:00AM: Break time!

10:30AM: ArcGIS 10 will be the focus for the next 90 minutes. John Calkins is out hitting on the point again that ArcGIS 10 is a complete system for geographic information. It looks like the focus is on abstract demos rather than just giving us the beef on ArcGIS 10 Desktop/Mobile/Server.

10:35AM: First up is productivity; ArcGIS 10 productivity highlights. Of course they do some sort of weird government demo of infrastructure. Searching is demonstrated, finding both datasets and symbol sets. Optimized Map layers allow you to load data into a group that allows you to pan around without refreshing of the data. The new editing tools are really slick, the template editing stuff really changes how you work, leveraging rules set in our geodatabase. The data validator extension is really useful because it finds and documents the issues in your data.

10:45AM: ArcGIS Mobile on a tablet running Windows XP? Sexy! Well now we are on to an ArcGIS Viewer for Flex and of course we see the edits made on the old busted Windows XP Tablet. What is nice is the same workflow that happens on the desktop comes across to the mobile. Same methods of editing on Desktop – Mobile – Browser.

10:52AM: Map Automation; ArcGIS 10 brings python into Desktop/Server. ArcPy could change how GIS Analysts work with data analysis. Bob Pool a GIS Manager from Washington State is giving the Python demo himself. Migrating AML to Python scripts is up first (Migrating AML’s in 2010?). Python editor is now built into ArcGIS 10. No one will be using VBA to automate ArcGIS Desktop moving forward.

11:00AM: Imagery demo is up now. ArcGIS Server can now render a mosaic on the fly. I’m out of my league here with imagery analysis, but imagery analysis inside a browser is impressive. The image analysis tools are nice because they can fix bad looking imagery. All done on-the-fly and with no modifications to the original data. What was the company called again that did image analysis? Erdas? They have no place in ArcGIS 10 anymore.

11:13AM: Network analysis improvements are the next demo. Network analyst has always been a hugely valuable. There has been much in the ArcLogistics area, but ArcGIS 10 doesn’t miss upgrades as well. The new Location-Allocation tool gives some great site selection tools to users. And that segue into an ArcLogistics demo. I love the ArcLogistics online (with Business Analyist Online, these two tools really just kick ass) and the simple web based tools are going to be analysis to anyone.

11:22AM: Apple iOS demos are next. Business Analyst Online (BAO) will be available soon for the iPhone/iPad for free. I love BAO in the web browser, but the iPad/iPhone app puts that power into your iOS device. Again, like ArcGIS for iOS, the UI is simple and easy to use. ArcGIS for iOS (which was demo’d earlier) was show again. Under featured content you’ll find a Rupert’s Places to go in San Diego available on your iPhone or iPad. Very nice!

10:26AM: 3D has been a big focus for ESRI at ArcGIS 10. You can now edit or perform analysis in 3D in ArcGIS 10. The problem I have with this is that you still need those ArcGlobe or ArcScene apps, why they can’t just roll it all into ArcMap is beyond me. 3D won’t be part of workflows if I have to start ArcGlobe/Scene every time I want to work with 3D data. The 3D demo was similar to what we’ve seen at the DevSummit/BPC, but at this point all this stuff is shipping. Support for SketchUp models is very welcome, but ArcGlobe is depressing.

10:36AM: Space and Time, 4D. Time in ArcGIS has always been somewhat of a PITA. I mean there has been NetCDF support, but the UI was never really aware of it. Now not only ArcGIS Desktop supports time, but the APIs do as well. Tweets to find oil demo was interesting, but I’m not sure it really shows off mobile analysis with social media. Time aware attributes are going to need to be entered with geo-data moving forward. Maybe ArcGIS 10.x can automatically tag edits (I know geodatabases can do this, but it should be done to any data edited in ArcGIS)

10:50AM: ESRI President’s Award went to the City of Frisco, Texas. Susan Olson, the GIS Manager for Frisco accepted the award.

11:00AM: Lunch!

Well I’m sure you’ve all [heard the news](http://www.esri.com/news/releases/10_1qtr/amazon.html). ESRI is now an [Amazon Independent Software Vendor](http://aws.amazon.com/solutions/solution-providers/).

[](http://aws.amazon.com/solutions/solution-providers/)

This means of course that we’ll see some ArcGIS Server in the Amazon Cloud very soon. In fact if you are an [ESRI ELA](http://www.esri.com/industries/ela/index.html) user, you can take advantage of this right now using one of the pre-built AWS AMIs. Licensing still hasn’t been outlined by ESRI, which is probably why the ELA is required, but it seems like we could be close to hourly ArcGIS Server instances by next year. The AMI isn’t anything special, just a Windows Server AMI with ArcGIS Server at this point.

The [WeoGeo](http://www.weogeo.com) booth was right next to Amazon (or maybe Amazon was right next to WeoGeo, hmmm) and there seemed to be some traffic and lots of questions. Answers weren’t that concrete from what I heard and Amazon looked rushed into being there, but it did appear people made an effort to seek them out and talk about GIS in AWS. At this point ESRI and Amazon is so early in the public relationship that we’ll have to wait for the BPC/DevSummit or most likely the International UC to get the real details.

### The Plenary ###

OK so ESRI in the cloud didn’t knock your socks off, the inevitability of the whole thing at this point seemed to make many feel like it was anticlimactic. Tough world we live in.

Jack’s plenary talk was as always razor sharp on what ESRI is doing for their Federal customers and as always sets the stage for the year. As I alluded to early, the phrase “Cloud Ready” is something we’ll be hearing a ton about with ArcGIS 10. This means a couple things, first off it integrates with other cloud services with the REST API (something many have already been doing for years), second they’ve got this Amazon AWS AMI which you can license to run a full ArcGIS Server (without any scaling of course) in Amazon’s cloud and third I think it means that ESRI’s web services are going to essentially make even private or internal clouds “GIS Ready” (that’s my term in the spirit of Cloud Ready).

I think the Plenary was well received by the crowd, but they seemed quiet. I’d probably feel the same way if Mother Nature dumped a ton of snow on me for a couple weeks. Some interesting take-aways from the talk is ESRI’s focus on private clouds, which I think aligns very well with the FedUC crowd. Their focus on mobile was very apparent and I think at this point every reference to a Windows Mobile device has been removed from Jack’s slides and replaced with an iPhone. ESRI’s focus on web services means that they can transition to mobile devices with their mobile APIs (Ah, [here is the iPhone API](http://events.esri.com/bpc/2010/dev_agenda/index.cfm?fa=Session_Detail_Form&SessionId=18&ScheduleId=147) ready to work).

Jack focused on the large picture architecture of ArcGIS 10 and then it came for others on the ESRI team to come out and demo. We saw a good overview of ESRI’s ArcGIS Online map services. This world Topo map ESRI has been working on is really special. The cartography just catches your eye and [that it goes down to 1:1000k 1:1k](http://blogs.esri.com/Support/blogs/arcgisonline/archive/2010/02/18/arcgis-online-at-the-federal-user-conference.aspx) (off by a little scale factor there) scale in large cities really makes me want to use it instead of street map services. ArcGIS Explorer Online is a really slick Silverlight app that seems to emulate much of the ArcGIS Explorer (except 3D of course), which might be a good general GIS web services browser for ESRI users. They keep hiding the URL so so I can’t share it, but it was something like [http://explorerweb.arcgis.com](http://explorerweb.arcgis.com) or similar. We’ll see it soon enough I guess.

### It’s About Servers ###

Then the most surreal part of the whole FedUC occurred. John Calkins ran over his overview of ArcGIS Desktop 10 as he always does. If you’ve never seen John give this talk, you can [view one here](http://www.esri.com/software/arcgis/whats-new/play-johncalkins.html). John as always did a really good job and some of the refinement of ArcGIS Desktop 10 is simply amazing. The editing environment, threaded geoprocessing and symbology improvements really puts ArcGIS Desktop way beyond anything any other GIS vendor is doing. But what caught me off guard was the crowd’s reaction to it. As I said earlier, the crowd seemed tired and not into things, but during the Desktop demo I heard some things that really amazed me.

>”Why isn’t this demo in Flash (or Silverlight)?”

>”Why isn’t he using an iPhone to do this?”

>”Do people still use ArcMap?”

Here was a crowd that I thought would eat Desktop alive because they spend all day in it and many just didn’t care anymore. (Note: I don’t have super hearing so I could only listen to those in front or behind me) Could we finally be at a big shift in mentality where we are breaking out of these large legacy desktop clients and toward lightweight mobile and web clients for analysis? Are users finally listening to our “web is where the magic happens” talks and taking it to heart? Not sure, but it was interesting.

Now before everyone declares desktop GIS dead, lets be realistic here. Content creation tools are still not developed on mobile devices or web clients to the point were you can get the accuracy you need so for many users Desktop is still a required element and will probably be for decades to come. But I do think that average users of GIS, even those institutionalized in the federal service, are ready for this mobile, crowdsourcing future that we are just about to enjoy.

### Crowdsourcing? ESRI? ###

Yep, Jack talked quite a bit about VGI ([Volunteered Geographic Information](http://en.wikipedia.org/wiki/Volunteered_Geographic_Information)) which of course is a term used to describe crowdsourcing/neogeogrpahy/participatory GIS or whatever else is the term of the hour. Dave Smith did a really good job of summarizing crowdsouricng and ESRI [on his blog](http://surveying-mapping-gis.blogspot.com/2010/02/esri-and-volunteered-geographic.html) so I’d like to point you there from some reading. ESRI has put thought into ArcGIS’ place in VGI and how users will want to get information in and out. I think as ArcGIS 10 progresses we’ll see much more on this and how ESRI users can edit things such as OpenStreetMap directly from their ArcMap clients. I think the International UC should show us much more detail on how this is going to all work.

### On the Floor of the Expo ###

We at [WeoGeo](http://www.weogeo.com) of course were on the floor showing what [we are doing with ArcGIS and the cloud](http://blogs.weogeo.com/jamesfee/2010/02/16/introducing-weogeo-tools-for-arcgis/) but so were many others. Amazon was there of course as I said. GeoEye was but DigitalGlobe wasn’t. NAVTEQ and DeLorme were, but TeleAtlas wasn’t. I saw friends at [VoyagerGIS](http://voyagergis.com) and [Arc2Earth](http://www.arc2earth.com) (who was at the New Light Technologies booth) were there showing their latest products. Ran into [Stu Rich](http://www.spatialexplorations.net/) at PenBayMedia showing off some of their very impressive building interior modeling and of course everyone else from SAIC to lone GIS professionals who stopped by to say hello.

### 2010 in the ESRI Community ###

So as always the FedUC kicks off the ESRI year. We’ll see much more at the BPC and DevSummit next month, but the message is simple. ArcGIS 10 will interact with “the cloud” no matter what that term means to you. The more I see with ArcGIS 10, the more I can see why they named it 10 rather than 9.4. It really is a break from what ESRI was doing in the past on both the Desktop and the Server. ArcGIS 10 should arrive early Summer (not to jinx anything of course), probably before the International UC so we can all give it a test run before we show up in San Diego.

I hadn’t been to a FedUC in more years than I can recall. It was really great to see how much this conference has grown and how many more people are interested in geospatial technology as well as how people have embraced the concept of web services, web clients and mobile GIS as more than just a display tool. Should be a very exciting year.

It features interactive maps at the state, metro, county, and tract level, along with detailed statistics for each area. You can search in various ways, and also add overlays showing Congressional districts, ZIP Codes, tract-level maps of 2000 Census mail return rates, and recent foreclosure risk. There’s a FAQ that goes into details about the data and their methodology.

Clearly larger states have a bigger problem with hard to count populations but Alaska, Hawaii and New Mexico probably point out that there are socioeconomic factors as well.  Using the demographic layers available in the web app shows that this problem is very difficult to pinpoint and my hat is off to those trying to crack it.

The UI from the Census Hard to Count 2010 Application

Oh, Ruby!