Category: Thoughts

Jeremy Bartley and Keyur Shah (of http goodness fame) presented the first ArcGIS Server REST API session. ArcGIS Server can work with practically any client moving forward. To use the REST API, you publish your ArcGIS Server service like you’ve been doing (or reading about) for years. This means you can serve up simple maps like you would have done with ArcIMS, but also Geoprocessing services as well can be use (imagine having Google Maps work with a Geoprocessing service). The change from 9.2 is that rather than using SOAP to access the services, you can use use REST.

you want to get into what REST is go ahead and use Google to search. The simple explanation for those who don’t have time to research search is that everything is a URL, exchange standard formats (JSON, HTTP) using standard verbs, and every request asks a full question and every response includes the full answer. Just think of Keyur’s “http goodness” idea. So everything is a URL. This means that your resources can be accessed via URL and thus wget, curl, JavaScript, Python, Ruby, Perl, Java and .NET. I’ll put a link here to the presentation when it gets put up on EDN and you can see the detal that Keyur explains what REST is and how it applies to ArcGIS Server (or ArcIMS ) developers

Services formats you can use are html (the Services Explorer) JSON, KMZ, image, help, Layer file that you can add to ArcMap or ArcGlobe, ArcGIS Explorer documents, jsapi, Google Maps and Virtual Earth. The Server Explorer gives developers (or nosy GIS professionals) simple and instant access to Server Level Metadata. On every Server Explorer page, you will see a link to a help file that will help you understand what is going on. JSON will be used by the JavaScript libraries (or any programming language). There is also a “pretty JSON” feature that will return a more readable JSON (f=json&pretty=true) for debugging purposes.

ArcGIS 9.3 supports KML out of the box. Therefore you can build web pages that include links to KML content. Not only does it support KML/KMZ, but it also supports query results, geocode results, Geoprocessing task results and custom raster or vector results corresponding to selected layers from a map service. So you can pass the results from analysis performed in ArcGIS Server and offer them up in your webpages for users. Rather than export a KML/KMZ out of ArcMap and linking to that on your webpage, you would just make a REST call to the KMZ format on that mxd that you published on ArcGIS Server.

REST API Admin allows developers to clear the REST API Cache that gets created. If your add, delete or updates services, you’ll want to clear the cache. You can clear the cache by clicking on a button or have it clears periodically depending on your needs. REST API Security ties in with general ArcGIS Server Security policies so you won’t have to set anything different just to use the REST API.

Jeremy demonstrated how you want to set your correct projection in your MXD file before publishing (to match Google Maps or Virtual Earth). This of course is very simple, but you want to make sure it is set correctly if you want to match the web Mercator projection. ESRI and Microsoft/Google use different cache schemes, but ESRI has a tool that will convert your tile cache from the ESRI “format” to the Microsoft/Google tile format. This is only needed for 3D as ESRI handles the conversion automatically for the 2D. One nice function is the REST API support KML regions as well.

The simplicity that you can build applications via REST is clear. If you want to run a query, all you have to do is add a url and then call the url via JavaScript. So you pass the url to the query service with the coordinates you want to query and the RESTful server will pass back the results via another url to the browser. For those who have struggled with the .NET and Java WebADFs, this simplicity will be refreshing and I think we’ll see many developers come back into the web mapping fold because they can now work with ArcGIS Server services.

Jeremy also ran a Python demo to access the ArcGIS Server services via REST. If you aren’t a .NET or Java “guy” and want to use Ruby, you can now work within your favorite language. Heck, you like Yahoo! Pipes? Just add your RESTful URLs to Pipes and run the pipe to use the Yahoo! services with your ArcGIS Server services.

The freedom to use these RESTful services where you are comfortable, or where you client requires will stop the need to “sell” the need for installing .NET or Java on a server. Any “old” http server will be able to run these applications and there is definitely no reason to be afraid of ArcGIS Server anymore. As one ArcIMS developer told me, I finally see a reason to leave AXL and move to ArcGIS Server. The REST API is well thought out, very extensive and allows access to the power of ArcGIS Server in a “simple” html page.

Jim McKinney opened the Plenary by introducing Jack Dangermond to welcome everyone to the DevSummit. The DevSummit has gotten much bigger over the last 3 years. It had more than doubled since the first DevSummit. The conference has changed from previous years. It is a longer a conference than before (pre-conference seminars). More sessions than before, presentations will be recorded and put up on EDN, larger community center and ESRI showcase (the server island was described as a continent). ESRI has also added the demo theaters to the showcase to augment the tech sessions. It definitely feels much larger than before. On top of it all, the plenary is much shorter (only 2 hours) than before so everyone can get right into tech sessions.

ESRI has committed to releasing service packs every quarter for 9.2. The future includes SP5 for 9.2 out soon and 9.3 in “June” (or before the User Conference) and 9.4 should arrive in “early” 2009.

ArcGIS Server

Ismael Chivite, the ArcGIS Server product manager, lead the talk on ArcGIS Server. The tag line that ESRI has been using is “Making GIS Knowledge Available To Anyone…”. This aligns with what was send at the FedUC and the BPC. Putting GIS into consumer clients (Google Maps, Google Earth, Virtual Earth) is clearly a huge change for ESRI. ArcGIS Server 9.3 performance is improved because of improved caching workflows (partial cache and on demand cache). This means you don’t need to create map tiles for the world, but pick areas you feel are important. The areas that are not tiled, can be created on demand as users view the world. The “simple” javascript API will speed development over “traditional” .NET or Java ADFs. Security can now be managed from within the Server Manager on each service. Much like the ArcGIS Explorer resource center that everyone can see, there will be a detailed ArcGIS Server 9.3 resource center (only available for Beta users at this point) that will aggregate the documentation and other documents in one place. ESRI is committed to delivering more code samples and web tasks for developers to use. (I can confirm that there are more samples available than I’ve ever seen for an ESRI release)

Web ADFs (.NET and Java)

Rex Hansen demonstrated how you can quickly take code samples out of the Resource Center and create WebADF and ASP.NET AJAX applications quickly and easily. The sample Rex demonstrated was simple and valuable which is definitely something that has been missing on ESRI samples from before. Rex showed how you can quickly take the ESRI sample and create a Sharepoint Web part using your own data. Hopefully this will improve usability of creating quick WebADF applications that leverage Microsoft development tools.

Rex apparently lost his mind and started talking about Java and AGS 9.3. The Java example was very similar in the sense that the code was simple and any Java developer should be able to customize it to work with their own data and projects. Rex looked pretty familiar with Eclipse so maybe he’s been cheating on us .NET folks.

Javascript API

The ArcGIS Javascript API is probably the biggest new feature of ArcGIS Server at 9.3. Now instead of being limited to .NET or Java, you can now pick a simple Javascript API to publish to ESRI’s Javascript API, or Virtual Earth and Google Maps.

Jeremy Bartley demonstrated the new Javascript API by visiting the Javascript API section of the ArcGIS Resource Center. Javascript API resource center allows you to interactively explore the API. Much like how Google has described the Google Maps API, ESRI has really taken the time to explain their Javascript API (Now I’m sure it isn’t as detailed at Google’s Docs, but the change from previous ESRI documentation is very noticeable.) The Javascript API reference is clean and easy to use and there are even plans for a “community” section where developers will be able to upload their own code or tricks that they have for the Javascript API (this will also be available for the .NET and Java sections as well). The speed that you can add ArcGIS Server services to your maps is just slick. One line of Javascript code and a url to the service and you are running. Many have already browsed the Services Explorer on the demos that ESRI has posted and have been able to see the capabilities of the service.

Jeremy also demonstrated the Google Maps and Virtual Earth extenders. Seeing ArcGIS Server being leveraged in Google Maps really gives developers the ability to take classic GIS analysis and put it in a context that “ordinary” users can work with. I will say when Jeremy was done, the crowd really gave ESRI a good cheer. I think that proves that this Javascript API is going to be used quite a bit moving forward. One point about the Javascript API’s is that no development or deployment license is required on the Web server hosting your application.

ArcGIS Mobile

Developers I’ve talked with this week have really been interested in what ESRI has been doing with ESRI Mobile solutions. What is interesting at 9.3 is that ESRI includes ArcSDE SQL Express with ArcGIS Engine. You can do both one and two way replication with ArcGIS Server. ArcGIS Mobile integrates deeply with Visual Studio and enables quick development and deployment. Being able to develop a mobile application on Windows Mobile and Smartphone with a couple lines of code should increase Mobile usage among ESRI Developers.

ArcGIS Desktop/Engine

Euan Cameron talked about changes to ArcGIS Engine 9.3. The biggest news I think was the ability to include SQL Server Express Support with deployments. At 9.3 ESRI increased the amount of documentation and code. Desktop now allows the ability to have “Z Aware” editing, HTML popup windows (like what you are seeing with ArcGIS Explorer), more online support content and VBA 6.5 (thus Vista and Office integration).

Bernie Szukalski started to demo some 9.3 enhancements (the new “left hand tool”), and ArcGIS Desktop 9.3 crashed. We got to see the error detection and debugging tools. Crash dumps created automatically and ESRI will use these to help track down crashes. This should result in Service Packs addressing problems quicker than before.

ArcGIS Explorer

Now the real reason that Bernie Szukalski was up on stage (no he wasn’t there to crash ArcGIS Desktop) was to demonstrated ArcGIS Explorer The future releases of ArcGIS Explorer include 480 will be released in May and followed by releases 600 and 700 by the end of th year. Bernie was demonstrating how you can include ArcGIS Server Globe Services, ArcIMS Services, WMS Services, local imagery, file geodatabases and KML. The new icon symbols really look great and very sharp (and will be available with ArcGIS 9.3 Desktop for use in any ArcMap or ArcGIS Server production). Bernie even when to the Google 3D Warehouse to grab a KMZ of a 3D building. Bernie also demonstrated how you can even embed YouTube videos right into the info bubbles. Any HTML content can be used, there is no limits so you can put any web content into ArcGIS Explorer.

ArcGIS Explorer 480 will increase performance (multi-threaded), Direct Connect to SDE!!!!!, GPX support, GeoRSS, improved task framework and popups and ability to load AGX in your web browser. The future moving forward includes a new user interface, integrated 2D and 3D display, markup and collaboration and a Map Control to embed Explorer objects. You’ll be able to take the map display and use it in your applications.

ESRI demonstrated build 600 (which isn’t the next version) and it has the new “ribbon” interface that you’ll recognize from Microsoft Office. Now tasks aren’t hidden in the table of contents, but available quickly and easily. It really does look like a Microsoft Office application which should help with its adoption. The usability of build 600 is really striking compared to the existing ArcGIS Explorer builds and even Google Earth. Another key feature of build 600 is the ability to view maps in 2D. The enhanced Explorer SDK which will be available with either build 600 or 700. Embedding AGX inside your applications will be big for many developers.

Lastly Scott Morehouse discussed where the ArcGIS Platform is going. ESRI is at the “envisioning” stage with 9.4 so the User Conference will be where we’ll see what will be part of the release. It will build upon what happened in 9.2 and 9.3 so it shouldn’t rock the boat and will probably be a nice stable release. The release schedule of ArcGIS Explorer will be adopted by the Server and Desktop teams so we’ll be seeing more and quicker releases of software, rather than these big Service Packs and large jumps in technology we have been getting. There will be more support for standards, much improved Linux support, faster services in ArcGIS Server and more stability. 9.4 will also see Flex and Silverlight be part of the Server platform. Desktop will continue to be improved with stability/performance and new editing tools showing up. ArcGIS Online is now an integral part of ArcGIS and is not a separate product. More online documentation, best practices and code galleries will also begin to show up with 9.3 and on to 9.4.

Jim Barry described where EDN was heading including more blogs coming online in the next few months (Desktop Developer and Engine) as well as highlighting some of the improved online support tools at 9.3. Also the forums are being updated as part of the 9.3 beta program and when 9.3 is released, will replace the old forums that have existed for years.

That ended the plenary and we will move on to the “super sessions”.

How about this choice at the ESRI Store at the Developer Summit?

Editor’s Note – 01-09-2017

Sadly I lost the image that was posted here. Not only that I can’t recall what book it was. Likely it was an open source web mapping book but alas I can’t recall.

There really wasn’t that much going on yesterday other than a couple points Jack made on stage that will probably be built upon during the Dev Summit:

1. Jack says Microsoft and Google are the key to collaborative GIS using ArcGIS Server as the back-end. The hope is ArcGIS will integrate with everything moving forward.
2. ArcGIS 9.3 will be in everyone’s hands by June. That means that this will be a very sort beta period. Reading between the lines, this could mean that there are very few new features and this really will be a release that focuses on quality and documentation.
3. As others mentioned at the FedUC, Jack talked about Open API support. Want to use OpenLayers with ArcGIS Server, no problem.
4. The new Javascript API will increase production thousandfold. Now you can use the simple APIs to deploy quick and simple mapping websites and use the WebADF for the detailed heavy sites.
5. Did you use AML with ArcPlot? 9.4 might add the ability to script map production.

Now there was much more, but we’ll leave that for the DevSummit where we’ll learn the details.

If you want to see some of the topics that people came up with to ask ESRI before the DevSummit next week, ESRI has posted them on their website. Here are some of the ones that caught my eye.

Q: What are future plans for releases following ArcGIS 9.3?
ESRI Director of Research and Development Scott Morehouse will provide a brief overview of the themes for ArcGIS 9.4 during the Plenary. If you have suggestions for ArcGIS 9.4, we encourage you to share them with development staff in the ESRI Showcase and during the Meet the Development Teams sessions.

We’ve known that 9.4 was coming down the pike and it appears that ESRI is going to talk about what it is and how it might impact ArcGIS 10. I am really interested to see what the timeline moving forward will be.

Q: Are you planning 64-bit versions of ArcGIS Desktop and ArcGIS Server?
ArcGIS Desktop is fully supported on 64-bit Windows at 9.2 SP 3 and beyond. While ArcGIS is a 32-bit application, it has been tested and certified on the 64-bit versions of Windows. We have no immediate plans to release a native 64-bit version of ArcGIS Desktop, although we do continue to research this possibility.

At ArcGIS 9.2, we released 64-bit versions of the ArcSDE component of ArcGIS Server for some UNIX platforms. At 9.3, we will also release a native 64-bit version of ArcSDE technology for Windows and Linux.

Nothing new there but I don’t see anything aut ArcGIS Server (non ArcSDE). That is the big issue for us. Sure Desktop 64-bit might be nice, but all of ArcGIS Server as 64-bit is critical.

Q: Will there be a replacement for MapObjects? In other words, will there be options for desktop deployments that don’t require the full functionality of ArcGIS Engine?
Yes, we are doing some research work on building a focused, lightweight .NET API deployment option with functionality similar to MapObjects. We will discuss our thoughts at the DevSummit Plenary. We encourage your feedback both at the summit and in the postsummit survey.

Every MapObjects developer just sat up and smiled. Wonderful news.

Q: Discuss performance improvements in ArcGIS Server 9.3.
Performance improvements in ArcGIS Server 9.3 are significant. First of all, Web mapping applications built with the Web ADFs (both .NET and Java) have been greatly optimized in different ways (reduced number of requests to the server, more lightweight, etc.). This has resulted in more responsive applications and increased scalability of the whole system. Additionally, 9.3 features a new core service for efficiently publishing imagery called image services. This new service provides superior map throughput compared to standard map services. Performance improvements also derive from optimizations at the geodatabase level. Finally, we have made map caching workflows easier and faster and provided you with the choice of partially caching your map services. The new caching-on-demand feature of map services will allow you to leverage the power of cached maps without having to build an entire cache of your maps.

I know most ArcIMS devs are usually unhappy with ArcGIS Server performance, but it sounds like they’ve addressed this. And building portion of the map cache on the fly will be great. More good news.

Q: Will ArcGIS 9.3 support backwards direct connect compatibility?
Yes. ArcGIS 9.3 clients will have backwards direct connect compatibility with 9.2, 9.1, and 9.0 ArcSDE geodatabases. This will make ArcGIS software upgrades easier, especially for large enterprise systems, by giving more flexibility to perform a sequenced migration. For example, clients can be updated first followed by server updates.

Nice, as ESRI says it will make ArcGIS software upgrades much easier.

Q: How and when will ESRI support the spatial types in Microsoft SQL Server 2008?
ArcGIS 9.3 will support the two new spatial types in Microsoft SQL Server 2008. A geodatabase implemented on SQL Server 2008 will be able to store vector geometry in either spatial type (Geography or Geometry). This will enable users to perform SQL queries and operations on spatial data within SQL Server.

Basically ESRI will enable access to SQL Server 2008 beta at 9.3, but will not support it. When SQL Server 2008 goes final, ESRI will release a SP to address any concerns (if any) and then support SQL Server 2008. Seems logical and better than having to wait for SQL Server 2008 to go final before getting any ability to use with ArcGIS.

Q: To what extent does ESRI support interoperability with KML?
KML is one of the supported formats of ArcGIS Explorer and the ArcGlobe application within ArcGIS 3D Analyst. You can simply use these 3D applications to explore the contents of KML files. Additionally, you can use ArcGIS Server to publish your GIS data and output of GIS analysis in KML format. You can either create map services to publish GIS data as KML overlays or placemarks or use the new REST interface of ArcGIS Server to enable your users to get KML outputs from map queries and geoprocessing services.

It looks like we’ll still have to use third party extensions to import/export KML out of Desktop. I’ve heard good things about KML support in Server though so that takes the sting out of the lack of support on Desktop.

Q: How are you improving software documentation especially for developers? More sample code is required.
We have clearly heard this feedback and have made additions to the online help and SDK documentation. For example, in 9.3, there are three or four times more help topics and samples for developers. In addition, we are introducing resource centers for each product that bring together online resources, including samples, blogs, and best practices information, to help make you successful. ArcGIS 9.3 beta testers have access to the beta documentation and are encouraged to provide feedback.

Good news, but ESRI needs to improve the search as well. More documentation and code is very welcomed, but not being able to find it reduces its value. Documentation at 9.2 was poor, so if 9.3 can get that back up to where it should be, the we all win.

What a horrible way to end your season and possibly cost you a NCAA tournament birth. If you aren’t UCLA or USC in the Pac-10, nothing goes your way. To make that call at that point in the game is atrocious.

[youtube https://www.youtube.com/watch?v=agurNAiAPPg&w=560&h=315]

I’ll be driving out from Tempe early tomorrow morning to the 2008 ESRI Business Partner Conference. I’m looking forward to meeting and talking with everyone. Feel free to email me if you want to hook up and get drinks or food (I’m staying at Hotel Zoso down the street). I’ll of course be staying over the week for the Developer Summit as well. Looks like the weather will be much colder than the nice warm days we’ve been having so make sure you pack accordingly (note: my definition of cold will probably be different than others).

Alas, our 9.3 beta didn’t show up before I left work today so I guess I’ll have to just watch, rather than participate. Oh and if you want something to read on the airplane on your way to Palm Springs, why not print out some of the slide shows before and bone up on your 9.3 knowlege.

Look I believe in the concept, but I’m so sick and tired of these freaking terms. Personally I think folks use the term VGI to segregate themselves from the community at large. I mean look at this list of participants at a workshop late last year.

Does it bother anyone else that a workshop meeting on social cartography, limited to 40 persons is elitist? This infatuation with VGI is becoming unhealthy to the direction we all want GIS and Geography to move. Why waste efforts down a side street when the main street geospatial crowd is working so hard to get more open. This VGI fad is just more of that “University” mentality that drags down innovation.

Update Fantomplanet says:

You take VGI for what it is. Someone has to sort that crap out half the time……. I asked Goodchild how his concept isn’t any different that the stuff the wiki-blog crowd is trying to sort out. “It isn’t.

Well if Wikipedia is a goal of the concept, then VGI is a non-starter. The world needs less, not more gatekeepers and the whole Wiki concept is based on editors deciding what is right and wrong, not the community. Just because you can edit something doesn’t mean that it is collaborative. Wikipedia is flawed and cult-like and if that is the direction of Geography, count me out.

I wish I was able to go but if you are like me, you can live the experience through others.

Peter Batty’s FME User Conference Report

Jason Birch on KML Goodness

Matt Ball on 3D and FME

We’ve hit on this discussion before and with the Developer Summit coming up maybe it is a good time to think about the direction processors and their movement to multi-cores and 64-bit processors.

At 9.3, ArcGIS Server Enterprise (or whatever ArcSDE is called these days) will move to 64-bit. This is a huge improvement as I would guess most new database deployments are built on 64-bit servers. But clearly ArcGIS Server itself is not going to be 64-bit at 9.3. When spec’ing out new servers, it is impossible not to go down the 64-bit route and servers going multi-core only in the next year ArcGIS Server will only get slower because it cannot take advantage of new technology. With the focus moving from clock speeds to cores, ArcGIS Server users run the risk of being stuck at a level of performance that is going to be unacceptable in the future.

I’ve been told that ArcGIS 10 will support multi-core/64-bit, but given that it probably won’t go final until at least late 2009/early 2010 that means we’ll be running into trouble way before we can even deploy beta version of ArcGIS 10. But is this really a concern for users? Generally speaking, most folks I’ve talked to don’t seem to really be bothered by this and maybe that is why ESRI is waiting until v10 to ship multi-core support.

Is the lack of 64-bit ArcGIS Server going to impact your business moving forward?

Moony only runs 64-bit servers, do you?