Scripting in GIS

December 27th, 2006 · 27 Comments · GIS

Python Icon There has been a ton of discussion about scripting in GIS in this thread. No matter what GIS platform you use, at some point you are writing a script to accomplish some sort of geoprocessing task. For me it was all AML until Python showed up. I never got into the VBScript/JScript stuff as AML did all I needed. For some though it isn’t just the ability to script in their favorite language, but what kind of scripting help the application has. IntelliSense, integrated debugging, syntax highlighting are all big ones in almost everyones book, but does having them integrated into your desktop GIS matter at all?

Tags:

27 responses so far ↓

1 Christopher // Dec 27, 2006 at 8:55 am

For me it is Python and I don’t want it integrated into ArcGIS (or any other platform). The GIS platform shouldn’t care about intellisense, or anything else. It is just bloat.
2 Christopher // Dec 27, 2006 at 8:56 am

VBScript or JScript for scripting? What is this kindergarden?
3 Morten // Dec 27, 2006 at 9:30 am

I´d rather be completely without scripting languages. I want compile-time errors. Script languages are interpreted per-line, so syntax errors or invalid operations are not catched until that line is hit during runtime. It significantly increases development time.
Christopher: You might consider JScript as kindergarten, but show me a cool web GIS page what doesn´t use it.
4 mfuser // Dec 27, 2006 at 9:33 am

intellisense is bloat? Do you develop applications? I don’t know of any application developers who aren’t thrilled with intellisense in things like .NET Framework.
5 Dave // Dec 27, 2006 at 10:21 am

I think that Chris’ point is that building an IDE into an application leads to bloat - not specifically that Intellisense is bloat in an IDE.

All that aside, adding VBA into an application is actually relatively easy, since there is a VBA SDK (http://msdn.microsoft.com/isv/technology/vba/overview/default.aspx)
6 Christopher // Dec 27, 2006 at 11:42 am

@Morten

I was talking more GIS Scripting, than web scripting. JScript with ArcGIS is dumb, dumb, dumb.
7 Morten // Dec 27, 2006 at 12:50 pm

@Christopher:
Can’t help asking: Why is it dumb dumb dumb, and why is it Python is not? They are both scripted languages, with many of the same limitations.
8 Christopher // Dec 27, 2006 at 1:58 pm

Python is much better when you write longer scripts. When you have a simple script it is true that JScript and Python are very similar, but Python supports writing much larger programs and better code reuse with a object-oriented programming style. Classes and inheritance are big pluses in the Python camp.
9 Chris // Dec 27, 2006 at 2:18 pm

@Morten,
“I´d rather be completely without scripting languages. I want compile-time errors. Script languages are interpreted per-line, so syntax errors or invalid operations are not catched until that line is hit during runtime. It significantly increases development time.”

It might help if everyone would quit calling Python, Ruby, et al. “scripting” languages. They’re not. Python/Ruby are robust, dynamic programming languages. To call them scripting languages in the same vein as VBScript is an insult.

Compile-time errors are highly over-rated. And as for the development time associated w/ dynamic languages, it’s well documented that on-the-whole they’re much more productive than statically-typed languages (even with their shiny IDEs).
10 anon // Dec 27, 2006 at 2:38 pm

I think it is far in this conversation to call Python a scripting language. Generally though since Python is interpreted rather than complied, it is somewhat more of a middle ground vs .compiled or even JIT compiled.

I don’t consider Python or Ruby on the same level as VBScript or JScript myself, but they aren’t close to C++ or even .NET.
11 Chris // Dec 27, 2006 at 3:07 pm

“Stay on target….stay on target!”

Personally, I’m not opposed to IDE-like conveniences built-in to software like ArcGIS, but the catch is that it must perform well. Performance in ArcGIS is questionable just for GIS stuff, so what’s the point of Intellisense that takes 5 seconds to auto-complete something for you?

The VBA environment w/in ArcGIS isn’t bad (as far as VBA goes), but I don’t know how a similar environment for Python would work out. One benefit to keeping them separate is that you aren’t dependent on having ArcMap running while editing Python scripts. I can just pop open PythonWin or Scite (for Ruby) and G.S.D. (Get Shit Done).
12 anon1 // Dec 27, 2006 at 3:14 pm

Avenue for me…
13 -Rob // Dec 27, 2006 at 4:00 pm

I’d say leave the integrated scripting environment out of the IDE. Anytime an application starts adding non-core functionality to the IDE, key features are lost and not implemented.
14 The Memory Leak » Blog Archive » Domain Specific Languages for GIS? // Dec 27, 2006 at 4:17 pm

[...] When I first heard about domain specific languages (DSLs) a couple of years ago, I figured there would soon be folks really digging into this for GIS. The discussion here about GIS scripting is interesting, but would be more helpful to have a DSL discussion. I guess ESRI’s modelbuilder is a DSL IDE, looks similar to the one discussed here. I always liked the way unix allows you to chain commands together into one long single command, piping the output of one command as the input of another. No IDE needed - its all command line. Seems like unix-friendly DSLs would be a natural step for the opensource folks. OGC mentions DSLs here as “GML Application Languages”, but I can’t really follow this to anything concrete. [...]
15 Morten // Dec 27, 2006 at 4:21 pm

@Chris:
“Compile-time errors are highly over-rated”
So what you are saying is that avoiding a large portion of errors before executing your code is over-rated?

@anon
Since it’s an interpreted language (as you say yourself), it is also a scripted language. A script if something you follow line-by-line interpreting each line on at a time. This is not the case with compiled code.

Don’t get me wrong. Python (or any other scripted language) is not the root of all evil. It has it’s benefit. But James started a discussion on what we prefer, and I simply prefer compiled code over almost anything.
16 Shane // Dec 27, 2006 at 9:21 pm

Silly rabbit, scripts are for kids. Keep scripting out of desktop GIS products. The likes of geoprocessing models fulfill this need. The experience using esri’s geoprocessing classes with Python was lackluster at best. Regardless of the vendor, the object of scripting is to batch processes together independent of the IDE. With this in mind, there’s no need for the ability to script directly in the GIS platform.
17 matt m // Dec 28, 2006 at 7:30 am

Absolutely need it to simplify complex tasks for the user…even at the level of a macro. What’s your button going to do without a script to call?

For my own use, I like applications with a command prompt where you can do interactive processing…
18 Lefty // Dec 28, 2006 at 8:32 am

I never got into scripting ArcGIS (I still used AML).

Until Python showed up. I’ve not use an AML since James pointed me to geoprocessing with Python.
19 Chris // Dec 28, 2006 at 8:36 pm

@Morten,
No, I’m not saying compile time error checking is worthless, just over-rated. Advanced dynamic languages are pretty smart about reconciling data types/structures, so the extra time it saves can be used to make sure your code actually *does* what you think it will (e.g. unit tests, etc.) .

Don’t you think that in the year 2006+ we should be past the point of having to manually check to see if our code is type safe?

I don’t know why Python performance in ArcGIS is lackluster, but it’s not Python’s fault (it’s a pretty fast dynamic language). Avenue wasn’t compiled and some people swear by Avenue + ArcView 3 vs. Python + ArcGIS.

The biggest benefit of dynamic languages is that they save developer cycles vs. CPU cycles. Last time I checked it’s way more expensive to pay a developer than buy more hardware.
20 Dave Bouwman // Dec 29, 2006 at 10:42 am

Re: “Last time I checked it’s way more expensive to pay a developer than buy more hardware.”

Kinda, sorta, not really. The issue is the incremental licensing costs. On the server side, we’re back to caring about how performant the code is because the incremental (licensing) cost of scaling out is high. In terms of developer time (say a senior dev @ $200/hr), $40,000 in licensing is 200 hours of effort - which should be more than enough to optimize almost anything if you know what you are doing.

Dave
21 mdsumner // Dec 30, 2006 at 5:57 am

I’d suggest those of you who rail against the term “scripting” _ get_over_ it_. The standard for programming was set in prehistoric times and it’s only now that we have the easy computing power that many languages are taking advantage of all the “good programming” theory. Finally we can dispense with worrying about whether it’s a “real” program.

Choose your poison and drink heavily, that’s my advice.
22 mdsumner // Dec 30, 2006 at 5:58 am

[Post hoc wishful edit]: pluralize “poison”.
23 Valery Hronusov // Dec 30, 2006 at 2:43 pm

Now best languages for GIS it is XSLT and head

Best language it is best results with minimal work.
Script vs extensions vs geoprocessing?
24 matt wilkie // Jan 9, 2007 at 4:42 pm

pushing language wars aside, I think Shane’s comment is the closest to the heart of the topic so far: how to batch process GIS tasks?

Personally I’ve had more luck with python scripts than geoprocessing models. Maybe that’s because I’m coming from an AML background {shrug}. I like the fact that with python I can pull in the likes of GDAL if need be.

I’m also in accord with ‘matt m’, interactive command line can be *very* useful. My dream is to have some sort of fusion betwixt the command line and the scripting, something like I-Python. I guess I’m hankering for an Arc: workstation prompt with tab completion (intellisense), no 1024 character limit, and inline FOR loops.
25 Mohan // Oct 29, 2007 at 4:17 pm

Is there any book on Python Scripting for ArcGIS Users?
26 Steve Larch // Oct 30, 2007 at 8:43 am

Wouldn’t it be nice? IMO, your best bet would be to poke around the EDN or play around with Model Builder and then export it out to python.
27 katka // Feb 18, 2008 at 10:22 am

Hello, I’m student and my dissertation goes deep in the Pathon scripting. When I choose my topic i didn’t realize this. But now I am little lost and only ArcInfo Desctop Help can’t help me. And I don’t have any other material how some processes write. Would be somewhere somebody who could help me or share some more material? Do you know some good website or something? Thank you very much!!