Spatially Adjusted

Tag: esri

  • A GIS Degree

    My son decided to change majors from biodesign to GIS. I had a short moment when I almost told him not to bring all this on himself but then thought differently. I could use my years of experience to help him get the perfect degree in GIS and get a great job and still do what he wants.

    He’s one semester into the program so he really hasn’t taken too many classes. There has been the typical Esri, SPSS and Google Maps discussion, but nothing getting into the weeds. Plus he’s taking Geography courses as well so he’s got that going for him. Since he’s at Arizona State University, he’s going through the same program as I did, but it’s a bit different. When I was at ASU, Planning was in the Architectural College. Now it’s tied with Geography in a new School of Geographical Sciences & Urban Planning.

    I have to be honest, this is smart, I started my GIS career working for a planning department at a large city. The other thing I noticed is a ton of my professors are still teaching. I mean how awesome is that? I suddenly don’t feel so old anymore.

    I’ve stayed out of his classes for the past semester in hopes that he can form his own thoughts on GIS and its applicability. I probably will continue to help him focus on where to spend his electives (more Computer Science and less History of the German Empire 1894-1910). He’s such a smart kid, I know he’s going to do a great job and he was one who spent time in that Esri UC Kids Fair back when I used to go to the User Conference. Now he could be getting paid to use Esri software or whatever tool best accomplishes his goals.

    I plan to show him the Safe FME Minecraft Reader/Writer.

  • Unreal and Unity are the new Browsers

    Someone asked me why I hadn’t commented on Cesium and Unreal getting together. Honestly , no reason. This is big news honestly. HERE, where I work, is teaming up with Unity to bring the Unity SDK and the HERE SDK to automotive applications. I talk about how we used Mapbox Unity SDK at Cityzenith (though I have no clue if they still are). Google and Esri have them too. In fact both Unreal and Unity marketplaces are littered with data sources you can plug in.

    HERE Maps with Unity

    This is getting at the core of what these two platforms could be. Back in the day we had two browsers, Firefox and Internet Explorer 6. Inside each we had many choices of mapping platforms to use. From Google and Bing to Mapquest and Esri. In the end that competition to make the best API/SDK for a mapping environment drove a ton of innovation. What Google Maps looks like and does in 2021 vs 2005 is amazing.

    This brings up the key as to what I see happening here. We’ll see the mapping companies (or companies that have mapping APIs) deliver key updates to these SDK (which today are pretty limited in scope) because they have to stay relevant. Not that web mapping is going away at any point, but true 3D world and true Digital Twins require power that browsers cannot provide even in 2021. So this rush to become the Google Maps of 3D engines is real and will be fun to watch.

    Interesting in that Google is an also-ran in the 3D engine space, so there is so much opportunity for the players who have invested and continue to invest in these markets without Google throwing unlimited R&D dollars against it. Of course it only takes on press release to change all that so don’t bet against Google.

  • Apple’s Digital Twin is All About Augmented Reality

    Now before we get too far, Apple has not created anything close to a Digital Twin as we know them. But what they have done is created an easy way to import your building models into Apple Maps. Apple calls this their Indoor Maps program.

    Easily create detailed maps of your indoor spaces and let visitors see where they are right in your app. Organizations with large public and private spaces like airports, shopping centers, arenas, hospitals, universities, and private office buildings can register for the Indoor Maps Program. Indoor maps are built using industry standard tools and require only your existing Wi-Fi network to enable GPS-level location accuracy so visitors can navigate your spaces with ease.

    Victoria Airport in the Apple IMDF Sandbox

    OK, so clearly this is all about navigation. How do I know where I am in a building and how do I get to a place I need to be. Of course, this is somewhat interesting on your iPhone or iPad in Apple Maps, but clearly, there is more to this than just how do I find the restroom on floor 10 of the bank tower.

    To load your buildings in Apple you need to use Mapkit or Mapkit.js and convert your buildings into Indoor Mapping Data Format (IMDF). IMDF is actually a great choice because it is GeoJSON and working toward being an OGC standard (for whatever that is worth these days). I did find it interesting that Apple highlights the following as the use case for IMDF:

    • Indoor wayfinding
    • Indoor routing
    • Temporal constraints
    • Connectivity amongst mapped objects
    • Location-based services
    • Query and find by location functionality

    If you’re familiar with GeoJSON, IMDF will look logical to you:

    {
  "id": "11111111-1111-1111-1111-111111111111",
  "type": "Feature",
  "feature_type": "building",
  "geometry": null,
  "properties": {
    "category": "parking",
    "restriction": "employeesonly",
    "name": {
      "en": "Parking Garage 1"
    },
    "alt_name": null,
    "display_point": {
      "type": "Point",
      "coordinates": [1.0, 2.0]
    },
    "address_id": "2222222-2222-2222-2222-222222222222"
  }
}

    I encourage you to review the IMDF docs to learn more but we’re talking JSON here so it’s exactly how you’d expect it to work.

    Because IMDF buildings are generalized representations of the real-world data, this isn’t actually a Digital Twin. It also means that you need to do some things to your files before converting them to IMDF. Autodesk, Esri, and Safe Software all support IMDF so you should be able to use their tools to handle the conversions. I’ve done the conversion with Safe FME and it works very well and probably the best way to handle this. In fact, Safe has an IMDF validator which does come in handy for sure.

    Safe FME support of IMDF

    What does make moving your buildings to Apple’s Indoor platform is the new iPhone 12 and iPad Pro LiDAR support. This brings out some really great AR capabilities that become enabled with Apple’s devices. As I said last week, the LiDAR support in the current devices is more about getting experience with LiDAR use cases than actual LiDAR use. This is all about eventual Apple Glass (and Google Glass too) support and the AR navigation that can be done when you have hyper-accurate indoor models in your mapping software.

    I’ve been dusting off my MapKit skills because I think not only is this capability useful for many companies but it really isn’t that hard to enable. I am spending some time thinking about how to use the extension capability of IMDF to see how IoT and other services can be brought in. Given the generalized nature of IMDF, it could be a great way to allow visualizing IoT and other services without the features of a building getting in the way. Stay tuned!

  • GIS Data Formats and My Stubborn Opinons

    Taking this break I’ve been looking over my spatial data and trying to figure out how to best organize it. The largest public project I manage is the GeoJSON Ballparks and this one is easy to manage as it is just a Git repository with text files. GeoJSON makes sense here because it is a very simple dataset (x/y) and it has been used for mapping projects mostly which makes the GeoJSON format perfect. I used to maintain a Shapefile version of it in that repository but nobody ever downloaded it so I just killed it eventually.

    But my other data projects, things I’ve mapped or worked on the past are in a couple of formats:

    VECTOR

    • Shapefile
    • File Geodatabase
    • Personal Geodatabase
    • GeoJSON
    • KML
    • SpatiaLite

    Raster

    • TIFF (mostly GeoTIFF)
    • Esri Grid

    Now you can tell from some of these formats, I haven’t touched these datasets in a long time. Being Mac centric, the Personal Geodatabase is dead to me and given the modification dates on that stuff is 2005-2007 I doubt I’ll need it anytime soon. But it does bring of the question of archival, clearly PGDB isn’t the best format for this and I probably should convert it soon to some other format. Bill Dollins would tell me GeoPackage would be the best as Shapefile would cause me to lose data given limits of DBF, but I’m not a big fan of the format mostly because I’ve never needed to use it. Moving the data to GeoJSON would be good because who doesn’t like text formats, but GeoJSON doesn’t handle curves and while it might be fine for the Personal Geodatabase data, it doesn’t make a ton of sense for more complex data.

    Generic Icon Image
    This is as close to a shapefile icon as I could find, tells you everything doesn’t it?

    I’ve thought about WKT as an archival format (specifically WKB) which might make sense for me given the great WKT/WKB support in databases. But again, could I be just making my life harder than it needs to be just to not use the GeoPackage? But there is something about WKT/WKB that makes me comfortable for storing data for a long time given the long term support of the standard among so many of those databases. The practical method might be everything in GeoJSON except curves and those can get into WKT/WKB.

    Raster is much easier given most of that data is in two fairly open formats. GeoTIFF or TIFF probably will be around longer than you or I and Esri grid formats have been well support through the years making both fairly safe. What are some limits to data formats that I do worry about?

    1. File size, do they have limits to how large they can be (e.g. TIFF and 32-bit limit)
    2. File structure, do they have limits to what can be stored (e.g. GeoJSON and curves)
    3. File format issues (e.g. everything about the Shapefile and dbf)
    4. OS centric formats (PGDB working only on Windows)

    I think the two biggest fears of mine are the last two, because the first to can be mitigated fairly easily. My plan is the following; convert all vector data into GeoJSON, except where curves are required, I’m punting curves right now because I only have 3 datasets that require them and I’ll leave them in their native formats for now. The raster data is fine, TIFF and grid is perfect and I won’t be touching them at all. The other thing I’m doing is documenting the projects and data so that future James (or whomever gets this hard drive eventually) knows what the data is and how it was used. So little of what I have has any documentation, at least I’m lucky enough the file names make sense and the PDFs help me understand what the layers are used for.

    One thing I’ve ignored through this, what to do with those MXDs that I cannot open at all? While I do have PDF versions of those MXDs, I have no tool to open them on Mac and even if I could, the pathing is probably a mess anyway. It bring up the point that the hardest thing to archive is cartography, especially if it is locked in a binary file like an MXD. At least in that case, it isn’t too hard to find someone with a license of ArcMap to help me out. But boy, it would be nice to have a good cartography archival format that isn’t some CSS thing.

  • Esri Community Maps Data Sharing

    I’ll be honest, I really don’t follow Esri as closely as I used to. Not so much in that I don’t care to learn about what they are working on, more just that they do so many more things these days. It’s honestly hard to follow along sometimes, but every once in a while I see something that catches my eye.

     Esri is now offering a new option in our Community Maps Program for contributors to have Esri share their data with selected Esri partners and other organizations (e.g. OpenStreetMap) that maintain popular mapping platforms for businesses and consumers. If contributors choose to share their data with others, Esri will aggregate the data and make it available to these organizations in a standardized way to make the data more easily consumable by them and accessible to others. It will be up to those organizations whether they choose to include the data in their mapping platforms.  Where the data is used, attribution will be provided back to Esri Community Maps Contributors and/or individual contributing organizations.

    Community Maps Data Sharing

    I have to admit this intrigues me. Not so much that Esri is trying to insert themselves into a process, but that it makes sharing data easier for users of Esri software. In the end that’s probably more important than philosophical differences of opinion about closed fists and the such. The data is shared via the CC by 4.0 license that Esri uses for the Community Maps AOIs. I really like this, anything that helps share data much easier is a good thing for everyone, including OpenStreetMap. I’m sure we’ll hear more about this during the Esri UC later this month but it’s still a great announcement. I’ve always been a big users of OSM and getting more organizations to update their data in OSM is a huge win in my book.

  • It is Different With COVID-19…

    I started blogging in May of 2005. Right before Katrina hit and everything we knew about GIS disaster response changed. Katrina was that moment where the static image PDF of a map changed to a map service that ran on almost any modern (at the time) web browser. Immediately every GIS map server that was out there became irrelevant at best, dead to the world at worst. Remember though, Google bought Google Earth almost a year before Katrina and Google Maps didn’t launch until early 2005. The tools that created this disaster response revolution were in place, but not too many people used them or had heard of them. But less than 6 months after Google Maps hit the web, Katrina response was almost entirely driven by their tools.

    Remember this? Don’t try and pan!

    If you look at my blog entries from September and October, you can see attempts by Esri, Microsoft, Yahoo! and others to try and address this new paradigm of mapping but none of them stuck. Everyone, and I mean everyone, was using Google. Esri ArcScripts back then probably had 50 tools to convert SHP to KML or MXD to KML. We had tools like Arc2Earth that specialized in making maps easier with Google. And while Esri tools were still being used to generate the data, the display was happening on other platforms.

    This of course gave rise to the Neogeography revolution. I’ll spare you the bare breasted Andrew Turner graphic but at this time we had so many people doing things with GIS that had no idea what GIS was let alone what Esri was. The limitations on getting started with mapping went down and all you needed was a computer and a text editor to make a map. My blog is littered with examples of Neogeography, from EVS Islands to all that great Flickr mapping that Dan Catt and crew did back then. People didn’t ask for permission, they just did it. It all culminated in what I consider the greatest crowdsourced disaster mapping effort, the wildfires in San Diego back in 2007 (feel free to choose the Haiti response over this, that’s fine. I really like the example of using Google My Maps in your backyard for this).

    In all fairness, Andrew wasn’t literally saying it killed GIS.

    But something happened after this, it isn’t that people stopped mapping. Look at OSM growth. The amount of crowd sourced data continues to grow exponentially. But responses to disasters seemed to be run by Google and Microsoft themselves. Tools like Google My Maps continue to exist, but I truly can’t recall using one in the past 10 years. Or if the disaster was not interesting enough for Google, you’d see people using government websites to get that information. The Esri mapping had finally caught up that people would use the fire maps from the DOI other 3 letter agencies without complaining. The citizen effort moved to Twitter where it continues to show great promise, just not as a Google My Map. Take a look at the Bush Fire here in Arizona on Twitter. So many great posts by people but maps are either static images shared or links to traditional InciWeb maps.

    This brings us full circle to COVID-19 mapping. Think of the best and most up to date COVID websites. They are built on Esri technology. Google has websites, Microsoft has them too. But the Esri dashboard has finally had its moment in the sun. I wonder if this is because the market has matured, that the tools have matured or the data set lends itself to a more scientific approach to display rather than simple lines and points. The Johns Hopkins COVID-19 Maps & Trends website is the bible for this epidemic.

    GIS is no longer a side show on this response. I’m guessing that because this is more structured government data, Esri is uniquely positioned to be in the middle of it but even then, their tools have come a long way from the ArcIMS/ArcWeb madness that we dealt with during Katrina. COVID-19 dashboard is the opposite of Neogeography and that is OK. The influence of the citizens on mapping is clearly shown in the Esri tools we deal with today. They still drive me nuts from time to time but let’s be honest, they really do work for this situation. As we close out 1/2 of the way through 2020, hopefully we can keep the need for disaster response to a minimum.

  • The Esri 2020 Dev Summit Has Gone Virtual

    Let’s be honest, there is a bit of love for the Dev Summit. Those who attended the first one, we look each other in the eye and do that subtle nod knowing we were part of something amazing.

    Now the funny thing about the Dev Summit. I don’t think I’ve been back since 2009. I ended up going a different direction with my career after WeoGeo and while I don’t miss the Web ADF, I do miss the Dev Summit. Well the 2020 Dev Summit has been canceled.

    Due to the continuously evolving circumstances surrounding the coronavirus, the Developer Summit will be a virtual event and not a live, in-person conference this year. This was a difficult decision, made after careful consideration for all registered attendees and Esri staff.

    Makes total sense. The Business Partner Conference is still going to happen and they are going to take into consideration things:

    The events team is working directly with all of our venues to provide readily available hand-sanitizing stations. Alcohol wipes will also be distributed at various locations throughout the event. The custodial personnel will be regularly disinfecting all common surfaces. Information regarding basic health practices will be displayed on signage that recommends how to avoid the flu and other illnesses.

    Boy I can’t imagine going given the BPC is going to be such a small thing but maybe that is what makes it manageable. I know a couple people who have told me they aren’t going to attend the BPC this year, even before the cancelation of the Dev Summit because of COVID-19.

    But let’s not focus on the bad, let’s focus on the great outcome of this. They are still going to do a livestream of the plenary as usual and make the sessions virtual. I’m still waiting to see what this looks like but it really could be useful. I’m not saying that conferences don’t have a part in today’s workplace, but having the virtual option helps immensely for those who just can’t break away to learn the latest technology from Esri.

  • Esri Arcade

    When we think of Esri scripting and authoring languages, we think Python. Esri jumped in with two feet with Python and we were all much better off for it. But alas, as awesome as Python is, it isn’t as portable across the Esri ecosystem as they would like. To solve the problem either you choose another language to use that is more portable (JavaScript) or you write your own expression language and make is appear like Python and JavaScript had a baby.

    At the Esri Arcade

    Well that’s what Esri did, take Python, take JavaScript and a new expression language. Now in an open world this would be great because anything I wrote in Arcade would be usable anywhere else. But this is an Esri only solution as I can’t imagine other companies jumping in on it. But in Esriland, that’s OK because the ecosytem is large enough to support learning a proprietary language.

    I don’t use Esri software anymore so I can’t play with it but it is a logical solution to their problem of having to write code to work with data in different platforms. Theoretically one can now use Arcade to author and render maps and let the Esri software handle the rest. I’d wait to see what happens with Arcade and the eventual 1.x release. It’s the Esri Web ADF talking but…

    So another proprietary scripting language…

  • Jack Answers Your Questions — 2016 Edition

    Ever since GIS was created in a dark damp room in Canada, Jack has answered your questions. In the past this was a secret Q&A that only registered Esri users going to the UC could see but for the past 10 years or so Esri has posted it online. I’ve read through most of it and I think the part that has me most excited is the Python Web API. Just think about that for a bit, some amazing opportunities. 10.5 is on it’s way. We should take bets to see how far the 10.x releases get before they go into maintenance mode. I’m going with 10.8.

    That said, the problem with the list is it’s so damn hard to read so as a service I’m pulling out what I view as the important questions and what Jack answered.

    Q1: What is the meaning of this year’s User Conference Theme: GIS — Enabling a Smarter World?

    A1: GIS is so smart that they’ve named whole industries after it. Smart Cities, Smart Homes, Smart Cars and Smart & Final. You can put Smart in front of anything and you sound smart. So in the spirit of Smart being smarter, we are using the power word, “Enabling” with Smart + World. You’d better believe we’ll also be mentioning Smart Maps later.

    Q2: How does the Internet of Things (IoT) work and how does it connect and integrate with GIS?

    A2: IoT is the new hotness. We looked at the Gartner Hype Cycle and saw that much of our previous GIS integration buzz words were dropping off. IoT should be on the Hype Cycle for years to come so it’s safe. Basically we’ll show you how to connect to things that you don’t care about with APIs that are difficult to use. It will be a fun time. Your Washing Machine is so in need of being a dot on a map.

    Q3: What is the Esri Map Book and how should I use it?

    A3: The Map Book is something we’ve given out every year for just about forever. You put the book in your suitcase where it gets bent during transport back to your GIS cube. Then you stick it on your shelf where it sits next to all your other Map Books as a badge of honor to show your peers that you collect Esri Map Books.

    Q4: Is ArcGIS too complicated for a small nonprofit to understand and use?

    A4: Yes of course. It’s too complicated for GIS users to understand and use. That is the whole point. If it was easy, then you wouldn’t feel like you’re getting value out of it. Super Mario Bros. was hard to beat, but you kept at it. Same with ArcGIS. Get that perfect score folks! This isn’t consumer software, it’s scientific.

    Q5: What is a GIS Hub?

    Q5: Our latest attempt to try and find ways to convince you to use ArcGIS Online. Rather than refine the message, we’re of the mindset to create new products and terms that confuse users. We call it the GIS Industrial Complex. GIS Hubs are collections of data on the web that don’t get updated very often.

    Q6: How is Esri doing?

    A6: Doing great thanks!

    Q7: What is the big idea with Web GIS?

    A7: What is the big idea with all these stupid questions?

    Q8: What does the future hold for GIS?

    A8: Lots of Microsoft Excel, DBF management and data silos.

    Q9: What are some of the important innovations for ArcGIS 10.5?

    A9: First, we’ll drop some big data terms on you. Second, we’ll create something called Insights which sounds cool because it’s “intuitive enterprise charting”. Third, we are reminding you we have a product called CityEngine. Fourth, we’ll introduce more wizard based tools that require no understanding of why you’re clicking next. Fifth, we have another Python library because that’s what you all want.

    Q10: Can you explain what a Web GIS is?

    A10: Um, GIS on the Web stupid.

    Q11: What is ArcGIS Open Data?

    A11: Something the boys in the lab cooked up. Allows both Esri and our business partners to check a box on responses to RFPs. You’re welcome…

    Q12: What is the best way to give feedback to Esri about software issues?

    A12: Complain on Twitter.

    Q13: What is smart mapping?

    A13: I swear to god that nobody asked this question.

    Q14: Why does Esri have two desktop solutions?

    A14: Because through feedback you let us know you liked it when we had ArcView 3.2a and ArcGIS Desktop 8.0.1 at the same time. We know you want something new but new sucks because it’s so different. So we’ll have two paths you’ll have to navigate. Keeps you busy doesn’t it? Think back to your old APRs and how much fun you had migrating them to MXDs. We have wizards that will simplify this but you still know it will be a nightmare. So enjoy ArcGIS for Desktop, we’re keeping it around for a long time.

    Q15: Will ArcMap be deprecated anytime soon?

    A15: Nope, it will be around for a long time, you’re still using ArcView 3.x on that WinXP computer aren’t you? Make a VM with Widows 10 and ArcGIS for Desktop 10.5 stand alone. You’ll be safe for a long time.

    Q16: What are the new capabilities coming with the summer release, ArcGIS Pro 1.3?

    A16: We’re getting close to having 15% of the functionality of ArcGIS Desktop. It takes time to migrate the kitchen sink.

    Q17: How would you describe ArcGIS for Server today?

    A17: Expensive

    Q18: What is ArcGIS Online?

    Q18: We’re still trying to figure that out. The plan is to eventually move everyone to ArcGIS Online named users for licensing. Be ready and embrace your new GIS Hub.

  • Esri Geodatabase Archive Updated for 10.4

    My archive of Esri Geodatabase versions since 8.3 adds the File and Personal Geodatabases for 10.4. If you ever need a native version of the Geodatabase, this is your archive.