Categories
Thoughts

Apple’s Digital Twin is All About Augmented Reality

Now before we get too far, Apple has not created anything close to a Digital Twin as we know them. But what they have done is created an easy way to import your building models into Apple Maps. Apple calls this their Indoor Maps program.

Easily create detailed maps of your indoor spaces and let visitors see where they are right in your app. Organizations with large public and private spaces like airports, shopping centers, arenas, hospitals, universities, and private office buildings can register for the Indoor Maps Program. Indoor maps are built using industry standard tools and require only your existing Wi-Fi network to enable GPS-level location accuracy so visitors can navigate your spaces with ease.

Victoria Airport in the Apple IMDF Sandbox

OK, so clearly this is all about navigation. How do I know where I am in a building and how do I get to a place I need to be. Of course, this is somewhat interesting on your iPhone or iPad in Apple Maps, but clearly, there is more to this than just how do I find the restroom on floor 10 of the bank tower.

To load your buildings in Apple you need to use Mapkit or Mapkit.js and convert your buildings into Indoor Mapping Data Format (IMDF). IMDF is actually a great choice because it is GeoJSON and working toward being an OGC standard (for whatever that is worth these days). I did find it interesting that Apple highlights the following as the use case for IMDF:

  • Indoor wayfinding
  • Indoor routing
  • Temporal constraints
  • Connectivity amongst mapped objects
  • Location-based services
  • Query and find by location functionality

If you’re familiar with GeoJSON, IMDF will look logical to you:

{
  "id": "11111111-1111-1111-1111-111111111111",
  "type": "Feature",
  "feature_type": "building",
  "geometry": null,
  "properties": {
    "category": "parking",
    "restriction": "employeesonly",
    "name": {
      "en": "Parking Garage 1"
    },
    "alt_name": null,
    "display_point": {
      "type": "Point",
      "coordinates": [1.0, 2.0]
    },
    "address_id": "2222222-2222-2222-2222-222222222222"
  }
}

I encourage you to review the IMDF docs to learn more but we’re talking JSON here so it’s exactly how you’d expect it to work.

Because IMDF buildings are generalized representations of the real-world data, this isn’t actually a Digital Twin. It also means that you need to do some things to your files before converting them to IMDF. Autodesk, Esri, and Safe Software all support IMDF so you should be able to use their tools to handle the conversions. I’ve done the conversion with Safe FME and it works very well and probably the best way to handle this. In fact, Safe has an IMDF validator which does come in handy for sure.

Safe FME support of IMDF

What does make moving your buildings to Apple’s Indoor platform is the new iPhone 12 and iPad Pro LiDAR support. This brings out some really great AR capabilities that become enabled with Apple’s devices. As I said last week, the LiDAR support in the current devices is more about getting experience with LiDAR use cases than actual LiDAR use. This is all about eventual Apple Glass (and Google Glass too) support and the AR navigation that can be done when you have hyper-accurate indoor models in your mapping software.

I’ve been dusting off my MapKit skills because I think not only is this capability useful for many companies but it really isn’t that hard to enable. I am spending some time thinking about how to use the extension capability of IMDF to see how IoT and other services can be brought in. Given the generalized nature of IMDF, it could be a great way to allow visualizing IoT and other services without the features of a building getting in the way. Stay tuned!

Categories
Thoughts

The iPhone 12 Pro LiDAR Scanner is the Gateway to AR, But Not in the Way You Think

I’m sure everyone knows about it by now, the iPhone 12 Pro has a LiDAR scanner. Apple touts it to help you take better pictures in low light and do some rudimentary AR on the iPhone. But, what this scanner does today isn’t where the power will be tomorrow.

Apple cares a ton about photo quality, so a LiDAR scanner helps immensely with taking these pictures. If there is one reason today to have that scanner, it is for pictures. But the real power of the scanner is for AR. And AR isn’t ready today, no matter how many demos you see in Apple’s event. Holding up an iPhone and seeing how big a couch in your room is interesting, just as interesting as using your phone to find the nearest Starbucks.

Apple has spent a lot of time working on interior spaces in Apple Maps. They’ve also spent a ton of time working on sensors in the phone for positioning inside buildings. This is all building to an AR navigation space inside public buildings and private buildings in which owners share their 3D plans. But what if hundreds of millions of mobile devices could create these 3D worlds automatically as they go about their business helping users find that Starbucks?

The future is so bright though with this scanner. It helps Apple and developers get familiar with what LiDAR can do for AR applications. This is critically important on the hardware side because Apple Glass, no matter how little is known about it, is the future for AR. Same with Google Glass too, the eventual consumer product (ignoring the junk that the first Google Glass was) of these wearable AR devices will change the world, not so much in that you’ll see an arrow as you navigate to the Starbucks, but give you the insight into smart buildings and all the IoT devices that are around.

The inevitable outcome is in the maintenance of smart buildings

Digital Twins are valuable when they link data feeds to a 3D world that can be interrogated. But the real value comes when those 3D worlds can be leveraged using Augmented Reality to give owners, maintenance workers, planners, engineers, and tenants the information they need to service their buildings and improve the quality of building maintenance. The best built LEED building is only as good as the ongoing maintenance put on it.

The iPhone 12 Pro and the iPad Pro that Apple has released this year both have LiDAR to improve their use with photo taking and rudimentary AR, but the experience gained seeing the real-world use of consumer LiDAR in millions of devices will bring great strides to making these Apple/Google Glass devices truly usable in real-world use. I’m still waiting to get my iPhone 12, but my wife’s arrived today. I’m looking forward to seeing what the LiDAR can do.

Categories
Thoughts

The iPhone U1 UWB Chip, Digital Twins and Data Collection

Oddly enough the biggest news this week from the iPhone 11 introduction by Apple barely got any play. In fact, on the iPhone 11 Pro website, you have to scroll past Dog Portrait mode to get any information about it. Apple describes the U1 chip thusly:

The new Apple‑designed U1 chip uses Ultra Wideband technology for spatial awareness — allowing iPhone 11 Pro to understand its precise location relative to other nearby U1‑equipped Apple devices.4 It’s like adding another sense to iPhone, and it’s going to lead to amazing new capabilities. With U1 and iOS 13, you can point your iPhone toward someone else’s, and AirDrop will prioritize that device so you can share files faster.4 And that’s just the beginning.

https://www.apple.com/iphone-11-pro/

Makes sense right? A better way to AirDrop. But there is so much more there, “precise location relative to other nearby equipped Apple Devices“. But what is UWB and why does it matter? The UWB Alliance says:

UWB is a unique radio technology that can use extremely low energy levels for short-range, high-bandwidth communications over a large portion of the radio spectrum. Devices powered by a coin cell can operate for a period of years without recharge or replacement. UWB technology enables a broad range of applications, from real-time locating and tracking, to sensing and radar, to secure wireless access, and short message communication. The flexibility, precision and low-power characteristics of UWB give it a unique set of capabilities unlike any other wireless technology.

So that’s really interesting, low energy use, high bandwidth and is very secure. I thought Jason Snell did a great job looking into the U1 on Six Colors:

From raw data alone, UWB devices can detect locations within 10 centimeters (4 inches), but depending on implementation that accuracy can be lowered to as much as 5 millimeters, according to Mickael Viot, VP of marketing at UWB chipmaker Decawave.

That’s pretty amazing. Basically it takes what makes Bluetooth LE great for discover, secures it and then makes it faster and more accurate. So we can see the consumer use cases for UWB, sharing files and finding those tiles we’ve heard so much about. But where this gets very interesting for our space is for data collection and working inside digital twins. You can already see the augmented reality use case here. A sensor has gone bad in a building, I can find it now with millimeter accuracy. But it’s not just what direction it’s how far. UWB uses “time of flight” to pinpoint location (measuring the time of signal to gauge distance), enabling it to know how far away it is. Just knowing a sensor is ahead of you is one thing, but knowing it is 20 feet away, that’s really a game changer.

You can see this through a little known app Apple makes called Indoor Survey. Small side note, back in late 2015 I blogged about Apple’s Indoor Positioning App which ties into all this. Where you really see this use is when you go to the signup page see how data is brought into this app using a standard called Indoor Mapping Data Format. Indoor Mapping Data Format (IMDF) provides a generalized, yet comprehensive data model for any indoor location, creating a basis for orientation, navigation and discovery. IMDF is output as an archive of GeoJSON files.  Going to the IMDF Sandbox really shows you what this format is about.

Apple’s IMDF Sandbox

Basically you see a map editor that allows you to really get into how interiors are mapped and used. So Apple iPhone 11 UWB devices can help place themselves more accurately on maps and route users around building interiors. Smart buildings get smarter by the devices talking to each other. Oh and IMDF, Apple says, “For GIS and BIM specialists, there is support for IMDF in many of your favorite tools.“. I will need to spend a bit more time with IMDF but its basically GeoJSON objects so we already know how to use it.

The thing about GPS data collection is it works great outdoors, but inside it is much harder to get accuracy, especially when you need it. With Indoor Survey, devices can collect data much more accurately indoors because they know exactly where they are. If you’ve ever used Apple Maps in an airport and seen how it routes you from gate to gate, you get an idea how this works. But with UWB, you go from foot accuracy to sub centimeter. That’s a big difference.

Now we’re a long way away from UWB being ubiquitous like Bluetooth LE is. Right now as far as I can tell, only Apple has UWB chips in their devices and we don’t know how compatible this all is yet. But you can see how the roadmap is laid out here. UWB, GeoJSON and an iPhone 11. Devices help each other get better location and in turn make working with Digital Twins and data collection so much easier.

Categories
Thoughts

Friday Links

Back in the day I used to always have a Friday link blog post and I’ve noticed I’ve been doing a lot more reading so it just feels right to visit this back.

  1. Apple owes everyone an apology and it should start with me, specifically – You can’t but own one of the latest Apple MacBook Pros and not hate the keyboard. I’ve been “lucky” enough to experience all three versions of it. The latest is on my new laptop from Spatial Networks which I have to admit feels the best of any of them but I’m just waiting for the “f” key to stop working like it has on all my other ones. I used to enjoy typing on MacBooks but not anymore. The thing is they keep trying to fix broken and not just go back to something that worked.
  2. Electric scooters have zipped by docked bikes in popularity – Here in Tempe, AZ we get to see all of them. Bird, Uber/JUMP, Lime, Razor and various ones I can’t even tell the brand. Their are like lice on every corner just fallen over and broken. I noted in St. Pete that they didn’t have any scooters and it was surreal walking around on sidewalk without jumping out of the way of some idiot on a scooter. I don’t understand the business model but I hate to say they are here to stay.
  3. A look at IBM S/360 core memory: In the 1960s, 128 kilobytes weighed 610 pounds – I mean the title says it all. These things were HUGE! X and Y wires. It’s madness but apparently it worked!
  4. Notre Dame Cathedral will never be the same, but it can be rebuilt – Thanks to all the pictures and Lidar imagery, the Catheral will be rebuilt and be very close to original. But…

While architects have enough detailed information about the cathedral to pull off a technically very precise reconstruction, the craftsmanship is unlikely to be the same. Today, the stone that makes up the cathedral would be cut using machinery, not by hand by small armies of stonemasons as in the 12th century. “Nineteenth-century and 20th-century Gothic buildings always look a little dead, because the stone doesn’t bear the same marks of the mason’s hand,” Murray told Ars Technica.

Still I look forward to watching this happen.

Categories
Thoughts

Apple Gets Ready to Release Indoor Positioning Service

Over the weekend Apple apparently pushed out an indoor positioning appIndoor Survey’ into the iOS App Store.

“By dropping ‘points’ on a map within the Survey App, you indicate your position within the venue as you walk through,” reads the app description. “As you do so, the indoor Survey App measures the radio frequency (RF) signal data and combines it with an iPhone’s sensor data. The end result is indoor positioning without the need to install special hardware.”

Interesting in the sense it appears to be an app that stores can use to map their interiors with iOS devices.  It’s not a crowd sourced indoor mapping application.  This dovetails nicely with the other announcement this morning about their new Maps Indoor service.

For now, Apple is focusing its efforts on a handful of venues that meet specific criteria. These requirements include:

  • The venue must be accessible to the general public
  • Only locations that draw more than a million visitors per year
  • Apple requires “complete, accurate, and scaled reference maps” for consideration
  • The venue must have Wi-Fi throughout, and an official app available on the App Store

The groundwork is set for Apple to start mapping interiors of these large open venues.  But with an app and an iPhone, clearly Apple is planning to scale this out to just about every indoor location.  I suspect we’ll see stadiums, amusement parks and other entertainment venues appear first over the next year.

Categories
Thoughts

Ed Parsons Confirms Apple Still Exists

Link – Visiting the Mothership – despite Microsoft !!

Ed was lucky enough to make it by Apple’s Campus and confirm that they are still around. If you remember last week there was some “discussion” about the digital imagery that Microsoft was using for their Virtual Earth web mapping program. Ed makes the point that using “free” imagery from USGS can result in problems such as this. I agree with that, but I also think they could have done more to mitigate the problem. We’ve all used “dated” imagery in our cartographic maps before and usually we usually put a note somewhere on the map letting the reader know the source and date of that imagery. Google Maps does this and I think Microsoft should start doing it also. DOQQs aren’t bad, but we all know they are outdated compared to most satellite imagery. It might have been a good idea for Microsoft to send some of their programmers to “GIS Camp” at the ESRI User Conference to learn more about GIS.

Categories
Thoughts

Ed Parsons Confirms Apple Still Exists

Link – Visiting the Mothership – despite Microsoft !!

Ed was lucky enough to make it by Apple’s Campus and confirm that they are still around. If you remember last week there was some “discussion” about the digital imagery that Microsoft was using for their Virtual Earth web mapping program. Ed makes the point that using “free” imagery from USGS can result in problems such as this. I agree with that, but I also think they could have done more to mitigate the problem. We’ve all used “dated” imagery in our cartographic maps before and usually we usually put a note somewhere on the map letting the reader know the source and date of that imagery. Google Maps does this and I think Microsoft should start doing it also. DOQQs aren’t bad, but we all know they are outdated compared to most satellite imagery. It might have been a good idea for Microsoft to send some of their programmers to “GIS Camp” at the ESRI User Conference to learn more about GIS.