Founder of Hyper @HyperARCo, “Indoor Location that actually works”. Built largest project for Apple ARKit, pioneered AR navigation, Mirador 🏳️🌈
Aug 4 • 5 tweets • 7 min read
Craziest DM I ever received, from a VP at a global retailer: "Our app is shit and we know it's shit". I met her for coffee and she asked me if I could solve the biggest unsolved problem in retail.
This is a deep dive into why and how Hyper built a 1m-accurate indoor GPS.
This DM arrived in 2017. My outdoor AR navigation demos had just gone viral, and my new open-source project for Apple had elevated me to be the top trending iOS developer on GitHub.
The retail exec told me they wanted to bring indoor maps and navigation to their retail stores, so customers could find what they’re looking for, and they could pop up relevant promotions along the way. It turns out that every office, university, events venue, hotel, airport, warehouse, factory — basically everywhere indoors have some need to navigate people around, provide relevant information, and improve efficiency.
I assumed this was a solved problem. No. They do have maps on their app, but they aren’t able to navigate people because GPS doesn’t work indoors. They tried every solution out there to provide the blue dot, but nothing worked.
I did know something about maps and location already — the first startup I worked at built an early version of Pokemon Go. I’d been tasked with generating the gamified maps, and populating the monsters and rewards. So I knew a bit about maps, coordinates and GPS — and monster training. But indoor navigation was new to me.
Over the years, I’ve slowly become an expert in this, so let me explain. For indoor navigation to work well, the blue dot location needs to be 2x as accurate as a strong GPS signal. An aisle in a store is usually about 2 meters wide, so an accuracy wider than 2 meters would be fixing you in the wrong aisle.
There are many research studies aimed at solving this, and Apple and Google have made acquisitions to help them in this area over the years. There were also many startups who claimed to have solutions, but when I spoke to their customers, I discovered that they weren’t happy with anything they’d tried:
- Bluetooth beacons. Install thousands of these small sensors, which are a bit like AirTags, and use them for triangulation. But the bluetooth signals are noisy, making the location about 5 meters accurate, so it would jump you between multiple aisles in a store. Plus, lots of infrastructure to maintain.
- WiFi. More promising than beacons, because every business has WiFi installed already. But the same radio signals problem means the location isn’t accurate enough.
- Magnetomers, which use the earth’s magnetic field. This one sounded more promising. But it takes several minutes of walking around until it will give you a “blue dot”. So this was a bad user experience.
- Computer Vision, which works like Google Street View. The user holds up their phone to scan the environment, it recognises their surroundings and locks them in. But this is clunky for the user, and they need to do this repeatedly every time they want a location update. Once again, bad user experience.
Here’s an example of Apple’s own accuracy using WiFi. (I’ll show our own performance on these same sessions further down).
After seeing that nothing out there worked, I decided to start Hyper, a startup with the mission to build accurate indoor navigation, and scale it to a billion people.
I’ll skip us forward — past the years of fundraising, hiring, R&D, failed pilots, successful pilots — and tell you how we built a 1-meter accurate indoor GPS.
Requirements for a working solution:
- The solution should scale, with no new hardware install
- The user should not have to think about blue-dot accuracy; the technology should “just work”.
- The user’s location should appear instantly, without them having to scan the environment or walk around.
- It’s fine to start with a lower accuracy blue dot, such as 3 meters, as long as it quickly hones in to the desired 1 meter accuracy within seconds.
- It should stay accurate and reliable throughout the journey.
- And it should provide updated navigational routes, turn-by-turn directions, and AR navigation.
This sounds like a tall order — and it is. We persevere.
We decided to use WiFi as a foundation to give us that initial blue dot.
WiFi location
We start by building a map of signal strength, and how it flows across a venue. This is just like the bars of signal strength you see on your phone, except it’s measuring it for each individual access point.
To collect this wifi data, we would send a member of our team to do an on-site survey. This survey involves walking around a venue, while the phone collects wifi data at each position. The large black dots you see on this visual are where we plotted our position. We’re using our algorithms to track precise motion between each truth point, so we can collect as much WiFi data as possible.
Jun 18, 2023 • 6 tweets • 3 min read
I’ve just finished watching Apple’s visionOS sessions. Some of these concepts and the way Apple has implemented the design is mind blowing 🤯
Here are 5 examples that I loved:
1. Eye zoom. Just look at _where_ on an image you want to zoom in, and pinch your fingers apart
2. "You had me at scrolling" - the famous quip someone made to Steve Jobs when he demoed the iPhone.
With visionOS, Apple uses your eyes and hands to control the entire interface in a new, intuitive way.