Craziest DM I ever received, from a VP at a global retailer: "Our app is shit and we know it's shit". I met her for coffee and she asked me if I could solve the biggest unsolved problem in retail.
This is a deep dive into why and how Hyper built a 1m-accurate indoor GPS.
This DM arrived in 2017. My outdoor AR navigation demos had just gone viral, and my new open-source project for Apple had elevated me to be the top trending iOS developer on GitHub.
The retail exec told me they wanted to bring indoor maps and navigation to their retail stores, so customers could find what they’re looking for, and they could pop up relevant promotions along the way. It turns out that every office, university, events venue, hotel, airport, warehouse, factory — basically everywhere indoors have some need to navigate people around, provide relevant information, and improve efficiency.
I assumed this was a solved problem. No. They do have maps on their app, but they aren’t able to navigate people because GPS doesn’t work indoors. They tried every solution out there to provide the blue dot, but nothing worked.
I did know something about maps and location already — the first startup I worked at built an early version of Pokemon Go. I’d been tasked with generating the gamified maps, and populating the monsters and rewards. So I knew a bit about maps, coordinates and GPS — and monster training. But indoor navigation was new to me.
Over the years, I’ve slowly become an expert in this, so let me explain. For indoor navigation to work well, the blue dot location needs to be 2x as accurate as a strong GPS signal. An aisle in a store is usually about 2 meters wide, so an accuracy wider than 2 meters would be fixing you in the wrong aisle.
There are many research studies aimed at solving this, and Apple and Google have made acquisitions to help them in this area over the years. There were also many startups who claimed to have solutions, but when I spoke to their customers, I discovered that they weren’t happy with anything they’d tried:
- Bluetooth beacons. Install thousands of these small sensors, which are a bit like AirTags, and use them for triangulation. But the bluetooth signals are noisy, making the location about 5 meters accurate, so it would jump you between multiple aisles in a store. Plus, lots of infrastructure to maintain.
- WiFi. More promising than beacons, because every business has WiFi installed already. But the same radio signals problem means the location isn’t accurate enough.
- Magnetomers, which use the earth’s magnetic field. This one sounded more promising. But it takes several minutes of walking around until it will give you a “blue dot”. So this was a bad user experience.
- Computer Vision, which works like Google Street View. The user holds up their phone to scan the environment, it recognises their surroundings and locks them in. But this is clunky for the user, and they need to do this repeatedly every time they want a location update. Once again, bad user experience.
Here’s an example of Apple’s own accuracy using WiFi. (I’ll show our own performance on these same sessions further down).
This is a deep dive into why and how Hyper built a 1m-accurate indoor GPS.
This DM arrived in 2017. My outdoor AR navigation demos had just gone viral, and my new open-source project for Apple had elevated me to be the top trending iOS developer on GitHub.
The retail exec told me they wanted to bring indoor maps and navigation to their retail stores, so customers could find what they’re looking for, and they could pop up relevant promotions along the way. It turns out that every office, university, events venue, hotel, airport, warehouse, factory — basically everywhere indoors have some need to navigate people around, provide relevant information, and improve efficiency.
I assumed this was a solved problem. No. They do have maps on their app, but they aren’t able to navigate people because GPS doesn’t work indoors. They tried every solution out there to provide the blue dot, but nothing worked.
I did know something about maps and location already — the first startup I worked at built an early version of Pokemon Go. I’d been tasked with generating the gamified maps, and populating the monsters and rewards. So I knew a bit about maps, coordinates and GPS — and monster training. But indoor navigation was new to me.
Over the years, I’ve slowly become an expert in this, so let me explain. For indoor navigation to work well, the blue dot location needs to be 2x as accurate as a strong GPS signal. An aisle in a store is usually about 2 meters wide, so an accuracy wider than 2 meters would be fixing you in the wrong aisle.
There are many research studies aimed at solving this, and Apple and Google have made acquisitions to help them in this area over the years. There were also many startups who claimed to have solutions, but when I spoke to their customers, I discovered that they weren’t happy with anything they’d tried:
- Bluetooth beacons. Install thousands of these small sensors, which are a bit like AirTags, and use them for triangulation. But the bluetooth signals are noisy, making the location about 5 meters accurate, so it would jump you between multiple aisles in a store. Plus, lots of infrastructure to maintain.
- WiFi. More promising than beacons, because every business has WiFi installed already. But the same radio signals problem means the location isn’t accurate enough.
- Magnetomers, which use the earth’s magnetic field. This one sounded more promising. But it takes several minutes of walking around until it will give you a “blue dot”. So this was a bad user experience.
- Computer Vision, which works like Google Street View. The user holds up their phone to scan the environment, it recognises their surroundings and locks them in. But this is clunky for the user, and they need to do this repeatedly every time they want a location update. Once again, bad user experience.
Here’s an example of Apple’s own accuracy using WiFi. (I’ll show our own performance on these same sessions further down).
After seeing that nothing out there worked, I decided to start Hyper, a startup with the mission to build accurate indoor navigation, and scale it to a billion people.
I’ll skip us forward — past the years of fundraising, hiring, R&D, failed pilots, successful pilots — and tell you how we built a 1-meter accurate indoor GPS.
Requirements for a working solution:
- The solution should scale, with no new hardware install
- The user should not have to think about blue-dot accuracy; the technology should “just work”.
- The user’s location should appear instantly, without them having to scan the environment or walk around.
- It’s fine to start with a lower accuracy blue dot, such as 3 meters, as long as it quickly hones in to the desired 1 meter accuracy within seconds.
- It should stay accurate and reliable throughout the journey.
- And it should provide updated navigational routes, turn-by-turn directions, and AR navigation.
This sounds like a tall order — and it is. We persevere.
We decided to use WiFi as a foundation to give us that initial blue dot.
WiFi location
We start by building a map of signal strength, and how it flows across a venue. This is just like the bars of signal strength you see on your phone, except it’s measuring it for each individual access point.
To collect this wifi data, we would send a member of our team to do an on-site survey. This survey involves walking around a venue, while the phone collects wifi data at each position. The large black dots you see on this visual are where we plotted our position. We’re using our algorithms to track precise motion between each truth point, so we can collect as much WiFi data as possible.
I’ll skip us forward — past the years of fundraising, hiring, R&D, failed pilots, successful pilots — and tell you how we built a 1-meter accurate indoor GPS.
Requirements for a working solution:
- The solution should scale, with no new hardware install
- The user should not have to think about blue-dot accuracy; the technology should “just work”.
- The user’s location should appear instantly, without them having to scan the environment or walk around.
- It’s fine to start with a lower accuracy blue dot, such as 3 meters, as long as it quickly hones in to the desired 1 meter accuracy within seconds.
- It should stay accurate and reliable throughout the journey.
- And it should provide updated navigational routes, turn-by-turn directions, and AR navigation.
This sounds like a tall order — and it is. We persevere.
We decided to use WiFi as a foundation to give us that initial blue dot.
WiFi location
We start by building a map of signal strength, and how it flows across a venue. This is just like the bars of signal strength you see on your phone, except it’s measuring it for each individual access point.
To collect this wifi data, we would send a member of our team to do an on-site survey. This survey involves walking around a venue, while the phone collects wifi data at each position. The large black dots you see on this visual are where we plotted our position. We’re using our algorithms to track precise motion between each truth point, so we can collect as much WiFi data as possible.
Once we started to work with large global retailers, we needed a better way to scale this process. Ideally, the staff at the store could do this themselves — rather than us flying our team across the world — and then we could lower the cost and timelines. So we built a self-serve version of our survey app, with a tutorial mode designed for beginners. Over time, we collected millions of data points, and so we were able to develop an algorithm which would auto-correct mistakes. In other words, if the surveyor accidentally placed their ground-truth location in the wrong place on the map, we could use our algorithms to detect it, and correct it.
So now we have WiFi, and with and our efforts on producing a high quality survey, we have the best WiFi positioning available. With WiFi on its own, it’s achieving 3 meter accuracy. This is a great foundation to build on.
WiFi + Motion data
To refine this down to 1-meter accuracy, we realised that we could combine WiFi with the same technology behind self-driving cars and robotics: a motion system called SLAM (Simultaneous Localization and Mapping). SLAM uses the accelerometer, gyroscope and camera system to understand precise device motion. Imagine a car driving through a tunnel, using the motion since its last GPS ping to keep location accurate until it comes out the other side. On a phone, this technology is very reliable, and measures device motion with high precision.
But SLAM is measuring motion within its own coordinate space, it’s not aligned with the real world. SLAM tracks the user’s relative motion, like “moved forward 2 meters, then turned left”, but does “forward” mean “north”, or some other direction? It’s not calibrated, so it could mean any location, any direction. We can’t rely on the compass to help us out with this, because phone compasses are notoriously incorrect — everyone knows the frustration of being sent the wrong way down a street.
So our job was to align this motion data with the triangulation data we were receiving from WiFi. We designed an algorithm that could simulate every possibility, filter the unlikely scenarios, and hone in your location, using WiFi as an anchor.
So WiFi gives us the initial blue dot, SLAM gives us motion, and as the user starts walking and we receive more data, our algorithms can refine location accuracy down to a consistent 1-meter accuracy.
We’ve tested these algorithms in many locations, on hundreds of hours of ground-truth data:
So now we have WiFi, and with and our efforts on producing a high quality survey, we have the best WiFi positioning available. With WiFi on its own, it’s achieving 3 meter accuracy. This is a great foundation to build on.
WiFi + Motion data
To refine this down to 1-meter accuracy, we realised that we could combine WiFi with the same technology behind self-driving cars and robotics: a motion system called SLAM (Simultaneous Localization and Mapping). SLAM uses the accelerometer, gyroscope and camera system to understand precise device motion. Imagine a car driving through a tunnel, using the motion since its last GPS ping to keep location accurate until it comes out the other side. On a phone, this technology is very reliable, and measures device motion with high precision.
But SLAM is measuring motion within its own coordinate space, it’s not aligned with the real world. SLAM tracks the user’s relative motion, like “moved forward 2 meters, then turned left”, but does “forward” mean “north”, or some other direction? It’s not calibrated, so it could mean any location, any direction. We can’t rely on the compass to help us out with this, because phone compasses are notoriously incorrect — everyone knows the frustration of being sent the wrong way down a street.
So our job was to align this motion data with the triangulation data we were receiving from WiFi. We designed an algorithm that could simulate every possibility, filter the unlikely scenarios, and hone in your location, using WiFi as an anchor.
So WiFi gives us the initial blue dot, SLAM gives us motion, and as the user starts walking and we receive more data, our algorithms can refine location accuracy down to a consistent 1-meter accuracy.
We’ve tested these algorithms in many locations, on hundreds of hours of ground-truth data:
As the icing on top, precise location feeds into our other area of research: AR navigation. We’ve refined this user experience to be best-in-class, driven by Hyper-accurate location.
Most of Hyper’s story still hasn’t been told. One day I’ll explain how we worked secretively for years with the worlds largest retailers, rolling out to many thousands of shoppers, who were all able to navigate with that precise blue dot.
These breakthroughs could only have been achieved by a small, visionary startup with high conviction. We’ve achieved the first half of our mission — to build accurate indoor navigation. Now for the second half: how do we scale this to a billion people?
This is a very different challenge: we’ve learnt it requires deep skills in enterprise-scale distribution, being able to handle the complex commercial relationships with large organisations.
Over the years I’ve watched other startups go on this journey — building their technology independently, in a way that only they could have done, then joining a much larger organisation with the commercial muscle and resource to sell to enterprise customers. And that’s the journey we intend to go on next.
I’m now busy meeting many businesses to figure out which of them can help us take everything we’ve built, and scale to a billion people.
Most of Hyper’s story still hasn’t been told. One day I’ll explain how we worked secretively for years with the worlds largest retailers, rolling out to many thousands of shoppers, who were all able to navigate with that precise blue dot.
These breakthroughs could only have been achieved by a small, visionary startup with high conviction. We’ve achieved the first half of our mission — to build accurate indoor navigation. Now for the second half: how do we scale this to a billion people?
This is a very different challenge: we’ve learnt it requires deep skills in enterprise-scale distribution, being able to handle the complex commercial relationships with large organisations.
Over the years I’ve watched other startups go on this journey — building their technology independently, in a way that only they could have done, then joining a much larger organisation with the commercial muscle and resource to sell to enterprise customers. And that’s the journey we intend to go on next.
I’m now busy meeting many businesses to figure out which of them can help us take everything we’ve built, and scale to a billion people.
Many people are asking what comes next: we’re seeking acquisition opportunities, to help us take this technology to a billion people.
My DMs are open, as always. Maybe we can make this the start of the next act!
My DMs are open, as always. Maybe we can make this the start of the next act!
• • •
Missing some Tweet in this thread? You can try to
force a refresh
