With apologies to @elonmusk and the @tesla #autopilot team, we believe that Skydio 2 is the most advanced autonomous device of any kind that has ever been commercialized. Take a look under the hood and let us know if you agree. bit.ly/skydio2autonomy.
To keep a cinematic framing of the biker while dodging obstacles at high speeds, Skydio 2 has to estimate its own motion, build an obstacle map, estimate the biker’s path, and handle the aerodynamics of fast flight, all in real time onboard the drone.
Each of Skydio’s six 4K navigation cameras has a super-fisheye lens with 200° field-of-view, so the autonomy engine can confidently guide Skydio 2 in any direction.
To combine the cameras to create a 360 view of the scene, we build a detailed camera warping model and then calibrate them in the factory and in flight.
Camera data is incredibly rich but also presents very difficult challenges like thin objects, sun glare, motion blur, reflections, vibrations, dirt, smudges, fog. We handle these challenges by carefully combining 3D geometric modeling with deep learning. 
In June 2018, we believe we were the first to ever fly an obstacle avoidance system based on deep learning in a complex environment.
Now that system has shipped with the Skydio 2 and produces 3D maps that look more like they come from a LIDAR system (too costly and bulky to fly) than from visual sensors.
We use deep networks and geometric reasoning to track multiple objects in a scene, estimating each of their positions, trajectory, and then solving an optimization problem to decide where each object has moved in every new frame.
The most important signal for getting cinematic footage is not where the subject is, but where it's going. Our motion planner considers the future motion of the subject and decides on a flight plan to get the best shot given the nearby environment.
