Thread by @pesarlin

🚨 This week at #ICCV2021, check out "Pixel-Perfect Structure-from-Motion with Featuremetric Refinement" w/ @PhilippCSE V. Larsson @mapo1
➡️ oral & *best student paper award*
Website: psarlin.com/pixsfm
Paper: arxiv.org/abs/2108.08291
Video:
thread ⬇️

We improve the accuracy of Structure-from-Motion (= camera poses & 3D points), making COLMAP subpixel accurate & closer to Lidar performance. This enables:
➡️ accurate visual localization of new images
➡️ mapping with fewer images - critical for large-scale crowd-sourced AR 😎
2/

We align dense deep features with featuremetric optimization in 2 steps
1️⃣ Before SfM: keypoint adjustment
2️⃣ After SfM: bundle adjustment
This works well with many dense CNNs (even VGG ImageNet) & sparse local features (SuperGlue!) and suitable for both global/incremental SfM
3/

Pixel-perfect SfM uses dense features but only needs *local* image information ➡️ scales to large scenes, memory-efficient & only <20% overhead to COLMAP
🎯 All of this with 2.5x more accurate camera localization at 1mm!
🏙🌃 And it is robust to light changes for day-night SfM
4/

➡️ Don't be shy & come have a chat with us at our #ICCV2021 poster in Session 5, Today 4PM EDT 1 & Thursday 9AM EDT - we're excited to tell you more about it 😁
➡️ The code will be released very soon!
➡️ Work done at @CSatETH @ETH_en @Microsoft
5/5