Introducing DeltaConv: an INSANE convolution layer for point clouds. It's INtrinsic, Surface-first, ANisotropic, and... erm... Easy to use! Coming to #SIGGRAPH2022 💾code 📜paper and 🧙summary at rubenwiersma.nl/deltaconv 1/8
Anisotropic convolution is a central building block of CNNs, but challenging to transfer to surfaces, because there's no global coordinate system to align our filters. 2/8
The missing global coordinate system is addressed in differential geometry by using coordinate-independent operators, such as the Laplacian ❤️ but a downside of only using the Laplacian is that it is isotropic 💔 3/8
This has also been addressed before 🤩 Anisotropic diffusion breaks up the Laplacian into the gradient and divergence and applies a non-linearity on the vector field in-between. On images, that looks like this (courtesy NASA) 4/8
DeltaConv builds on this idea by learning to combine geometric operators that map between scalars and vectors. 5/8
A simple ResNet with DeltaConv can approximate anisotropic diffusion, where other convolutions struggle. 6/8
DeltaConv is intrinsic, surface-first, anisotropic, easy to use, and - last, but not least - it achieves state-of-the-art results with a simple architecture on ModelNet40, ShapeNet, ScanObjectNN, and more💫 7/8
🤓Details 📰paper and 🐍code are available at rubenwiersma.nl/deltaconv 8/8
Work done at @TUDelft_CGV in collaboration with @ahmadnasikun, Elmar Eisemann and Klaus Hildebrandt 9/8
Share this Scrolly Tale with your friends.
A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.