Thread by @jenner_erik

Excited to announce that my paper with @maurice_weiler on Steerable Partial Differential Operators has been accepted to #iclr2022! Steerable PDOs bring equivariance to differential operators. Preprint: arxiv.org/abs/2106.10163 (1/N)

Equivariance has become a popular topic in deep learning, but it has played a huge role in physics long before that. So wouldn't it be great if we could bring equivariant deep learning and physics closer together, to transfer more ideas? (2/N)

The issue is that equivariant NNs are usually defined using convolutions, whereas physics is described in the language of equivariant partial differential operators (PDOs). Our paper bridges that gap by developing the theory of equivariant PDOs in a deep learning setting. (3/N)

Equivariance can be described using feature fields: if the input is transformed (e.g. by a rotation), then the features should also be transformed in a certain way. E.g. in a vector field, each vector will be rotated, which distinguishes it from a stack of scalar fields. (4/N)

PDOs can map between these feature fields. For example, the gradient maps from scalar to vector fields, whereas the divergence maps from vector to scalar fields. In general, we can combine partial derivatives in lots of ways to get many more PDOs than just these common ones (5/N)

Not all PDOs are equivariant under all symmetries (e.g. the curl is not equivariant under reflections). Our main result is a steerability constraint for PDOs, which precisely characterizes the space of equivariant PDOs (for any group and representation). (6/N)

This is very reminiscent of the existing steerability constraint for kernels, which characterizes the kernels that make a convolution equivariant. We exploit this similarity to show how existing solutions for steerable kernels can be transferred to PDOs. (7/N)

We apply our procedure to subgroups of O(2) and O(3), in order to describe complete bases of equivariant PDOs for these groups and their representations. This allows us to implement equivariant PDOs by learning coefficients for these bases. (8/N)

In practice, these equivariant PDOs can then be discretized. We benchmark three different methods on regular grids (finite differences, RBF-FD, and derivatives of Gaussians). Our derivation of equivariant PDOs is entirely independent of the discretization method. (9/N)

Steerable PDOs generalize the previously proposed PDO-eConvs (arxiv.org/abs/2007.10408). PDO-eConvs are equivariant PDOs between certain types of feature fields (regular ones). But this doesn't cover other important types, such as vector fields, nor PDOs like the gradient. (10/N)

These PDOs (gradient, divergence, curl, ...) are the ones that occur all the time in physics, so they are particularly important for connecting equivariant DL to physics. Steerable PDOs are provably the most general equivariant PDOs between feature spaces. (11/N)

In addition to introducing Steerable PDOs, we introduce a framework that unifies equivariant PDOs with equivariant kernels, using convolutions with Schwartz distributions. We prove that this covers all translation-equivariant linear continuous maps between feature spaces. (12/N)

Our code is available at github.com/ejnnr/steerabl…. We extend the amazing e2cnn library by @_gabrielecesa_, and our work has already been merged. So you can now easily use steerable PDOs, and even combine them seamlessly with steerable convolutions in a single network! (13/13)