Q. What does Noether’s theorem tell us about the “geometry of deep learning dynamics”?
A. We derive Noether’s Learning Dynamics and show:
”SGD+momentum+BatchNorm+weight decay” = “RMSProp" due to symmetry breaking!

w/ @KuninDaniel
#NeurIPS2021 Paper: bit.ly/3pAEYdk
1/ @KuninDaniel Geometry of data & representations has been central in the design of modern deepnets.
e.g., #GeometricDeepLearning arxiv.org/abs/2104.13478 by @mmbronstein, @joanbruna, @TacoCohen, @PetarV_93

What are the geometric design principles for “learning dynamics in parameter space”?
2/