Discover and read the best of Twitter Threads about #GeometricDeepLearning

Most recents (4)

Q. What does Noether’s theorem tell us about the “geometry of deep learning dynamics”?
A. We derive Noether’s Learning Dynamics and show:
”SGD+momentum+BatchNorm+weight decay” = “RMSProp" due to symmetry breaking!

w/ @KuninDaniel
#NeurIPS2021 Paper: bit.ly/3pAEYdk
1/
@KuninDaniel Geometry of data & representations has been central in the design of modern deepnets.
e.g., #GeometricDeepLearning arxiv.org/abs/2104.13478 by @mmbronstein, @joanbruna, @TacoCohen, @PetarV_93

What are the geometric design principles for “learning dynamics in parameter space”?
2/
We develop Lagrangian mechanics of learning by modeling it as the motion of a particle in high-dimensional parameter space. Just like physical dynamics, we can model the trajectory of discrete learning dynamics by continuous-time differential equations.
3/
Read 10 tweets
Very glad to share our latest review on #geometricdeeplearning in molecular sciences, with special emphasis on drug discovery, quantum chemistry and synthesis prediction. 1/6
We put strong focus on the various existing molecular representations and their individual symmetries and advantages for different modeling applications. 2/6 Image
We discuss the major network architectures, such as RNNs, GNNs, 3D CNNs and Transformers, their applications on molecular systems, and the relevant equivariances they incorporate. 3/6 Image
Read 7 tweets
The recording of my #inaugural talk at @imperialcollege is now available online



This geometric view on deep learning is the convergence of many old and recent research threads and joint work with @joanbruna @PetarV_93 and @TacoCohen
Joan's 2014 paper on spectral #graphneuralnetworks was my inspiration to get into this field and write our paper with him, @ylecun @trekkinglemon and Arthur Szlam that popularized the term #geometricdeeplearning
Some of the works I mention are due to my amazing students and colleagues @jonathanmasci Davide Boscaini @EmanueleRodola @befcorreia Pablo Gainza @FreyrSverrisson @emaros96 @ffabffrasca @gbouritsas @frederickmonti @b_p_chamberlain @aittalam Kirill Veselkov Guadalupe Gonzalez
Read 10 tweets
In summer 2020, I started writing a blog on #GraphML in @TDataScience

towardsdatascience.com/graph-deep-lea…

It was a new and rewarding experience, from which I learned a lot. I was surprised that such a technical topic would attract >200K views and >6K claps in less than half a year. Image
I am grateful to all the readers and colleagues whom I used as "guinea pigs" to proof-read my posts especially @frederickmonti @emaros96 @b_p_chamberlain @ffabffrasca @gbouritsas @__lucab @fhuszar and to @kmborgwardt @thomaskipf @guennemann whose comments helped improve them
The first post was based on a seminar I gave for the @IEEEsps on the successes and challenges of #GraphML and #geometricdeeplearning

towardsdatascience.com/deep-learning-…
Read 9 tweets

Related hashtags

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!