Our work has been published in @Nature!!

(G)NNs can successfully guide the intuition of mathematicians & yield top-tier results -- in both representation theory & knot theory.

dpmd.ai/nature-maths
arxiv.org/abs/2111.15161
arxiv.org/abs/2111.15323

See my 🧵 for more insight...
It’s hard to overstate how happy I am to finally see this come together, after years of careful progress towards our aim -- demonstrating that AI can be the mathematician’s 'pocket calculator of the 21st century'.

I hope you’ll enjoy it as much as I had fun working on it!
I was leading the GNN modelling on representation theory: working towards settling the combinatorial invariance conjecture, a long-standing open problem in the area.

My work earned me the co-credit of 'discovering math results', an honour I never expected to receive.
We showed that combining algorithmically-inspired GNNs and rudimentary explainability techniques enables mathematicians to make meaningful progress on such challenging tasks.

AI is well-positioned to help as meaningful patterns emerge only once inputs get very unwieldy in size.
To be clear: AI did _not_ discover the specific theorems and conjectures we expose here.

Mathematician experts did, _in synergy_ with predictions and interpretations derived from carefully-posed machine learning tasks. And this synergy is what makes the method so exciting, imho.
I’d like to deeply thank our collaborators: András Juhász, Marc Lackenby and Geordie Williamson. They successfully derived these results, put faith in our effort when we had no clear way to justify its promise, and remained a constant source of inspiration and encouragement.
Besides András, Marc & Geordie, our team comprised @ADaviesAI, Lars, Sam, Dan, @weballergy, Rich, @PeterWBattaglia @BlundellCharles @demishassabis @pushmeet

Thanks Alex, for asking me to join this effort during its formative stages and being a fantastic orchestrator throughout!
Last but not least: if you’d like to hear more about our work, I will be giving a keynote at the #AI4Science workshop @NeurIPSConf (ai4sciencecommunity.github.io).

Hope to see many of you there, and can’t wait to see the follow-up work that comes out of this!

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Petar Veličković

Petar Veličković Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @PetarV_93

5 Nov
Delighted to announce two papers we will present at #NeurIPS2021: on XLVIN (spotlight!), and on transferable algorithmic reasoning.

Both summarised in the wonderful linked thread from @andreeadeac22!

I'd like to add a few sentiments on XLVIN specifically... thread time! 🧵1/7
You might have seen XLVIN before -- we'd advertised it a few times, and it also featured at great length in my recent talks.

The catch? The original version of XLVIN has been doubly-rejected, from both ICLR (in spite of all-positive scores) and ICML. 2/7

However, this is one of the cases in which the review system worked as intended! Even AC-level rejections can be a blessing in disguise.

Each review cycle allowed us to deepen our qualitative insight into why exactly does XLVIN work as intended... 3/7
Read 7 tweets
5 Nov
I've recently been asked the following question:

"Let's say I have two spare days and want to really understand GNNs. What should I do?"

My answers led me to revisit my old 'hints for GNN resources' in light of the new material I've (co)produced. See the thread for a summary!
I'd say it is good to start with something a bit more theoretical, before diving into code. Specifically, I've been recommending my @Cambridge_CL talk on Theoretical GNN Foundations:



Why do I recommend this talk, specifically?
It is good to (a) have a rule-of-thumb to categorise the architectures you encounter, as GNNs evolve at an outrageous pace; (b) have a feel for the connections across different fields that propose GNNs, as each field (e.g. signal processing, NLP...) tends to use its own notation.
Read 8 tweets
21 Jul
We release the full technical report & code for our OGB-LSC entry, in advance of our KDD Cup presentations! 🎉

arxiv.org/abs/2107.09422

See thread 🧵 for our insights gathered while deploying large-scale GNNs!

with @PeterWBattaglia @davidmbudden @andreeadeac22 @SibonLi et al.
For large-scale transductive node classification (MAG240M), we found it beneficial to treat subsampled patches bidirectionally, and go deeper than their diameter. Further, self-supervised learning becomes important at this scale. BGRL allowed training 10x longer w/o overfitting.
For large-scale quantum chemical computations (PCQM4M), going deeper (32-50 GNN layers) yields monotonic and consistent gains in performance. To recover such gains, careful regularisation is required (we used Noisy Nodes). RDKit conformers provided a slight but significant boost.
Read 4 tweets
20 Jul
Delighted to share our work on reasoning-modulated representations! Contributed talk at @icmlconf SSL Workshop 🎉

arxiv.org/abs/2107.08881

Algo reasoning can help representation learning! See thread👇🧵

w/ Matko @thomaskipf @AlexLerchner @RaiaHadsell @rpascanu @BlundellCharles
We study a very common representation learning setting where we know *something* about our task's generative process. e.g. agents must obey some laws of physics, or a video game console manipulates certain RAM slots. However...
...explicitly making use of this information is often quite tricky, every step of the way! Depending on the circumstances, it may require hard disentanglement of generative factors, a punishing bottleneck through the algorithm, or necessitate a differentiable renderer!
Read 6 tweets
5 Jul
I firmly believe in giving back to the community I came from, as well as paying forward and making (geometric) deep learning more inclusive to underrepresented communities in general.

Accordingly, this summer you can (virtually) find me on several summer schools! A thread (1/9)
At @EEMLcommunity 2021, I will give a lecture on graph neural networks from the ground up, followed by a GNN lab session led by @ni_jovanovic. I will also host a mentorship session with several aspiring mentees!

Based on 2020, I anticipate a recording will be available! (2/9)
Alongside @mmbronstein @joanbruna @TacoCohen, I will be co-hosting a course on Geometric Deep Learning for the African Master of Machine Intelligence @AIMS_Next.

This will closely follow our recently-released proto-book and we hope to make materials more broadly available. (3/9)
Read 9 tweets
28 Apr
Proud to share our 150-page "proto-book" with @mmbronstein @joanbruna @TacoCohen on geometric DL! Through the lens of symmetries and invariances, we attempt to distill "all you need to build the architectures that are all you need".

geometricdeeplearning.com

More info below! 🧵
We have investigated the essence of popular deep learning architectures (CNNs, GNNs, Transformers, LSTMs) and realised that, assuming a proper set of symmetries we would like to stay resistant to, they can all be expressed using a common geometric blueprint.

But there's more!
Going further, we use our blueprint on less standard domains (such as homogeneous groups and manifolds), showing that the blueprint allows for nicely expressing recent advances in those areas, such as Spherical CNNs, SO(3)-Transformers, and Gauge-Equivariant Mesh CNNs.
Read 5 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(