Attending @NeurIPSConf #NeurIPS2021 today?
Interested in algorithmic reasoning, implicit planning, knowledge transfer or bioinformatics?
We have 3 posters (1 spotlight!) in the poster session (4:30--6pm UK time) you might find interesting; consider stopping by! Details below: 🧵
(1) "Neural Algorithmic Reasoners are Implicit Planners" (Spotlight); with @andreeadeac22, Ognjen Milinković, @pierrelux, @tangjianpku & Mladen Nikolić.

Value Iteration-based implicit planner (XLVIN), which successfully breaks the algorithmic bottleneck & yields low-data gains.
(2) "How to transfer algorithmic reasoning knowledge to learn new algorithms?"; with Louis-Pascal Xhonneux, @andreeadeac22 & @tangjianpku.

Studying how to successfully transfer algorithmic reasoning knowledge when intermediate supervision traces are missing.
(3) "Neural Distance Embeddings for Biological Sequences"; with @GabriCorso, @RexYing0923, @Mpmisko, @jure & @pl219_Cambridge.

Framework to embed biological sequences in a hyperbolic space, and derive useful relations between them (e.g. alignment, similarity, clustering).

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Petar Veličković

Petar Veličković Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @PetarV_93

1 Dec
Our work has been published in @Nature!!

(G)NNs can successfully guide the intuition of mathematicians & yield top-tier results -- in both representation theory & knot theory.

dpmd.ai/nature-maths
arxiv.org/abs/2111.15161
arxiv.org/abs/2111.15323

See my 🧵 for more insight...
It’s hard to overstate how happy I am to finally see this come together, after years of careful progress towards our aim -- demonstrating that AI can be the mathematician’s 'pocket calculator of the 21st century'.

I hope you’ll enjoy it as much as I had fun working on it!
I was leading the GNN modelling on representation theory: working towards settling the combinatorial invariance conjecture, a long-standing open problem in the area.

My work earned me the co-credit of 'discovering math results', an honour I never expected to receive.
Read 8 tweets
5 Nov
Delighted to announce two papers we will present at #NeurIPS2021: on XLVIN (spotlight!), and on transferable algorithmic reasoning.

Both summarised in the wonderful linked thread from @andreeadeac22!

I'd like to add a few sentiments on XLVIN specifically... thread time! 🧵1/7
You might have seen XLVIN before -- we'd advertised it a few times, and it also featured at great length in my recent talks.

The catch? The original version of XLVIN has been doubly-rejected, from both ICLR (in spite of all-positive scores) and ICML. 2/7

However, this is one of the cases in which the review system worked as intended! Even AC-level rejections can be a blessing in disguise.

Each review cycle allowed us to deepen our qualitative insight into why exactly does XLVIN work as intended... 3/7
Read 7 tweets
5 Nov
I've recently been asked the following question:

"Let's say I have two spare days and want to really understand GNNs. What should I do?"

My answers led me to revisit my old 'hints for GNN resources' in light of the new material I've (co)produced. See the thread for a summary!
I'd say it is good to start with something a bit more theoretical, before diving into code. Specifically, I've been recommending my @Cambridge_CL talk on Theoretical GNN Foundations:



Why do I recommend this talk, specifically?
It is good to (a) have a rule-of-thumb to categorise the architectures you encounter, as GNNs evolve at an outrageous pace; (b) have a feel for the connections across different fields that propose GNNs, as each field (e.g. signal processing, NLP...) tends to use its own notation.
Read 8 tweets
21 Jul
We release the full technical report & code for our OGB-LSC entry, in advance of our KDD Cup presentations! 🎉

arxiv.org/abs/2107.09422

See thread 🧵 for our insights gathered while deploying large-scale GNNs!

with @PeterWBattaglia @davidmbudden @andreeadeac22 @SibonLi et al.
For large-scale transductive node classification (MAG240M), we found it beneficial to treat subsampled patches bidirectionally, and go deeper than their diameter. Further, self-supervised learning becomes important at this scale. BGRL allowed training 10x longer w/o overfitting.
For large-scale quantum chemical computations (PCQM4M), going deeper (32-50 GNN layers) yields monotonic and consistent gains in performance. To recover such gains, careful regularisation is required (we used Noisy Nodes). RDKit conformers provided a slight but significant boost.
Read 4 tweets
20 Jul
Delighted to share our work on reasoning-modulated representations! Contributed talk at @icmlconf SSL Workshop 🎉

arxiv.org/abs/2107.08881

Algo reasoning can help representation learning! See thread👇🧵

w/ Matko @thomaskipf @AlexLerchner @RaiaHadsell @rpascanu @BlundellCharles
We study a very common representation learning setting where we know *something* about our task's generative process. e.g. agents must obey some laws of physics, or a video game console manipulates certain RAM slots. However...
...explicitly making use of this information is often quite tricky, every step of the way! Depending on the circumstances, it may require hard disentanglement of generative factors, a punishing bottleneck through the algorithm, or necessitate a differentiable renderer!
Read 6 tweets
5 Jul
I firmly believe in giving back to the community I came from, as well as paying forward and making (geometric) deep learning more inclusive to underrepresented communities in general.

Accordingly, this summer you can (virtually) find me on several summer schools! A thread (1/9)
At @EEMLcommunity 2021, I will give a lecture on graph neural networks from the ground up, followed by a GNN lab session led by @ni_jovanovic. I will also host a mentorship session with several aspiring mentees!

Based on 2020, I anticipate a recording will be available! (2/9)
Alongside @mmbronstein @joanbruna @TacoCohen, I will be co-hosting a course on Geometric Deep Learning for the African Master of Machine Intelligence @AIMS_Next.

This will closely follow our recently-released proto-book and we hope to make materials more broadly available. (3/9)
Read 9 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(