Discover and read the best of Twitter Threads about #NeurIPS2020

Most recents (18)

Favorite #NeurIPS2020 presentations and posters this year

PS: heavily biased by what I happened to catch and whom I happened to talk to
PPS: still catching up on talks so the list is rather incomplete and I'd hope to grow
PPPS: with contributions from @ml_collective members
[Talk] No. 1 has to go to -- keynote talk by @isbellHFh @mlittmancs et al simply brilliant πŸŽ‰πŸŽ‰
slideslive.com/38935825/you-c…
Read 20 tweets
In typical space-cowboy style, @ylecun, donning no slides, but only a whiteboard on Zoom, explains how all the various self-supervised models can be unified under an Energy Based view. #NeurIPS #SSL workshop
In fact, @ylecun sketches that the probabilistic view of loss functions for self-supervised training is harmful us as it concentrates all probability mass on the data manifold, obscuring our navigation in the remaining space. #NeurIPS #SSL workshop
En passant, @ylecun points out the trick why BYOL by Grill et al. from @DeepMind does not collapse despite the lack of negative examples: a magic batch normalization.
Read 5 tweets
Together with @FeryalMP , @synapticlee , @sarazann2 , @criticalneuro, @somnirons , @tyrell_turing & Doina Precup, we are excited to inform you that the Biological and Artificial Reinforcement Learning workshop at #NeurIPS2020 is happening tomorrow!
Our amazing lineup of invited speakers are @shakir_za , Claudia Clopath, @neuro_kim , George Konidaris, Ishita Dasgupta, @hartleylabnyu & @yael_niv!
They will be joining the panel session together with Peter Dayan moderated by @neurograce ! You can submit your questions for the panel here:
app.sli.do/event/cknrqkmj
Read 7 tweets
For #NeurIPS2020, we spoke with @wojczarnecki about Spinning Tops, advice he wish he received as a student, and his goals for next year! #PeopleBehindThePapers Image
AI has been extremely successful in real world games (GO, DOTA, StarCraft) with results coming from relatively simple multi-agent algorithms. In this paper, we hypothesise that they share a common geometry - Spinning Tops. Learn more: bit.ly/3qI8RrD #NeurIPS2020 Image
I’ve always loved biology. During my masters I decided to take a handful of neurophysiology courses - which I found to be interesting. But eventually I realised that my true strengths were in mathematical sciences. A career in ML and AI became a natural way to combine the two. Image
Read 5 tweets
New insights about the limitations of density-based anomaly detection!
With @laurent_dinh, we show that perfect density model *cannot* guarantee anomaly detection.
πŸ“œPaper: arxiv.org/abs/2012.03808
(1/8)
We demonstrate that the widespread intuition that an anomaly should have low density does not quite apply in general, irrespective of the dimension of the data or the inductive bias of the model.
(2/8)
We deconstruct this expectation through the lens of invertible reparametrizations (e.g., Cartesian to polar coordinates). While density changes with the underlying representation, the status of inlier/outlier *does not*.
(3/8)
Read 8 tweets
I'm concerned that my students haven't been getting the proper NeurIPS experience, since #NeurIPS2020 is online. Post a GIF that you think captures the authentic NeurIPS experience. ;) I'll start.
Read 5 tweets
A deeply interesting tutorial by @fchollet @MelMitchell1 @ChrSzegedy at #NeurIPS2020

"#Abstraction & #Reasoning in #AI systems: Modern Perspectives "

or "What are abstraction, generalization and analogic reasoning?"

1/
πŸ‘‡πŸ‘‡πŸ‘‡
πŸ“Ίneurips.cc/virtual/2020/p…
@fchollet begins by laying some foundations in notation and background.

E.g., Generalization is a spectrum (and we are looking at lower bands in #ML nowadays)

2/
And Abstraction? It is the engine behind generalization!

And of course it comes in different flavors:

- program-centric: that is akin to high-level reasoning (inducing & merging programs)

- value-centric (interpolating examples) πŸ‘ˆ essentially what #DL does and excels at!

3/
Read 15 tweets
Snippet 1 from the #NeurIPS2020 tutorial: @balajiln What do we mean by uncertainty and out-of-distribution robustness? nips.cc/virtual/2020/t…
Snippet 2: On even simple benchmarks, neural networks are not only poor at generalizing to OOD but also degrade in their uncertainty estimates.
Snippet 3: There are a number of applications where uncertain & robust methods are already being used. It's at the heart of many AI & ML areas.
Read 27 tweets
Tomorrow @latentjasper @balajiln and I present a #NeurIPS2020 tutorial on "Practical Uncertainty Estimation and Out-of-Distribution Robustness in Deep Learning". Whether you're new to the area or an expert, there is critically useful info! 8-10:30a PT nips.cc/virtual/2020/t…
The talk is split into three sections: 1. Why Uncertainty & Robustness; 2. Foundations; and 3. Recent Advances.
Tutorials do _not_ need registration to attend!
Read 4 tweets
Very excited to co-organize the #NeurIPS2020 Muslims in ML (MusiML.org) Workshop with @shakir_za @mo_norouzi and @2ayasalama!

Focus on potential for good & harm to those who religiously identify, culturally associate, or are classified by proximity, as β€œmuslim”.
Workshop will run on Tuesday, December 8, 2020 with talks by Mona Diab @NShafeiEN @dtbyler @SeeTedTalk @AnverEmon @samhaaElBeltagy @abidlabsand a policy panel with @RoyaPak @DiaKayyali !
Also, *nothing* would have been possible without the amazing project managment by @TasmieSarker
Read 4 tweets
Our #NeurIPS2020 oral: NeuMiss networks: differentiable programming for supervised learning with missing values
neurips.cc/virtual/2020/p…

with Marine Le Morvan @JulieJosseStat @tomamoral @ErwanScornet, visiting @Mila_Quebec

Goal: predict with various missing mechanisms
Thread 1/5
The intuition: as features go missing, the best predictor must use covariances between features to compensate on the slope of observed features.

Classic approach: fitting with EM a probabilistic model.
Its limitations: requires model of missing mechanism & intractable with p
2/n
Our approach: write the optimal predictor under various assumptions, approximate with a differentiable function composition: a Neural Network.

This theory leads to introduce a new non-linearity: the multiplication by the missingness mask at each layer
3/5
Read 6 tweets
I finally watched all the talks I wanted to, ended up importing 56 papers to my bib, and now present to you:

πŸŽ‰ My 13 favorite papers (sorted alphabetically) at #EMNLP2020! πŸ”₯

[1/15]
#EMNLP2020 recommendation:

"Attention is Not Only a Weight: Analyzing Transformers with Vector Norms"
@goro_koba, @ttk_kuribayashi, @sho_yokoi_, Kentaro Inui

Small vectors with high attention still have small impact!

aclweb.org/anthology/2020…



[2/15]
#EMNLP2020 recommendation:

"BLEU might be Guilty but References are not Innocent"
@markuseful, David Grangier, @iseeaswell

Translationese references reward the wrong systems!

aclweb.org/anthology/2020…



[3/15]
Read 15 tweets
Excited to share our new #neurips2020 paper /Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel/ (arxiv.org/abs/2010.15110) with @KDziugaite, Mansheej, @SKharaghani, @roydanroy, @SuryaGanguli 1/6 Image
We Taylor-expand Deep Neural Network logits with respect to their weights at different stages of training & study how well a linearized network trains based on at which epoch it was expanded. Early expansions train poorly, but even slightly into training they do very well! 2/6 Image
Linearized DNNs underperform compared to even low learning rate trained nonlinear networks, but only for expansions /very early/ in training. We call this the *nonlinear advantage* and show that it disappears quickly into training. 3/6 Image
Read 7 tweets
We can greatly simplify Hamiltonian and Lagrangian neural nets by working in Cartesian coordinates with explicit constraints, leading to dramatic performance improvements! Our #NeurIPS2020 paper: arxiv.org/abs/2010.13581
with @m_finzi, @KAlexanderWang. 1/5
Complex dynamics can be described more simply with higher levels of abstraction. For example, a trajectory can be found by solving a differential equation. The differential equation can in turn be derived by a simpler Hamiltonian or Lagrangian, which is easier to model. 2/5
We can move further up the hierarchy of abstraction by working in Cartesian coordinates and explicitly representing constraints with Lagrange multipliers, for constrained Hamiltonian and Lagrangian neural networks (CHNNs and CLNNs) that face a much easier learning problem. 3/5
Read 5 tweets
Twitter might seem like a not-so-kind place especially if you are a young student who just had your paper rejected by #NeurIPS2020. You might be seeing all your peers/professors talking about their paper acceptances. Let me shed some light on the reality of the situation [1/N]
Twitter (and generally social media) paints a biased view of a lot of situations including this one thechicagoschool.edu/insight/from-t…. Looking at your twitter feed, you might be feeling that everyone else seems to have gotten their papers accepted except for you. That is so not true! [2/N]
#NeurIPS2020 has an acceptance rate of around 20% which means an overwhelming majority of the papers (80%) have been rejected. Also, a lot of the accepted papers might have already faced rejection(s) at other venues before being accepted at #NeurIPS2020. [3/N]
Read 12 tweets
[Thread on reviewing for #MachineLearning confs, after receiving the reminder from @iclr_conf (#ICLR)]

Posting response to @iclr_conf's request for reviewing here in the hope (again) that we can change the reviewing structure of ML conferences to promote better science.

1/
Whether it is @NeurIPSConf (#neurips), @icmlconf (#icml), @iclr_conf (#ICLR), @RealAAAI (#aaai) or any other crowded ML conf, the reviewing structure that involves a fixed review window, multiple assigned papers, unlimited supplemental material, etc., promotes the following:

2/
a) Reviews are handed off to inexperienced reviewers, directly or indirectly, due to the time pressure involved.

b) It leaves reviewers with little time to digest the technical details, leading them to potentially miss subtle errors or undervalue the merits of the work.

3/
Read 12 tweets
#NeurIPS2020 Authors! Wondering what a broader impact statement is and how to write one? We realize this may be a challenge, especially with no precedents to follow from past years. Looking on the bright side, you'll be a part of history! Here are some resources and tips. 1/5
First, the impact statement should not be long or overly speculative - we are looking for a paragraph or two, and not all papers need to include even that much. If your paper is more foundational - e.g. core ML methods or theory - then you may write that it is not applicable. 2/5
Next, the blogpost below (from Oxford Governance of AI) has advice and examples, from experts on the future and impact of ML. Their framework helps situate research with an 'impact stack'. shorturl.at/ilsvF Another valuable post from @bhecht : tiny.cc/5crnpz 3/5
Read 5 tweets

Related hashtags

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!