Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
Zeta Alpha Profile picture
@ZetaVector
Dec 7, 2021 • 12 tweets • 10 min read • Read on X
Scrolly
Want to dive into #NeurIPS2021 but don't know where to start?

Here're some ideas! A thread🧵👇
1. "A 3D Generative Model for Structure-Based Drug Design" is one of the multiple papers at NeurIPS about drug discovery using neural networks.

This model generates molecules that bind to a specific protein binding site.

By Shitong Luo et al.

papers.nips.cc/paper/2021/has…
2. "The Emergence of Objectness: Learning Zero-shot Segmentation from Videos" by Runtao Liu et al.

Leveraging clever self supervision with videos to segment objects without labels.

papers.nips.cc/paper/2021/has…
3. "Multimodal Few-Shot Learning with Frozen Language Models" by @jacobmenick @serkancabi @arkitus @OriolVinyalsML @FelixHill84 .

Freeze a pre-trained LM and train a vision encoder for prompting the LM to perform vision/language tasks.

papers.nips.cc/paper/2021/has…
4. "Efficient Training of Retrieval Models using Negative Cache" by Erik Lindgren et al.

A proposal to train dense retrieval without needing huge batches and a lot of memory.

papers.nips.cc/paper/2021/has…
5. "VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text" by @AkbariH70 Liangzhe Yuan @RuiQian3 Wei-Hong Chuang, Shih-Fu Chang, @YinCui1 @BoqingGo.

The promised multimodal future is here!

papers.nips.cc/paper/2021/has…
6. "Robust Predictable Control" by @ben_eysenbach, @rsalakhu and @svlevine.

Seing RL through the lens of compression. How do agents behave when they favor policies that are compressible (i.e. easy to predict)?

papers.nips.cc/paper/2021/has…
7. "FLEX: Unifying Evaluation for Few-Shot NLP" by Jonathan Bragg et al.

Now we can apples-to-apples comparisons of few-shot performance!

papers.nips.cc/paper/2021/has…
8. "Partition and Code: learning how to compress graphs" by @gbouritsas @loukasa_tweet @AspectStalence @mmbronstein

How would you build a native compression algorithm for graphs? -> partition and code

papers.nips.cc/paper/2021/has…
9. "Learning to Draw: Emergent Communication through Sketching" by @DanielaMihai13 and @jon_hare

How two models pretty much learn to play Pictionary.

papers.nips.cc/paper/2021/has…
10. "Fixes That Fail: Self-Defeating Improvements in Machine-Learning Systems" by Ruihan Wu et al.

Watch out! Improving parts of an ML model might lead to worse overall performance...😔

papers.nips.cc/paper/2021/has…
Read our comments on this selection and more interesting papers on our blog zeta-alpha.com/post/neurips-2…

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Zeta Alpha

Zeta Alpha Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @ZetaVector

Zeta Alpha Profile picture
Apr 3
Google just published the paper about Gecko, their new text embedding model that punches way above its weight, with its 768-dim vectors being competitive with models that have 7x more parameters and 5x larger embeddings.
Here is a thread with our key takeaways: 🧵🦎Image
Gecko relies on knowledge distillation from LLMs in the form of synthetic queries, similar to previous work such as InPars and Promptagator.
They propose a two-step approach, where (1) a query is generated given a task description and a passage, and (2) an LLM reranks the top-N retrieved passages, using the highest-scoring one as the positive and the lowest as the negative.Image
Two LLM-based ranking functions are used:
- Query Likelihood: how likely is the query (ie, its perplexity for the LLM) given the passage
- Relevance Classification: what's the probability of the "true" token, given the query-passage pair (à la MonoT5)

The two lists are combined using Reciprocal Rank Fusion, and the final order is used to pick the positive and negative pair for each synthetic query.Image
Read 7 tweets
Zeta Alpha Profile picture
Nov 2, 2022
🎉Trends in AI — November 2022 is out on our blog🎉

Big investments, more diffusion models applications, FLAN-T5 from @GoogleAI , Neural Audio Compression from @MetaAI , Single Life RL, and much more by @SergiCastellaSa 👇

A thread🧵

zeta-alpha.com/post/trends-in…
🗞NEWS

💸AI content generation startups Jasper.ai raises $125M, @StabilityAI raises $101M both at >1B valuation

💀 Argo (self driving research by Ford & VW) once valued at $7Bn is shutting down

🇨🇳🇺🇸 US tightens bans on chip tech exports to China
👾 CODE

🚀 github.com/ELS-RD/kernl inference engine to accelerate Transformers

🧬github.com/gcorso/DiffDock molecular docking with diffusion

🎶 huggingface.co/spaces/fffilon… image to music generation
Read 13 tweets
Zeta Alpha Profile picture
Sep 21, 2022
To all of those who missed the Transformers at Work workshop last Friday, we have just published all the talks on our YouTube channel🎉

Here's the YouTube playlist with all of them + a thread🧵

youtube.com/playlist?list=…
1/9 Introduction by @jakubzavrel and @SergiCastellaSa from @ZetaVector

Company news, future, a little history on Transformers, and an overview of the workshop!

2/9 "From Transformers to Work: Advances in Neural Search" by Marzieh Fadaee

Research at @ZetaVector: Leveraging LMs to generate training data? What about Multilingual data? In-domain vs. out-of-domain evaluation? Distilled models vs. teacher models?

Read 10 tweets
Zeta Alpha Profile picture
Dec 21, 2021
🎙We're thrilled to share our new podcast on Neural Information Retrieval, co-hosted by @SergiCastellaSa and @andrewyates🎉

❓We're centering each episode around a recent IR paper. First up: "Shallow Pooling for Sparse Labels" by @NegarEmpr et al.

open.spotify.com/show/2X9ymNhv4…
Available on other podcasting platforms under the name "Neural Information Retrieval Talks" (+overcast, )

Apple Pods: podcasts.apple.com/es/podcast/neu…

Google Pods: podcasts.google.com/feed/aHR0cHM6L…

Overcast:
If the show still doesn't show up wherever you get your pods please reach out and we'll try to make sure it's available there (some providers might take a bit longer to publish it)
Read 4 tweets
Zeta Alpha Profile picture
Nov 9, 2021
In his @NVIDIAGTC keynote Jensen Huang demonstrates @NVIDIAAI leading position in powering the AI ecosystem in R&D, enterprise and edge computing, with a zillion new announcements. Here's a few notable ones.
Graph Neural Network acceleration with CUDA-X.
Nemo Megatron allows training GPT-3 scale Large Language Models on distributed hardware.
Read 5 tweets
Zeta Alpha Profile picture
Nov 8, 2021
At #EMNLP2021 Evelina Fedorenko makes a strong case to defuse criticism that neural language models cannot "think". Neither can the human language modules in the brain, she argues, based on human brain studies. #EMNLP2021livetweet
In contrast, due it's predictive coding nature, language is inherently very well-suited to communication. #EMNLP2021livetweet
As far as human brain studies suggest, language is *not suitable for complex thought*, Fedorenko concludes her keynote at #EMNLP2021, as she outlines her future research. #EMNLP2021livetweet
Read 5 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

Send Email!

:(