Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
Clémentine Fourrier 🍊 Profile picture
Clémentine Fourrier 🍊
@clefourrier
Evals @HuggingFace 🐍✨ "The future is already here, it’s just not very evenly distributed" (Gibson)
Mar 13 • 5 tweets • 2 min read
Dear community,

For the last 2 years, we've evaluated over 13K models with the Open LLM Leaderboard, using our research cluster to provide open, fair and reproducible evaluations to all.

However, all good things come to an end: the leaderboard is officially retiring! Image Why? As model capabilities change (hello reasoning and LM assistants), benchmarks need to follow!

The leaderboard is slowly becoming obsolete; we feel it could encourage people to hill climb in irrelevant directions.

So this is the end! (hold your breath & count to 10)
Jun 26, 2024 • 15 tweets • 4 min read
LLM performances have been plateauing... so we decided to make the Open LLM Leaderboard steep again 🏔️ 😈

Introducing the Leaderboard 2️⃣

Expect...
- new benchmarks
- fairer reporting
- cool features (did I hear voting and chat template?)

🧵

huggingface.co/spaces/open-ll… Over the last year, our benchmarks slowly became overused and saturated:
- models got way better at them and we reached saturation
- people starting to over optimize for the leaderboard
- and we also observed some contamination...

So it was time for a change! Image
Jan 3, 2023 • 4 tweets • 3 min read
For the last months @huggingface, I worked on transformers and... graphs!

So here is a small blog, if you wonder what one could use graphs for, or how to machine learn on them 🔎
(Spoiler: they are everywhere 🧬🚗✍️)

huggingface.co/blog/intro-gra… In this post, you'll discover:
- what graphs are, why they are used, how to represent them
- how people learn on graphs, from pre-neural methods to Graph Neural Networks
- the very recent world of Transformers for graphs
Mar 21, 2022 • 7 tweets • 5 min read
#acl2022nlp
What happens inside a multilingual neural cognate prediction model?
We show that predicting cognates between current Romance languages latently teaches the model about their proto-forms, allowing reconstruction without fine-tuning encoders on the task!🧵 In layman's terms, learning to predict special words (cognates) between related languages (French, Italian, Spanish, Portuguese, Galician, Catalan, Occitan, Romanian, and Aromanian) gives the model 'intuitive' knowledge about their parent, Latin!