Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
lmarena.ai Profile picture
@lmarena_ai
Aug 1, 2024 • 5 tweets • 3 min read • Read on X
Scrolly
Exciting News from Chatbot Arena!

@GoogleDeepMind's new Gemini 1.5 Pro (Experimental 0801) has been tested in Arena for the past week, gathering over 12K community votes.

For the first time, Google Gemini has claimed the #1 spot, surpassing GPT-4o/Claude-3.5 with an impressive score of 1300 (!), and also achieving #1 on our Vision Leaderboard.

Gemini 1.5 Pro (0801) excels in multi-lingual tasks and delivers robust performance in technical areas like Math, Hard Prompts, and Coding.

Huge congrats to @GoogleDeepMind on this remarkable milestone!

Gemini (0801) Category Rankings:
- Overall: #1
- Math: #1-3
- Instruction-Following: #1-2
- Coding: #3-5
- Hard Prompts (English): #2-5

Come try the model and let us know your feedback!
More analysis below👇Image
Gemini 1.5 Pro (Experimental 0801) #1 on Vision Leaderboard.
Image
Image
Gemini shows strong multilingual capability: #1 performance in Chinese, Japanese, German, Russian.
Image
Image
But in technical domains like Coding/Hard Prompt Arena, Claude 3.5 Sonnet, GPT-4o, Llama 405B are still leading the way.
Image
Image
Overall win-rate heatmap: Gemini 1.5 Pro (0801) wins 54% vs GPT-4o, 59% vs Claude-3.5-Sonnet.

Check out full data at and come chat with the model! leaderboard.lmsys.org
Image

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with lmarena.ai

lmarena.ai Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @lmarena_ai

lmarena.ai Profile picture
Jun 5
🚨Breaking: New Gemini-2.5-Pro (06-05) takes the #1 spot across all Arenas again!

🥇 #1 in Text, Vision, WebDev
🥇 #1 in Hard, Coding, Math, Creative, Multi-turn, Instruction Following, and Long Queries categories

Huge congrats @GoogleDeepMind! Image
New Gemini-2.5-Pro (06-05) ranks #1 in WebDev Arena (+35 pts from previous 2.5 Pro) Image
New Gemini-2.5-Pro (06-05) ranks #1 across all categories in Text Arena Image
Read 5 tweets
lmarena.ai Profile picture
May 21
📢We’re excited to share that we’ve raised $100M in seed funding to support LMArena and continue our research on reliable AI. Led by @a16z and UC Investments (@UofCalifornia), we're proud to have the support of those that believe in both the science and the mission.

We’re focused on building a neutral, open, community-driven platform that helps the world understand and improve the performance of AI models on real queries from real users.

Also, big news is coming next week!👀

We're relaunching LMArena with a whole new look built directly with community feedback from the ground up 🧱 Link in thread.
Check out the new UI at beta.lmarena.ai !

This beta addresses a lot of the feedback that we’ve received for over two years from our community: it has fewer errors, a better user experience, better performance, and chat history.

More improvements and new features to the new LMArena UI are coming! But this funding isn’t just about scaling infrastructure, it’s about supporting the community that makes LMArena possible. We’re doubling down on what matters: improving the diversity of our voters; more methodological research like style control and prompt-to-leaderboard; more modalities; more open data. Our mission remains the same. To bring the best AI models to everyone, and to improve them through real-world community evaluations.
This next chapter is about growth and delivering what the community is asking for. We’re hiring! If you’re excited about building reliable, transparent community evaluations in AI, come join us. We have a lot of community feedback to keep up with! See open roles and help shape the future of AI evaluation: jobs.ashbyhq.com/lmarena
Read 4 tweets
lmarena.ai Profile picture
Mar 27
News: the latest ChatGPT-4o (2025-03-26) jumps to #2 on Arena, surpassing GPT-4.5!

Highlights
- Significant improvement over the January version (+30 pts, #5->#2)
- Tied #1 in Coding, Hard Prompts. Top-2 across ALL categories
- Matching or surpassing GPT-4.5 with 10x cheaper price.

Congrats @OpenAI for the new milestone! View below for more insights ⬇️Image
We saw clear leaps in improvements over the previous ChatGPT-4o release:

- Math #14 -> #2
- Hard Prompts #7 -> #1
- Coding #5 -> #1 Image
Check out the ChatGPT-4o-latest and all frontier models at: lmarena.ai
Read 4 tweets
lmarena.ai Profile picture
Mar 25
BREAKING: Gemini 2.5 Pro is now #1 on the Arena leaderboard - the largest score jump ever (+40 pts vs Grok-3/GPT-4.5)! 🏆

Tested under codename "nebula"🌌, Gemini 2.5 Pro ranked #1🥇 across ALL categories and UNIQUELY #1 in Math, Creative Writing, Instruction Following, Longer Query, and Multi-Turn!

Massive congrats to @GoogleDeepMind for this incredible Arena milestone! 🙌

More highlights in thread👇Image
Gemini 2.5 Pro #1 across ALL categories, tied #1 with Grok-3/GPT-4.5 for Hard Prompts and Coding, and edged out across all others to take the lead 🏇🏆 Image
Gemini 2.5 Pro ranked #1 on the Vision Arena 🖼️ leaderboard! Image
Read 5 tweets
lmarena.ai Profile picture
Feb 26
Introducing Prompt-to-leaderboard (P2L): a real-time LLM leaderboard tailored exactly to your use case!

P2L trains an LLM to generate "prompt-specific" leaderboards, so you can input a prompt and get a leaderboard specifically for that prompt.

The model is trained on the 2M human preference votes from Chatbot Arena.

P2L Highlights:
🔹Instant leaderboard for any prompt 🗿
🔹Optimal model routing (hit #1 on Chatbot Arena in Jan 2025 with 1395 score 🧏)
🔹Fine-grained model strength & weakness analysis 🤓

Check out our demo and thread below for more details!
Use case 1: Optimal Routing

If we know which models are best per-prompt, that makes optimal routing easy!

- Performance: P2L-router (experimental-router-0112) is #1 on Chatbot Arena in Jan 2025 with a score of 1395. (+20 than the best model candidate)
- We also develop cost-constrained P2L achieving Pareto frontierImage
Use case 2: Domain-Specific Leaderboards

P2L can aggregate rankings of prompts within a category to produce an adaptive category ranking →

e.g., Find the best models for SQL queries instantly! Image
Read 6 tweets
lmarena.ai Profile picture
Feb 18
BREAKING: @xAI early version of Grok-3 (codename "chocolate") is now #1 in Arena! 🏆

Grok-3 is:
- First-ever model to break 1400 score!
- #1 across all categories, a milestone that keeps getting harder to achieve

Huge congratulations to @xAI on this milestone! View thread 🧵 for more insights into Grok-3's performance after ~8K votes in the Arena.Image
Here you can see @xai Grok-3’s performance across all the top categories:
🔹 Overall w/ Style Control
🔹 Hard Prompts & Hard Prompt w/ Style Control
🔹 Coding
🔹 Math
🔹 Creative Writing
🔹 Instruction Following
🔹 Longer Query
🔹 Multi-Turn Image
In the Coding category, Grok-3 surpassed top reasoning models like o1 and Gemini-thinking. Image
Read 5 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

Send Email!

:(