Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
elvis Profile picture
@omarsar0
Dec 5, 2018 • 9 tweets • 3 min read • Read on X
Scrolly
A simple method for fair comparison? #NeurIPS2018 Image
Considerations: Image
Reproducibility checklist: Image
There is room for variability, especially when using different distributed systems: Image
Complexity of the world is discarded... We need to tackle RL in the natural world through more complex simulations. Image
Embedding natural background? Image
Set the bar higher for the naturalism of the environment: Image
You learn a lot by considering this idea of stepping out in the real world: Image
Reproducibility test: Image

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with elvis

elvis Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @omarsar0

elvis Profile picture
Dec 17
Summary of today's OpenAI announcement (Day 9):

- o1 is launching out of preview in the API
- support for function calling, structured output, and developer messages
- reasoning_effort parameter to tell the model how much effort to spend on thinking
- vision inputs in the API is here too
Visual inputs with developer message (this is a new spin to system message for better steering the model) inside of the OpenAI Playground Image
Cool to see support for function calling and response format for o1 Image
Read 14 tweets
elvis Profile picture
Dec 6
Summary of today's OpenAI announcement:

- introduces reinforcement fine-tuning (RFT) of o1
- tune o1 to learn to reason in new ways in custom domains
- RFT is better and more efficient than regular fine-tuning; needs just a few examples

1/n
How it looks in the dev platform. Examples show how to select RFT on o1-mini Image
What does data look like to use RFT. Uses a grader to grade the answer of the model. Different graders will be provided, with the ability to use custom grading. Image
Read 8 tweets
elvis Profile picture
Jul 18
That's right! It's a huge week for small language models (SLMs)

Few new SLMs on my radar:
Mistral NeMo

Highlights:
- Introduced by Mistral + NVIDIA
- Apache 2.0 license
- outperforms Gemma 2 9B and Llama 2 8B
- multilingual capabilities
- efficient tokenizer (Tekken)

GPT-4o mini

Highlight: "15 cents per million input tokens, 60 cents per million output tokens, MMLU of 82%, and fast."

Read 7 tweets
elvis Profile picture
Feb 21
JUST IN: Google DeepMind releases Gemma, a series of open models inspired by the same research and tech used for Gemini.

Open models fit various use cases so this is a very smart move from Google.

Great to see that Google recognizes the importance of openness in AI science and technology.

There are 2B (trained on 2T tokens) and 7B (trained on 6T tokens) models including base and instruction tuned versions. Trained on a context length of 8192 tokens.

Commercial use is allowed.

These are not multimodal models but based on the reported experimental results they appear to outperform Llama 2 7B and Mistral 7B.

I am excited about those MATH, HumanEval, GSM8K, and AGIEval results. These are really incredible results for a model this size.

Excited to dive deeper into these models. The model prompting guide is dropping soon. Stay tuned!Image
Blog:

Technical report: blog.google/technology/dev…
storage.googleapis.com/deepmind-media…
When I said outperforms other models, I meant generally outperforms them on all the benchmarks. Llama has a lot of catching up to do but it is interesting to see Mistral 7B trail Gemma very closely. These numbers don't really mean much in the context of real-world applications.

If you follow me here on X, you know how excited I get about unlocking unique value and use cases with small language models (SLMs). It will also be fun to run these locally and other other small devices. As I have been saying, SLMs are underexplored. It's a mistake to just see them as research artifacts.Image
Read 4 tweets
elvis Profile picture
Dec 6, 2023
Gemini is here!

Google DeepMind just announced Gemini, their largest and most capable AI model.

A short summary of all you need to know:

1) What it is - Built with multimodal support from the ground up. Remarkable multimodal reasoning capabilities across text, images, video, audio, and code. Nano, Pro, and Ultra models are available to support different scenarios such as efficiency/scale and support complex capabilities.

2) Performance - The results on the standard benchmarks (MMLU, HumanEval, Big-Bench-Hard, etc.) show improvement compared to GPT-4 (though not by a lot). Still very impressive!

3) Outperforming human experts - They claim that Gemini is the first model to outperform human experts on MMLU (Massive Multitask Language Understanding), a popular benchmark to test the knowledge and problem-solving abilities of AI models.

4) Capabilities- Gemini surpasses SOTA performance on a bunch of multimodal tasks like infographic understanding and mathematical reasoning in visual contexts. There was a lot of focus on multimodal reasoning capabilities with the ability to analyze documents and uncover knowledge that's hard to discern. The model capabilities reported are multimodality, multilinguality, factuality, summarization, math/science, long-context, reasoning, and more. It's probably one of the most capable models by the looks of it.

5) Trying it out - Apparently, a fine-tuned Gemini Pro is available to use via Bard. Can't wait to experiment with this soon.

6) Availability - Models will be made available for devs on Google AI Studio and Google Cloud Vertex AI by Dec 13th.

blog:

technical report:
Image
Here is the model verifying a student's solution to a physics problem. Huge implications in education. Will be taking a very close look at applications here. Image
As is becoming common now, very little to no details on architecture but it's great to see distillation useful for the Nano series models. Image
Read 16 tweets
elvis Profile picture
Aug 2, 2023
You can now connect Jupyter with LLMs!

It provides an AI chat-based assistant within the Jupyter environment that allows you to generate code, summarize content, create comments, fix errors, etc.

You can even generate entire notebooks using text prompts!

You can also pass it… https://t.co/12DlystPJOtwitter.com/i/web/status/1…
Image
Official announcement: blog.jupyter.org/generative-ai-…
I am excited about the %%ai magic commands. Here is an example of how to use ChatGPT to generate working code within the notebook cells. Image
Read 5 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

Send Email!

:(