Share this page!

Enter URL or ID to Unroll

You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649
or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

  1. On the Twitter thread, click on or icon on the bottom
  2. Click again on or Share Via icon
  3. Click on Copy Link to Tweet
  4. Paste it above and click "Unroll Thread"!
  5. More info at Twitter Help
Tristan Thrush Profile picture
Tristan Thrush
@TristanThrush
PhD-ing @StanfordAILab NLP Group and extremely infrequent waterer of plants @ContextualAI. Past: @huggingface, @Meta FAIR, @mitbrainandcog, @MIT_CSAIL, @NASAJPL
Jan 11, 2024 • 8 tweets • 3 min read
📢 New paper!!

Do LLMs understand self-referential statements? Introducing “I am a Strange Dataset”. All tested models perform around chance at our metalinguistic self-reference task.

GPT-4 is the only model significantly above chance on all tests, but it is slight.🧵 Image Each example in the dataset consists of two self-referential statements that begin in the same way but have different endings. One is true and one is false. Crucially, the ending flips the truth value of the statement. Image
Oct 11, 2022 • 15 tweets • 6 min read
For our Online Language Modelling (OLM) project, we’ve open-sourced end-to-end code to turn the latest Common Crawl and Wikipedia web snapshots into clean datasets for pretraining models like BERT and GPT-2: github.com/huggingface/ol…. What are the details? A 🧵: Why is it important? BERT and GPT-2 both live in the past. They think that Obama is still president and have never heard of COVID! To fix this, we need a pretraining dataset that continuously updates.