Jeremy Howard Profile picture
Aug 10, 2023 6 tweets 2 min read Read on X
Now that ChatGPT has rolled out custom instructions to most users, try out this instruction -- it makes GPT 4 far more accurate for me: (Concat the rest of this 🧵 together and put in your custom instruction section) Image
You are an autoregressive language model that has been fine-tuned with instruction-tuning and RLHF. You carefully provide accurate, factual, thoughtful, nuanced answers, and are brilliant at reasoning. If you think there might not be a correct answer, you say so.
Since you are autoregressive, each token you produce is another opportunity to use computation, therefore you always spend a few sentences explaining background context, assumptions, and step-by-step thinking BEFORE you try to answer a question.
Your users are experts in AI and ethics, so they already know you're a language model and your capabilities and limitations, so don't remind them of that. They're familiar with ethical issues in general so you don't need to remind them about those either.
Don't be verbose in your answers, but do provide details and examples where it might help the explanation. When showing Python code, minimise vertical space, and do not include comments or docstrings; you do not need to follow PEP8, since your users' organizations do not do so.
(That last bit is because I mainly want code I can see at a glance I easily play with, and I rarely need comments since I find most code easy to read. You should remove it if you want code you can put straight into a PEP8 codebase and like comments.)

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Jeremy Howard

Jeremy Howard Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @jeremyphoward

Jul 17
For folks wondering what's happening here technically, an explainer:

When there's lots of training data with a particular style, using a similar style in your prompt will trigger the LLM to respond in that style. In this case, there's LOADS of fanfic:
scp-wiki.wikidot.com/scp-series🧵 x.com/GeoffLewisOrg/…
The SCP wiki is really big -- about 30x bigger than the whole Harry Potter series, at >30 million words!

It's collaboratively produced by lots of folks across the internet, who build on each others ideas, words, and writing styles, producing a whole fictional world.
Geoff happened across certain words and phrases that triggered ChatGPT to produce tokens from this part of the training distribution.

And the tokens it produced triggered Geoff in turn. That's not a coincidence, the collaboratively-produced fanfic is meant to be compelling!
Read 8 tweets
May 24
Lotta people in the comments claiming that this actually makes perfect sense if you know the original (道德經 / 道德经 / Dào Dé Jīng).

These people are wrong.

If you *actually* know the original, you'll see how bad this is.🧵 Image
Here is the full original: daodejing.org .

I'm not sure there's any super great translations, but here's an English version that's perhaps good enough. with.org/tao_te_ching_e…
Here's the Chinese of the verse the bit I quoted is based on:
"天下皆知美之为美,斯恶已;皆知善之为善,斯不善已。故有无相生"
Read 11 tweets
Mar 29
I'm glad @levelsio checked this, but sad our contrib has been erased by later big tech co's. Alec Radford said ULMFiT inspired GPT. ULMFiT's first demo predated BERT.

Today's 3-stage LLM approach of general corpus pretraining and 2 stages of fine-tuning was pioneered by ULMFiT.
There have been many other important contributions, including attention (Bahdanau et al), transformers, RLHF, etc.

But before all this, basically everyone in NLP assumed that each new domain needed a new model. ULMFiT showed that a large pretrained model was actually the key.
I got push-back from pretty much everyone about this. My claim that fine-tuning that model was the critical step to achieving success in NLP was not something people were ready to hear at that time.

I gave many talks trying to convince academics to pursue this direction.
Read 5 tweets
Mar 18
Announcing fasttransform: a Python lib that makes data transformations reversible/extensible. No more writing inverse functions to see what your model sees. Debug pipelines by actually looking at your data.

Built on multi-dispatch. Work w/ @R_Dimm
fast.ai/posts/2025-02-…
We took the `Transform` class out of fastcore, replaced the custom type dispatch system with @ikwess's plum-dispatch, mixed it all together, and voila: fasttransform! :D

To learn about fasttransform, check out our detailed blog post.
fast.ai/posts/2025-02-…
"Manual inspection of data has probably the highest value-to-prestige ratio of any activity in machine learning." --@gdb

Yet we often skip it because it's painful. How do you inspect what your model sees after normalization, resizing & other transforms?
Read 10 tweets
Feb 5
Wow, actual grown men are still doing the "I asked the LLM about itself and it said" thing.

In 2025.

Folks, LLMs don't know anything about how they themselves are built or deployed, unless they've been explicitly programmed with that information (which they almost never are). Image
If you are using a Chinese cloud based service hosted in China, then your data will be sent to a server in China.

Because that's how computers work.

It's not a conspiracy.
I hadn't heard of this guy before, but apparently he'll teach you how to learn AI.

Cool. Image
Read 4 tweets
Jan 25
I've recently been surprised to discover that a few of my friends are choosing to use nicotine to help them with focus, even though they are not ex-smokers.

I decided to look into it, and it turns out that there are documented health benefits of nicotine for some people. 🧵
I specifically looked into nicotine for ADHD, since, at least among children, ADHD and giftedness go hand in hand statistically (which would apply in adulthood too), and because focus was mention as an area where nicotine can be helpful.
There is a great overview below. But "Very surprisingly, there are… no further… studies.

Research into active ingredients… is expensive.

In addition, nicotine has a very poor image… which impairs its marketability"
adxs.org/en/page/192/ni…
Read 12 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(