Read on Twitter
π On with S02E01 of my saga for the AI-enabled developer
In this issue, I want to talk a bit about the elephant in the room π
A topic that's not really mentioned for #gpt4's web users cause you need to preserve the magic πͺ
Yep, I'm gonna talk about max context size π
1/13
In this issue, I want to talk a bit about the elephant in the room π
A topic that's not really mentioned for #gpt4's web users cause you need to preserve the magic πͺ
Yep, I'm gonna talk about max context size π
1/13
βοΈ #gpt4 doesn't remember everything
Have you heard of 8K and 32K contexts?
Apart from pricing πΈ, do you know what it really entails?
It means that your chat actually has a short-term memory πΆ
2/13
Have you heard of 8K and 32K contexts?
Apart from pricing πΈ, do you know what it really entails?
It means that your chat actually has a short-term memory πΆ
2/13
If you are a #gpt pro user, have you noticed that it keeps forgetting rules from time to time?
And have you noticed that the longer the messages & response, the faster it seems to forget?
Well, that's because of context limitations
3/13
And have you noticed that the longer the messages & response, the faster it seems to forget?
Well, that's because of context limitations
3/13
βοΈWith #gpt4, the context is your chat
No, the context is not your last message. As a matter of fact, it's the entire conversation.
A hard fact I learned when maxing out my context while playing in the OpenAI playground
4/13
No, the context is not your last message. As a matter of fact, it's the entire conversation.
A hard fact I learned when maxing out my context while playing in the OpenAI playground
4/13
The #gpt4 context is comprised of
πΈ System: a general prompt that should apply to the entire conversation (even as it starts "forgetting")
πΈ An array of User x Assistant messages
In order for #chatgpt to give you an answer, it needs the previous questions and answers
5/13
πΈ System: a general prompt that should apply to the entire conversation (even as it starts "forgetting")
πΈ An array of User x Assistant messages
In order for #chatgpt to give you an answer, it needs the previous questions and answers
5/13
βοΈThe context is #gpt4's top limitation
Sure, it "knows" stuff
But it doesn't know everything & there is a hard cutoff date in sept. 2021 on this knowledge
So you need to give it extra updated knowledge from time to time, which means...
You guessed it, context
6/13
Sure, it "knows" stuff
But it doesn't know everything & there is a hard cutoff date in sept. 2021 on this knowledge
So you need to give it extra updated knowledge from time to time, which means...
You guessed it, context
6/13
So let's say you've got tons of π΅ & don't mind paying. What can you fit in a 32K #gpt4 context?
Remember, my system prompt was around 1500 tokens
A 150 lines file of JS is 1500 tokens
@reactjs latest doc is ~640k tokens per a 1 token = 4 chars rule
7/13
Remember, my system prompt was around 1500 tokens
A 150 lines file of JS is 1500 tokens
@reactjs latest doc is ~640k tokens per a 1 token = 4 chars rule
7/13
https://twitter.com/yleflour/status/1638873342987083777
βοΈ But there are workarounds for #gpt4's context limitations
And I mentioned the main one in an earlier thread π
8/13
And I mentioned the main one in an earlier thread π
8/13
https://twitter.com/yleflour/status/1638148578471452674
So guess what's happening ?
We're creating a network of AI's services and language models to solve this issue
That's the smartest from @OpenAI considering everything still goes through their chat. They're turning into the gatekeeper just like Google in the old days
9/13
We're creating a network of AI's services and language models to solve this issue
That's the smartest from @OpenAI considering everything still goes through their chat. They're turning into the gatekeeper just like Google in the old days
9/13
https://twitter.com/OpenAI/status/1638952876281335813
βοΈThere's another solution to #gpt4's context limitation
Having the ability to fine-tune it
It's already achievable for @OpenAI's other models, so it might definitely be an option in the long run
This would allow you to dump new context once & for all
10/13
Having the ability to fine-tune it
It's already achievable for @OpenAI's other models, so it might definitely be an option in the long run
This would allow you to dump new context once & for all
10/13
"But it costs $$$" π© you say
Well, seeing how it's going, that might not be the case for long. And you might even be able to run it directly on your machine
11/13
Well, seeing how it's going, that might not be the case for long. And you might even be able to run it directly on your machine
11/13
https://twitter.com/BrianRoemmele/status/1637873462239952896
So back to #gpt4 web chat
3 things I noted:
πΈ I've saved a lot of money with #gpt pro, even with the rate limit
πΈ I'm not sure what context size it's been given
πΈ Externalizing knowledge is a smart way to overcome context
12/13
3 things I noted:
πΈ I've saved a lot of money with #gpt pro, even with the rate limit
πΈ I'm not sure what context size it's been given
πΈ Externalizing knowledge is a smart way to overcome context
12/13
So can you guess what I'll be exploring from now on?
Yep, setting up your own knowledge base and hooking it up to #gpt
And if you're in for the ride, follow me and help me by β€οΈ that first tweet π
13/13
Yep, setting up your own knowledge base and hooking it up to #gpt
And if you're in for the ride, follow me and help me by β€οΈ that first tweet π
13/13
https://twitter.com/yleflour/status/1640307712625102851
β’ β’ β’
Missing some Tweet in this thread? You can try to
force a refresh
γ
