Does anyone have a good ballpark estimate for how fast computers "read"?
Like, during pre-training, GPTs read approximately the whole internet. That's ~295 exabytes.
Apparently, it would take a human about 24 million years to read that.
.quora.com/How-long-would…
Like, during pre-training, GPTs read approximately the whole internet. That's ~295 exabytes.
Apparently, it would take a human about 24 million years to read that.
.quora.com/How-long-would…
Let's say that it takes a GPT instance one year to pre-train (I think it's currently shorter than that, but 1 year is a fine OoM).
So that means that a GPT instance, in training, is reading about 24 million times faster than a human?
Around 7.5 orders of magnitude.
So that means that a GPT instance, in training, is reading about 24 million times faster than a human?
Around 7.5 orders of magnitude.
Does that sound right? Anyone notice egregious errors?
I figure it's (currently) slower for a trained model doing "research" to answer a question, but this at least gives me a rough anchor.
I figure it's (currently) slower for a trained model doing "research" to answer a question, but this at least gives me a rough anchor.
@threadreaderapp. @Twtextapp @unrollthread unroll @threader_appcompile
• • •
Missing some Tweet in this thread? You can try to
force a refresh
