,
3 tweets,
1 min read
Read on Twitter
"Ever bigger steam engines burning ever more coal is the superior approach to build faster trains", they said.
More data + scaling up simple algos will get you better models, obviously. But there are diminishing returns -- meanwhile, new approaches can be transformational
More data + scaling up simple algos will get you better models, obviously. But there are diminishing returns -- meanwhile, new approaches can be transformational
I guarantee you that the dominant AI algorithm of the 2050s will not be a stack of differentiable layers trained with SGD.
At this point we are pretty close to training our language models on the entirety of text corpora that humanity has produced so far. Their language understanding capabilities are still close to zero.
