Thread by @summerlinARK

Data is the current limiting factor in AI. Infra is pretty good, and models simply need more training data to push the limits of what is possible. 1/4

Example: GPT-3 was trained on most of the internet’s available text data. But more data will probably result in a more performant model. So, where does one find more text data?? 2/4

One approach is to convert audio to text. @Spotify recently released audio-text data from 100,000 podcasts.

Another approach is to use synthetic generation, whereby one AI model generates data to train another AI model. 3/4

Companies that are already deploying AI models at scale have an inherent advantage in acquiring massive repositories of training data. Existing scale will likely lead to superior model performance in the near/medium-term.

Data advantages at work! 4/4