@SnorkelAI @uwcse / prev @StanfordAILab – Interested in data management systems for machine learning, weak supervision, and impactful applications.
Apr 2, 2023 • 10 tweets • 4 min read
1/ Prediction: Everyone will soon be using foundation models (FMs) like GPT-4.
However, they'll be using FMs trained on their own data & workloads:
"GPT-You", not GPT-X
Tl/dr:
- Closed APIs aren't defensible
- The durable moat is data
- The last mile generates the real value
2/ *Closed APIs aren't defensible*
- Recent examples like @StanfordCRFM Alpaca tinyurl.com/yc78bnct shows that cloning closed API-based FMs like ChatGPT can be done for a few $100 on top of small OSS base FMs (e.g. here, fine-tuning LLaMa 7B via exs from the ChatGPT API).
Dec 18, 2022 • 8 tweets • 2 min read
1/ 2023 AI prediction: the gap between generative and predictive AI will widen.
Despite product & business model innovation in generative AI, real-world ROI will remain concentrated around predictive AI- leading to frustrated expectations.
This gap will all come down to data... 2/ First, basic definitions:
- Generative (ie. LLMs / foundation models): Goal is to output a data point (e.g. an image)
- Predictive (or "discriminative"): Goal is to label a data point (e.g. predict whether an image contains offensive content).
