President @cognition_labs. Past: director of engineering @Scale_AI, startup founder, ML scientist @Tesla Autopilot, researcher @StanfordSVL.
8 subscribers
Aug 5 • 15 tweets • 2 min read
Predictions for the future of software engineering:
1/ Models will be extraordinarily good at coding, very soon. Research labs are investing more in coding + reasoning improvements than any other domain for the next model generation. Their efforts will bear fruit.
Jan 22, 2023 • 13 tweets • 6 min read
🔥 Thread of cool things hackers are building at Scale’s generative AI hackathon today:
The @krea_ai team is building the Game of Life, where each alive cell is a whimsical happy Stable Diffusion image and each dead cell is an eerie, dark Stable Diffusion image, all of which evolve over time. Built on a generative AI version of Canva they made.
Nov 30, 2022 • 6 tweets • 2 min read
New blog post: How Much Better is OpenAI's Newest GPT-3 Model?
We evaluate davinci-003 across a range of classification, summarization, and generation tasks using Scale Spellbook🪄, the platform for LLM apps. Some highlights: 🧵
Davinci-003 appears to be significantly better at zero-shot classification. On a sample Yelp review sentiment dataset, davinci-003 reached 92% classification accuracy zero-shot, versus less than 70% for davinci-002.
Apr 10, 2022 • 13 tweets • 3 min read
Second order effects of the rise of large language models:
1/ Soon, all products for creators will have embedded intelligence from massive language models (think Copilot in VSCode, DALL-E 2 in Photoshop, GPT-3 in GDocs). Companies making these products will need to roll their own massive language models or pay a tax to OpenAI/Google/etc.
Feb 6, 2022 • 8 tweets • 2 min read
What I’ve learned about making synthetic data work for training ML models:
1/ Context: synthetic data has matured drastically in the past 1-2 years. It’s gone from a research niche to a production dependency of many large-scale ML pipelines, especially in computer vision.
Jan 23, 2022 • 6 tweets • 1 min read
Today I saw the impact that AlphaFold is having on speeding up drug discovery firsthand:
1/ A friend runs a biotech startup designing drugs to fight cancer. In prior work, they found that tumor cells make a protein that binds to two receptors in the body. Binding to just one of them would inhibit the tumor’s growth, but binding to both makes the tumor grow faster.
Jan 14, 2022 • 8 tweets • 1 min read
Lessons learned debugging ML models:
1/ It pays to be paranoid. Bugs can take so long to find that it’s best to be really careful as you go. Add breakpoints to sanity check numpy tensors while you're coding; add visualizations just before your forward pass (it must be right before! otherwise errors will slip in).
