1/ one of the biggest questions in AI today is:

since GPT-4 was trained in fall 2022, we've collectively spent ~$100B on NVIDIA GPUs

will the next generation of AI models' capabilities live up to that aggregate investment level?

NVIDIA qtrly datacenter rev, by @Thomas_Woodside 2/ there are 2 schools of thought:

1) compute is the only real bottleneck to AI progress. the more we spend, the closer we get to AGI

2) we are hitting a data wall which will slow progress regardless of how much compute we have

https://x.com/EpochAIResearch/status/1798742418763981241

1/ We are launching SEAL Leaderboards—private, expert evaluations of leading frontier models.

Our design principles:
🔒Private + Unexploitable. No overfitting on evals!
🎓Domain Expert Evals
🏆Continuously Updated w/new Data and Models

Read more in 🧵

scale.com/leaderboard




2/ Evaluations are a critical component of the AI ecosystem.

Evals are incentives for researchers, and our evaluations set the goals for how we aim to improve our models.

Trusted 3rd party evals are a missing part of the whole ecosystem, which is why @scale_AI built these.

1/ Today is the 4th anniversary of the original GPT-3 paper—"Language Models are Few-Shot Learners"

Some reflections on how the last 4 years have played out, and thoughts about the next 4 years 2/ GPT-3 was when it first became clear what the potential of scaling language models was.

The efficacy of GPT-3 took the AI community by surprise for the most part—the capabilities were staggering compared to everything that came before in NLP.

1/ Some thoughts on the recent OpenAI and Google announcements, and what it indicates about what's next in AI.

Hint: post-training is REALLY important...

THREAD 2/ In many ways, Gemini 1.5 Flash was the gem of Google's announcements. A 1M-context small model with Flash performance is incredible.

OpenAI now has the best large model with GPT-4o, and Google has the best small model with Gemini 1.5 Flash.

The competition is on.

I'm posting some of my learnings from 2023, AI's biggest year yet.

🧵 for some highlights and link to post LEARNING 1: The conceit of an expert is a trap. Strive for a beginner’s mind and the energy of a novice.

Experience can often be a curse—the past is only mildly predictive of the future, and every scenario requires new techniques and insight. In novel situations, the novice tends to be at an advantage—their vitality and beginner’s mind lend themselves to faster adaptation.

With @MetaAI's the launch of Llama 2—@scale_ai will also be:

🌎 open-sourcing scale-llm-engine, our library for hosting and fine-tuning open-source LLMs
⚡️ releasing the fastest way to fine-tune Llama 2
💼 launching Scale Custom LLMs for enterprises

Read more in 🧵



We are open-sourcing scale-llm-engine, our library for hosting and fine-tuning open-source LLMs.

This can run on your own infra, as well as on Scale's cloud infrastructure.

Docs here:


Github link:
https://t.co/mrguGYVEAoscaleapi.github.io/llm-engine/
github.com/scaleapi/llm-e…

Last week, @scale_AI hosted a Generative AI hackathon.

It was the purest expression of builder energy, and an omen that the dark days of tech stagnation are over.

Roughly ~300 hackers converged @ our office in SF, and a day full of frenzy ensued

🧵 of cool things that happened 🥇Winning 1st place was a project—"GPT is all you need for backend"

Probably the most provocative POV on the future of software—that LLMs will replace backend code too!

https://twitter.com/karpathy/status/1618311660539904002

🧵 thread of some of my favorite AI-generated product images from @scale_AI Forge

AI-generated advertising only gets better as we keep improving our underlying models It works really well in conveying the feeling of cosmetic products.

Heard someone say “I don’t want to waste brain space on learning Chinese”

PSA—that’s not how it works at all.

Consistently *retrieving* information both deepens connections with the rest of your knowledge and frees up resources & working memory for more abstract thought.

🧵 Memorizing actually allows for new conceptual understanding, it’s not just rote BS.

And while there is some “wetware” limit based on the number of synapses, that limit is roughly the memory size of the movie of your entire life. It’s why some people can have photographic memory

I'm publishing a call to action: The AI War and How to Win It.

AI for national security will define the future of our world. Either the USA wins, or our authoritarian adversaries do.

I walk through The AI War, The China Threat, and How to Win It.

🧵

alexw.substack.com/p/war?sd=pf The Ukraine war demonstrates that the tech stack for war has changed.

The future is clear—AI-powered targeting and autonomous drones will define warfare.

Our legacy military platforms will be disrupted by cheaper autonomous drone fleets.

vox.com/2022/9/21/2335…

1/ We are launching a product we previewed 2 weeks ago—Scale Forge ⚒

We're enabling marketers to AI-generate UNLIMITED and INFINITELY CREATIVE product imagery for:
- brand campaigns
- ad creatives
- social media
- product images

See the product in the video!

Thread 🧵 2/ Scale Forge ⚒ is an AI-powered design studio that enables customers to create new product images that allow for high-fidelity brand preservation.

You can use one of our default products, or upload your own!

I wanted to preview of one of the coolest products from the @scale_AI labs.

We're enabling marketers to AI-generate UNLIMITED and INFINITELY CREATIVE images of their products for:
- ad creatives
- brand campaigns
- social media

Every image in this thread is AI generated🧵 The most creative ads are the ones we remember the best—they're striking, memorable, and cool.

With the new breakthroughs in AI, we can enable brands to unlock their imagination, and grow their customer base.

What are the most inspirational settings for your product?

.@scale_AI had our TransformX conference last week.

As part of that we announced a number of ⚡️new products⚡️ to unlock and operationalize AI for everyone—from startups to researchers and Fortune 500 companies to the US government.

Thread🧵 We announced the ✨Scale Applied AI✨ Suite.

Scale is at the forefront in advancing foundation models, especially applying them to specific tasks + industries.

These are real examples of how Scale is partnering with customers across industries.

Posting a memo I sent to the @scale_AI team back in 2019.

The core idea is that most organizations fall prey to a slow death of optimism, causing a slow, excruciating halt.

Thread below 👇

alexw.substack.com/p/optimism-sha… The scope, or how long we say something will take, influences how long something takes.

When we say things will take a long time, they will take a long time.

When we say things will take a short amount of time, they will take less time.

The nerds vs the cool kids, a short thread 🧵

The nerds are jealous of the cool kids for being, well, cool.

The cool kids are jealous of the nerds for the ability to build.

1/n It’s not totally true the cool kids don’t build. The nerds build things and the cool kids build relationships.

Nerds are jealous of the connectivity, cool kids are jealous of the substance

2/n

Breaking down my recent post:

1/ @scale_AI, I interviewed everyone we gave an offer to for a long time. I wrote a memo to the company about what I look for. I wanted to share it with the community because I don’t think people do this enough.

Thread👇
alexw.substack.com/p/hire 2/ I mainly screen for one key thing: giving a shit. To be more specific, there’s actually two things to screen for:

1. they give a shit about Scale, and
2. they give a shit about their work in general.

Incredible quotes from @BillGates and @reidhoffman's @mastersofscale episode:

Mitch Lasky on Bill:

“Bill is wrong. But he works so hard, he probably will succeed even though he’s wrong in this case.”

Hard work gives you the ability to bend the world in your direction. 1/ Bill on IBM as a partner:

"So if you can pick these toughest customers and meet their needs, then you can sit back and wait. You’re going to be fine. And you have to pick the ones that are highly visible and tough and you can mess up either of those."

Tough customers matter! 2/