Tom Goldstein Profile picture
Professor at UMD. AI security & privacy, algorithmic bias, foundations of ML. Follow me for commentary on state-of-the-art AI.
6 subscribers
Jun 20 9 tweets 4 min read
LLMs have low randomness: if you ask the same thing twice you get similar responses. Generator prompts are a way to boost the randomness of LLMs.

Using a few generator prompts, I had Gemini write an entire instruction tuning dataset from scratch. It outperform popular datasets. Image Let’s start with a toy example of why we need generator prompts. Suppose I want a list of different colors. So I feed this prompt to Gemini 1000 times. This does poorly - I only get 33 unique outputs from 1000 runs. I need more randomness. Image
Oct 12, 2023 8 tweets 2 min read
🚨 This one simple trick will level up your LLM🚀🚀

Wait...don't go. This isn't a blue check grifter tweet!

Instruction tuning with this easy trick will *actually* boost AlpacaEval scores, even for large (70B) and llama2-chat base models…by a lot 🧵 Image Ok, here's the trick: during instruction finetuning, we add uniform random noise to the word embeddings.

That's it. Nothing else.

We tried this on a bunch of base models and finetuning datasets. They all showed big gains. Image
Jul 19, 2023 11 tweets 4 min read
The Llama2 model is pretty impressive. Human evaluators rank it slightly *better* than ChatGPT on a range of things (excluding code and reasoning).

Here's a short TL;DR on what Meta did to improve the state of the art 🧵 Image Llama1: Small models (7B & 13B) were trained on 1 trillion tokens. Large models saw 1.4T tokens.

Llama2: All models trained on 2T tokens. This means the small models are "over trained" beyond what the scaling laws recommend, resulting in great performance for small models! Image
Jul 5, 2023 11 tweets 5 min read
Nvidia’s AI products follow a weird reverse Moore’s law: every two years, you get half as many FLOPS for your money. This is the opposite of the rest of the chip market 📈

With the H100 release, Nvidia had to reverse course.

A 🧵 on Nvidia losing its grip on the GPU market. Let’s focus in on the machine learning GPUs. You can see the value drop over time, until the H100 created an uptick. Note: I’m using today’s price for each card, but a similar downward trend also holds for the release prices.
Jun 19, 2023 12 tweets 4 min read
Training an LLM takes about 1 trillion words. That’s about 30,000 years of typing.
But where does this data come from?
And what does this have to do with the Reddit protests?
Here’s how OpenAI trains models on “the entire internet.” 🧵📜 Much of what we know about OpenAI is from urban legends. But the GPT3 paper does have a table showing their data sources. The cliché that LLMs are trained on “the whole internet” comes from the use of CommonCrawl. Image
Jun 13, 2023 11 tweets 5 min read
A common criticism of LLM watermarks is they can be removed by AI paraphrasing or human editing. Let's put this theory to the test! Can a watermark be automatically removed by GPT? Can a grad student do any better? The results surprised me 🧵
arxiv.org/pdf/2306.04634… Image First, if you don’t remember how watermarks work, you might revisit my original post on this issue.
TL;DR The watermark is a subtle pattern embedded in LLM outputs that labels it as machine generated. High accuracy detection usually requires 50-ish words.
May 30, 2023 12 tweets 3 min read
LLMs do many things more efficiently than humans. But there’s one thing humans still do WAY better than machines: learn. In this thread I compare the learning efficiency of machines to that of humans, and I use scaling laws to convert humans into equivalent LLMs. 🧵 Image A typical human hears 20K words per day. By age five, a typical child should have heard 37 million words. A 50 year old should have heard 370M words.
greatschools.org/gk/articles/wo…
May 2, 2023 7 tweets 3 min read
It is widely thought that neural networks generalize because of implicit regularization of gradient descent. Today at #ICLR2023 we show new evidence to the contrary. We train with gradient-free optimizers and observe generalization competitive with SGD.
openreview.net/forum?id=QC10R… An alternative theory of generalization is the "volume hypothesis": Good minima are flat, and occupy more volume than bad minima. For this reason, optimizers are more likely to land in the large/wide basins around good minima, and less likely to land in small/sharp bad minima. Image
Mar 13, 2023 13 tweets 4 min read
Here's the real story of #SiliconValleyBank, as told the boring way through tedious analysis of balance sheets and SEC filings 🧵 Throughout 2021 startups were raising money from VCs and stashing it in SVB. Deposits increased from $102B to $189B. That's an 85% change in one year. Wow! Image
Feb 27, 2023 15 tweets 5 min read
If you work for a US university, you have probably noticed the rollout of strict new policies mandating disclosures and approvals for funding, consulting, and COIs, and also threats of legal action for non-compliance. Here’s why this is happening now 🧵 Let's start at the beginning. In 2018, the DOJ implemented its new “China Policy.” The stated purpose of this program was to combat the perceived fears of Chinese espionage operations inside US Universities.
fbi.gov/investigate/co…
Feb 8, 2023 6 tweets 4 min read
We rack our brains making prompts for #StableDiffusion and Language Models. But a lot of prompt engineering can be done *automatically* using simple gradient-based optimization. And the cold calculating efficiency of the machine crushes human creativity. Prompts made easy (PEZ) is a gradient optimizer for text. It can convert images into prompts for Stable Diffusion, or it can learn a hard prompt for an LLM task. The method uses ideas from the binary neural nets literature that mashup continuous and discrete optimization.
Jan 25, 2023 12 tweets 5 min read
#OpenAI is planning to stop #ChatGPT users from making social media bots and cheating on homework by "watermarking" outputs. How well could this really work? Here's just 23 words from a 1.3B parameter watermarked LLM. We detected it with 99.999999999994% confidence. Here's how 🧵 This article, and a blog post by Scott Aaronson, suggest that OpenAI will deploy something similar to what I describe. The watermark below can be detected using an open source algorithm with no access to the language model or its API.
businessinsider.com/openai-chatgpt…
Dec 6, 2022 10 tweets 2 min read
How many GPUs does it take to run ChatGPT? And how expensive is it for OpenAI? Let’s find out! 🧵🤑 We don’t know the exact architecture of ChatGPT, but OpenAI has said that it is fine-tuned from a variant of GPT-3.5, so it probably has 175B parameters. That's pretty big.
Nov 25, 2022 9 tweets 5 min read
Neural algorithm synthesis is done by giving models a human-crafted programming language and millions of sample programs. Recently, my lab looked at whether neural networks can synthesize algorithms on their own without these crutches. They can, with the right architecture. 🧵 Here's an algorithmic reasoning problem where standard nets fail. We train resnet18 to solve little 13x13 mazes. It accepts a 2D image of a maze and spits out a 2D image of the solution. Resnet18 gets 100% test acc on unseen mazes of the same size. But something is wrong…
Nov 22, 2022 10 tweets 5 min read
I always thought #StableDiffusion prompts needed the right combination of words. But byte-pair encoding can represent anything you can type, including math formulas and emojis. Turns out you don't need any words at all! Here's how and why this works...🧵

Prompt: e=mc^2 Image Prompts are fed to stable diffusion as binary code, with each letter/symbol represented as several bytes. Then a "tokenizer" looks for commonly occurring spans of adjacent bytes and groups them into a single known "word". Stable diffusion only knows 49408 words.

Here's "🧛🦇🗡️" ImageImage
Nov 1, 2022 11 tweets 5 min read
My work on AI "invisibility cloaks" that suppress person detectors was on the Reddit front page last week! Now I've been approved to do an official "Ask me anything" on Reddit this Thurs. See you Nov 3rd at 12:30pm EST on reddit.com/r/IAmA/!
tinyurl.com/y2d4v29z Some background: it is well-known that adversarial attacks work well on image *classifiers*, but *detectors* are much more robust. The goal of our cloak project was to see whether physical adversarial examples could defeat a person detector.
Aug 24, 2022 23 tweets 10 min read
Diffusion models like #DALLE and #StableDiffusion are state of the art for image generation, yet our understanding of them is in its infancy. This thread introduces the basics of how diffusion models work, how we understand them, and why I think this understanding is broken.🧵 Diffusion models are powerful image generators, but they are built on two simple components: a function that degrades images by adding Gaussian noise, and a simple image restoration network for removing this noise.
Aug 18, 2022 9 tweets 2 min read
Why have diffusion models displaced GANs so quickly? Consider the tale of the (very strange) first DALLE model. In 2021, diffusions were almost unheard of, yet the creators of DALLE had already rejected the GAN approach. Here’s why. 🧵 DALLE is an image model, but it was built like a language model. The model trained on image-caption pairs. Captions were encoded as 256 tokens. Images were broken into a 32x32 grid of patches, which were each encoded as a token. All tokens were merged into a single sequence. Image
Jul 13, 2022 10 tweets 3 min read
SSIM has become a common loss function in computer vision. It is used to train monocular depth models for self-driving cars, invert GANs, and fit NeRF models to training images. The explosion of SSIM-based models raises a fundamental question: what the hell is SSIM? 🧵 SSIM measures the similarity between two images. Humans are insensitive to the absolute brightness/color of pixels, but very sensitive to the location of edges and textures. SSIM mimics human perception by focusing primarily on edge and textural similarities.
Jul 5, 2022 9 tweets 2 min read
Just how much have language models grown in the last 4 years? Let's have a look. In 2018, the puny BERT “large” model premiered with a measly 354M parameters. It can be trained on a single 8xA100 node in 5 days. That costs $2K on AWS - almost free by LLM standards! 🧵 Then came Facebook’s equally tiny RoBERTa model. Built on BERT-large, but with mods for faster mixed-precision training, it completed 40 epochs on its beefed up training set using 1000 GPUs for a week. You could train this on the cloud for $350K. NBD.
Feb 4, 2022 4 tweets 2 min read
"Plug-In" inversion directly produces images from ViTs and CNNs at the pixel level, with no GAN prior. We then see what networks really care about, not just what the GANs want us to see. Here's a few examples. First, I'll pull you in with these tugboats... My student @aminjuun has been working like a dog on this project. This dog, specifically.