Here's the real story of #SiliconValleyBank, as told the boring way through tedious analysis of balance sheets and SEC filings 🧵
Throughout 2021 startups were raising money from VCs and stashing it in SVB. Deposits increased from $102B to $189B. That's an 85% change in one year. Wow!
Most news sources claim that SVB stashed this money in relatively safe treasury securities. This is something important that most media sources got wrong. forbes.com/sites/billcone…
The balance sheet reveals that all of that money, over $80B, was invested in mortgage backed securities (MBS). MBS are far more susceptible to interest rate risk than treasury bonds, especially when interests rates get high.
Rather than increase liquid assets in proportion to their deposits, all this money (and more) got invested in MBS. By the end of 2021, SVB had *decreased* liquid assets (cash and short-term securities) by $7B, setting them up for a liquidity crisis if the tech economy reversed.
Then came 2022. The tech sector cooled off. Startups stopped raising money and began spending down their balances. Net withdrawals in 2022 were $16B - that’s more than SVB’s cash reserves, forcing them to start selling their bonds.
As inflation loomed, the Fed pumped up interest rates…a lot. SVB had purchased low-interest bonds and now had to sell before their maturity date. But nobody wants low interest bonds when the Fed is selling high interest bonds. SVB had to sell below face value, taking a loss.
By 2023, SVB was solvent...technically. They had $211B in assets (largely MBS), which was more than their $195B in liabilities (mostly deposits). $16B more, to be precise. But here’s the rub: Their balance sheet uses the face value of bonds, not the market value.
The market value of their bonds had decreased by nearly $16B because of interest rate hikes, making their fair market assets almost equal to their liabilities. If interest rates ticked up just a *teensy* bit more, the market value would be less than they owed depositors.
Interest rates didn't just tick up a teensy bit in 2023 - they went up a half point. We don’t have a Q1 report, but SVB’s fair market value almost certainly fell too low to cover its liabilities. Meanwhile, withdrawals almost certainly accelerated.
SVB's problem seems to have gone unnoticed until they announced a stock sale to raise money to close this gap in their finances. NBD, right? Unfortunately, a stock sale is a sure-fire way to get people to scrutinize your balance sheet. And the rest is history.
Some say SVB was just another victim of high interest rates. This overlooks some of the strange decisions they made. Bank of America holds 12% of its deposits in cash. Wells Fargo has 17%. SVB held only 8% after the 2021 rally, despite poor diversification and higher risk.
You’d think a bank that saw an 85% fluctuation in deposits in one year would have a plan to withstand volatility. But the storm arrived and SVB didn’t own a raincoat.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
If you work for a US university, you have probably noticed the rollout of strict new policies mandating disclosures and approvals for funding, consulting, and COIs, and also threats of legal action for non-compliance. Here’s why this is happening now 🧵
Let's start at the beginning. In 2018, the DOJ implemented its new “China Policy.” The stated purpose of this program was to combat the perceived fears of Chinese espionage operations inside US Universities. fbi.gov/investigate/co…
In practice, the DOJ used the policy to investigate people of Chinese descent, usually without evidence of espionage. Many people were arrested and jailed with no formal charges at all. reuters.com/world/us/trump…
We rack our brains making prompts for #StableDiffusion and Language Models. But a lot of prompt engineering can be done *automatically* using simple gradient-based optimization. And the cold calculating efficiency of the machine crushes human creativity.
Prompts made easy (PEZ) is a gradient optimizer for text. It can convert images into prompts for Stable Diffusion, or it can learn a hard prompt for an LLM task. The method uses ideas from the binary neural nets literature that mashup continuous and discrete optimization.
PEZ can even create a prompt to represent a face...as the hypothetical offspring of multiple celebrities ¯\_(ツ)_/¯
#OpenAI is planning to stop #ChatGPT users from making social media bots and cheating on homework by "watermarking" outputs. How well could this really work? Here's just 23 words from a 1.3B parameter watermarked LLM. We detected it with 99.999999999994% confidence. Here's how 🧵
This article, and a blog post by Scott Aaronson, suggest that OpenAI will deploy something similar to what I describe. The watermark below can be detected using an open source algorithm with no access to the language model or its API. businessinsider.com/openai-chatgpt…
Language models generate text one token at a time. Each token is selected from a “vocabulary” with about 50K words. Before each new token is generated, we imprint the watermark by first taking the most recent token and using it to seed a random number generator (RNG).
How many GPUs does it take to run ChatGPT? And how expensive is it for OpenAI? Let’s find out! 🧵🤑
We don’t know the exact architecture of ChatGPT, but OpenAI has said that it is fine-tuned from a variant of GPT-3.5, so it probably has 175B parameters. That's pretty big.
How fast could it run? A 3-billion parameter model can generate a token in about 6ms on an A100 GPU (using half precision+tensorRT+activation caching). If we scale that up to the size of ChatGPT, it should take 350ms secs for an A100 GPU to print out a single word.
Neural algorithm synthesis is done by giving models a human-crafted programming language and millions of sample programs. Recently, my lab looked at whether neural networks can synthesize algorithms on their own without these crutches. They can, with the right architecture. 🧵
Here's an algorithmic reasoning problem where standard nets fail. We train resnet18 to solve little 13x13 mazes. It accepts a 2D image of a maze and spits out a 2D image of the solution. Resnet18 gets 100% test acc on unseen mazes of the same size. But something is wrong…
If we test the same network on a larger maze it totally fails. The network memorized *what* maze solutions look like, but it didn’t learn *how* to solve mazes.
We can make the model synthesize a scalable maze-solving algorithm just by changing its architecture...
I always thought #StableDiffusion prompts needed the right combination of words. But byte-pair encoding can represent anything you can type, including math formulas and emojis. Turns out you don't need any words at all! Here's how and why this works...🧵
Prompt: e=mc^2
Prompts are fed to stable diffusion as binary code, with each letter/symbol represented as several bytes. Then a "tokenizer" looks for commonly occurring spans of adjacent bytes and groups them into a single known "word". Stable diffusion only knows 49408 words.
Here's "🧛🦇🗡️"
You might think 49408 is a lot. Well, it's not. Here's the first 1200 words in the vocabulary. They don't get you very far. The words are auto-selected by a simple algorithm and half are junk. And what are the weird "�" entries? We'll get back to them later...