This account has become mostly CT, but I still care (deeply) about deep learning and large language models.
While models have gotten bigger and better, it seems this is having surprisingly little effect on downstream applications...
🧵 (1/n)
The growth in parameter counts has been extraordinary. I had a tiny part to play, my friends and team-mates have been on the forefront, in the lab, taking state of the art language models from 300M params to 8B-11B (when I was there), to 1/2 T params
(2/n) developer.nvidia.com/blog/using-dee…
Work from Nvidia. MSFT, OpenAI, Google and FB research, has transformed NLP into a large scale deep learning field. It's amazing you can encode that many params, go through that many documents, in 100+ languages. Even handle everything as bytes...
I didn’t know, apparently all epidemiological models used by governments, assume a form of homogenous mixing — ie people infect each other randomly, either by region, or by age group.
Of course it’s not remotely true. IRL social graphs are very sparse!
There’s plenty of theory and software on modeling sparse interactions. But the health officials aren’t that good at math…
So they use a model with some form of underlying assumption, that the population is many N sub populations — children, pensioners, Floridians — who all infect each other randomly and between populations at a different lower rate.
Updated the Punks model.
* fixed how we decay bids over time
* Hoodies & Beanies with better prices (median around 300Ξ)
* About 300 Punks in that 250-400Ξ ranges
* Bottom value 100.7Ξ, median 113.6Ξ -- has been steady
* Market cap 1.69mΞ ~ 6.5B USD
In the model, we obviously want to fit to bids (as minimum) and offers (as max) as well as predicting tomorrow's sale.
Realistically, this means decaying the weight on bids over time. Say a bid for past 3 day is 100%... do you still care about a bid (or sale) from month ago? Yes
The question: hows quick to decay the data.
We now keep bids around for up to 120 days (!) -- seems high, but no Beanie has sold for two months.
No evidence that prices have dropped -- many relative to median/floor but not in absolute terms.
Also few good tips on extending the morning fast:
* water, coffee or tea if you like
* salt and lemon can make dramatic difference in haziness where you think you need sugar
Some protein relatively early in the day also good for athletes. Talking 10am or noon.
The fasted state is very important, for hormones, body regeneration, etc. Can’t be eating and digesting all day.
The fasted state does not begin as soon as you’re done eating.
A light walk after dinner helps. As do things like cinnamon.
A little surprised that Keith doesn’t do fasted workouts. Though he also trains twice a day. I do that…. sometimes. More during Covid at him in nyc actually. Went pretty Tsatsouline with pull-ups, dumbbells and kettlebells at home 👌