I had an argument with @PreetumNakkiran about MLPs 4 years ago. He said with enough data + compute the MLP/ConvNet gap would go to 0. I was convolution-pilled and convinced this wasn't possible. He was right: arxiv.org/abs/2306.13575 @PreetumNakkiran One caveat here is that their ImageNet number is still *really* bad for the amount of compute they used, for context in 2013 you could achieve ~53% top-1 with fisher vectors. So while we can use much more compute to fill the gap..should you?

Finally get to try the new bart trains! I've been literally waiting months for this. Will live tweet the experience. First impressions inside: much more colorful seats. Bright blue and neon green seats made of a rubber-like material as opposed to dull fabric.