1. It's extremely readable. Go check out the code yourself! It's only about 500 LoC. I recommend this fork I started which makes the code more idiomatic & adds performance improvements (though the code quality of the original was excellent to start with): github.com/fchollet/stabl…
2. It's fast. How much faster? It depends on your hardware (try switching on `jit_compile` to see if you can get a greater speedup that way). Benchmark it on your system and let me know!
3. It works out of the box on M1 MacBooPros GPUs. Just install the proper requirements `requirements_m1.txt` and get going.
(This required no extra work on the repo.)
4. It can do TPU inference out of the box: just get a TPU VM and add a TPU strategy scope to the code. This can yield a dramatic speedup (and cost reduction) when doing large-batch inference.
5. It can also do multi-GPU inference out of the box (same, with a MirroredStrategy scope). It just... works.
6. You can export the underlying 3 Keras models to TFLite and TF.js.
This means that you can create AI art apps that run on the device (in the browser, with local GPU acceleration, or on an Android / iOS device, also with local hardware acceleration). No server costs!
Sounds exciting? Go try it, fork it, hack it. And if you want to learn more about what it looks like to work with Keras and TensorFlow, go read the code: github.com/fchollet/stabl…
Huge thanks to @divamgupta for creating this port! This is top-quality work that will benefit everyone doing creative AI.
I'm always amazed by the velocity of the open-source community 👍
• • •
Missing some Tweet in this thread? You can try to
force a refresh
Today OpenAI announced o3, its next-gen reasoning model. We've worked with OpenAI to test it on ARC-AGI, and we believe it represents a significant breakthrough in getting AI to adapt to novel tasks.
It scores 75.7% on the semi-private eval in low-compute mode (for $20 per task in compute ) and 87.5% in high-compute mode (thousands of $ per task). It's very expensive, but it's not just brute -- these capabilities are new territory and they demand serious scientific attention.
While the new model is very impressive and represents a big milestone on the way towards AGI, I don't believe this is AGI -- there's still a fair number of very easy ARC-AGI-1 tasks that o3 can't solve, and we have early indications that ARC-AGI-2 will remain extremely challenging for o3.
This shows that it's still feasible to create unsaturated, interesting benchmarks that are easy for humans, yet impossible for AI -- without involving specialist knowledge. We will have AGI when creating such evals becomes outright impossible.
When we develop AI systems that can actually reason, they will involve deep learning (as one of two major components, the other one being discrete search), and some people will say that this "proves" that DL can reason.
No, it will have proven the thesis that DL is not enough, and that we need to combine DL with discrete search.
From my DL textbook (1st edition), published in 2017. Seven years later, there is now overwhelming momentum towards this exact approach.
I find it especially obtuse when people point to progress on math benchmark as evidence of LLMs being AGI, given that all of this progress has been driven by methods that leverage discrete search. The empirical data is completely vindicating that DL in general, and LLMs in particular, can't do math on their own, and that we need discrete search.
In the last Trump administration, legal, high-skilled immigration was cut by ~30% before Covid, then by 100% after Covid (which was definitely a choice: a number of countries kept issuing residency permits and visas). However illegal immigrant inflows did not go down (they've been stable since the mid-2000s).
If you're a scientist or engineer applying for a green card, you're probably keenly aware that your chances of eventually obtaining it are highly dependent on the election. What you may not know is that, if you're a naturalized citizen, your US passport is also at stake
The last Trump administration launched a "denaturalization task force" aiming at taking away US citizenship from as many naturalized citizens as possible, with an eventual target of 7M (about one third of all naturalized citizens). Thankfully, they ran into a little problem: the courts.
When we say deep learning models operate via memorization, the claim isn't that they work like literal lookup tables, only being able to make sense of points that are exactly part of their training data. No one has claimed that -- it wouldn't even be true of linear regression.
Of course deep learning models can generalize to unseen data points -- they would be entirely useless if they couldn't. The claim is that they perform *local generalization*: generalization to known unknowns, to degrees of variability for which you can provide a dense sampling at training time.
If you take a problem that is known to be solvable by expert humans via pure pattern recognition (say, spotting the top move on a chess board) and that has been known to be solvable via convnets as far back as 2016, and you train a model on ~5B chess positions across ~10M games, and you find that the model can solve the problem at the level of a human expert, that isn't an example of out-of-distribution generalization. That is an example of local generalization -- precisely the thing you expect deep learning to be able to do.
Fact check: my 3-year old builds Lego sets (age 5+ ones) on his own by following the instruction booklet. He started doing it before he turned 3 -- initially he needed externally provided error correction and guidance, but now he's just fully autonomous. Can't handle sets for ages 8+ yet though. We'll see what he does at 5.
He also builds his own ideas, which feature minor original inventions. Like this "jeep" which has a spare tire on the back -- not something he saw in any official set. Lego is the best toy ever by the way
Or this Lego garden (fresh from today). It has a hut with a cool door. It looks chaotic, but everything on here has a purpose. Everything is intended to be something (the tire on a stick is a tree, the tiny cone on the ground is a water sprinkler...)
I'm partnering with @mikeknoop to launch ARC Prize: a $1,000,000 competition to create an AI that can adapt to novelty and solve simple reasoning problems.
I published the ARC benchmark over 4 years ago. It was intended to be a measure of how close we are to creating AI that can reason on its own – not just apply memorized patterns.
ARC tasks are easy for humans. They aren't complex. They don't require specialized knowledge – a child can solve them. But modern AI struggles with them.
Because they have one very important property: they're designed to be resistant to memorization.