AI assistance has a bright future -- applications where humans can guide a search/generation process and correct its output as needed. But keep in mind that's not AI autonomy -- AI agents capable of action on their own in the real world beyond a very narrow scope of automation... AI autonomy is many orders of magnitude harder, and current techniques are infinitely far from it. They're not even pointing in this general direction.

The good news is, you can produce tons o value with AI assistance -- though that's as much a design problem as a tech problem.

If you want to define AGI in terms of outcomes rather than in terms of its function, then "pass this test designed for humans" doesn't cut it. I'd just use the bar set by Sam Altman in 2019: AGI is that which will "capture the light cone of all future value in the universe". We will know we have AGI when the majority of the world's GDP is being produced by autonomous AI agents.

(And once we get on this trajectory it will be pretty clear, so we'll know much earlier -- it will probably be obvious at 5-10% of global GDP)

What people don't realize is that this is the default state of housing in a market economy, and it's great for consumers. Turning aging homes into an investment vehicle by artificially constraining supply, the way it's done in the US and many other places, is extremely harmful. You can achieve the same with any good for which there is enough demand -- cars, professional licenses, etc. Just produce new units at a rate that's 50% that of demand growth and watch prices steadily increase over time.

It's accurate that can tell a lot about someone from their spouse. But the way it's framed in the post is unhinged, effectively saying "if you have a high status spouse, you see yourself as high status, but if you 'settle' for a low status spouse, you see yourself as low status"

https://twitter.com/paulg/status/1615813707346415650

Marriage isn't a status game, and it says a lot about you if you pick your spouse that way.

What you actually want to find out is:

- is their spouse a good person
- do they have a happy / loving relationship with their spouse
- how many times have they been married

I rarely make predictions about the future. But I did make one last year, so it's worth fact-checking it. On June 16 I predicted that the market was "not to far from" the year's bottom.

VTI (i.e. the overall stock market) closed at 183.02 that day.

https://twitter.com/fchollet/status/1537466493134262272

There were exactly 9 days in 2022 with lower closing prices, September 26, 27, 29, 30, and October 7, 10, 11, 12, 14. The absolute lowest close was 179.30, exactly 2.0% below the close for June 16.

It's *actual data* time...

TensorFlow & Keras usage is at an all-time high, at >2.5M users. It has increased ~30% yoy.

In the largest developer survey in the world last year (60k respondents), 13% of *all devs* said they used TensorFlow. That's 1.5x more devs than PyTorch. In the largest 2022 survey of the ML landscape specifically, 57% of ML devs said they used TF. Only 38% said they used PyTorch.

Again, that's 1.5x more devs than PyTorch. This 1.5x ratio is found in virtually all of our metrics.

It's clear that we're far from peak LLM performance -- these models will keep getting better. It's also clear that pure generation is just the first step -- we can largely alleviate the LLM reliability issue by using them as information retrieval devices over a knowledge corpus. Likewise we can interface LLMs with an array of symbolic tools that can shore up their weaknesses -- calculators, interpreters, discrete search programs, SAT solvers, etc.

We've recently released KerasNLP 0.4, the most complete suite of Keras functionality for NLP to date. Learn about it in the starter guide: keras.io/guides/keras_n…

Here's a quick intro thread! ⬇️ KerasNLP isn't a whole new framework, it's a direct extension of Keras. A modular toolbox for your Keras-based NLP workflows -- it just contains specialized Keras layers and models. So if you already know Keras, it will feel familiar.

The current climate in AI has so many parallels to 2021 web3 it's making me uncomfortable. Narratives based on zero data are accepted as self-evident. Everyone is expecting as a sure thing "civilization-altering" impact (& 100x returns on investment) in the next 2-3 years Personally I think there's a bull case and bear case. The bull case is way way more conservative than what the median person on my TL considers as completely self-evident. And the actual outcome we'll see is statistically likely to lie in between, somewhat closer to the bear case

When it comes to similarities between the brain and deep learning, what's really striking is that everything that was actually bio inspired (e.g. sigmoid/tanh activations, spiking NNs, hebbian learning, etc.) had been dropped, while... (Cont.) ...everything that has durably outperformed (backprop, relu, dropout, MultiheadAttention, MixUp, separable convs, BatchNorm, LayerNorm and many others) makes no sense biologically and has basically been developed by trying a bunch of things and keeping what worked empirically

All complex systems are modular and hierarchical -- by necessity. So when two complex systems are both modular and hierarchical (say, the brain and a neural network), that doesn't mean they're similar to each other. It just means they're both complex systems. Otherwise you could say that the human body is similar to a neural network, or the US military is similar to a neural network, etc... all complex systems will necessarily share the set of characteristics required for managing complexity.

First one has been the case since at least 2013 (automated linters that create GitHub PRs and the like). The other two predictions appear to stem from a profound misunderstanding of AI progress and the nature of software engineering.

https://twitter.com/dmvaldman/status/1610287031921569792

Then again folks have been making unhinged predictions like this for years. Radiologists would be out of a job by 2018. Taxi drivers by 2019. Not to mention, 2022's "art is dead".

"Software engs will be out of a job within a few years" has been a recurring theme since the 70s.

It's been a huge year for Keras! Big launches, adoption growth, learnings... Here's our 2022 year-in-review ⬇️ First of all -- Keras adoption has grown to its largest numbers to date. We're cruising at 10M downloads / month and 550k monthly unique visitors on keras.io (plus 2x more on tensorflow.org), the highest we've ever had.

Any task, even those canonically considered to be perception problems, can be solved with reasoning (if working with very little data). Inversely any task, even those canonically considered to be reasoning problems, can be solved with pattern recognition (given sufficient data).

https://twitter.com/fchollet/status/1606238157195853824

The latter is easy to picture -- if you've seen thousands of RPM IQ puzzles, you will develop pattern recognition intuition for the templates they follow and you'll become able to solve them in your sleep. Every new puzzle you see will be a small variation of a known pattern.

It's amazing how fast the Twitter UX has been degrading over the past few weeks. So far:

https://twitter.com/hardmaru/status/1606237890492526592

1. One of the most useful features of the app (at least to me), the "verified" notifications tab, has become entirely useless

2. The mentions tab never ever loads (didn't use it, but still)

3. Increased latency in the Android app (purely subjectively, I have no data)

So far all evidence that LLMs can perform few-shot reasoning on novel problems seems to boil down to "LLMs store patterns they can reapply to new inputs", i.e. it works for problems that follow a structure the model has seen before, but doesn't work on new problems. This is circumstantially confirmed by the fact that, when translating ARC problems to sequences, the largest LLMs out there (not just GPT-3, but *much* larger ones as well) score close to zero. Problems that do get solved are known ones, such as a simple left/right flip.

You still have 12 days to enter the Keras community prize. Submit your projects based on KerasCV StableDiffusion and win prizes. discuss.tensorflow.org/t/announcing-t… Some previous examples:

1. Latent space walks keras.io/examples/gener…

How it started / how it's going / wait what now? Seriously wondering how long I have till the ban hits

The top productivity hack in software development is to build foundations that are dependable enough that you can keep building on top without having to go back and modify them as you go. 100x time saver. This is extremely hard to achieve. You get there not via clever code by via clean abstractions.

Note on spam filtering ML. If you notice that all of your spam comes from the same 9 ISPs in SE Asia & Eastern Europe, you might be tempted to "solve" the problem by blocking these ISPs (at the cost of a lot of collateral damage, since you'd be blocking a lot of legit usage)... ...but that would be a terrible idea, because your spammers will adapt to your change in a matter of hours or days (they were using these ISPs out of convenience, not because they had to). You will have to restart from scratch -- except most of your historical data is now stale.

What's interesting about the journalists getting banned from Twitter by E.M. is how incredibly normie they are. Like Steve Herman from Voice of America, one of the most objective and least political journalists there was on this website. en.m.wikipedia.org/wiki/Steven_L.… Soon only the far-right pundits that E.M. is constantly replying to with feverish obsequity will be left. Truth Social but for Elon fans.