Thread by @ch402

States force the world into simplistic models to make it legible to bureaucracy, sometimes at the cost of great harm. (see James Scott)

Software often forces even more simple and rigid schemas.

But deep learning makes fuzzy human ideas computable. Can it reverse the trend?

Scott's _Seeing Like a State_ describes States as needing to make the world legible to their bureaucracies. In order to do this, they use simplified models of the world and then often force it on their subjects.

This causes grave harms and also just missed opportunities.

It strikes me this isn't unique to States. It's an issue of automation and working at scale.

A mundane example I was reading about today: Google making assumptions about names that harm outliers.

Programmers see this kind of issue all the time. Handling all the edge cases is _hard_. And the real world is full of edge cases.

"Nature is wiggly, everything wiggles, the outlines of the hills, the shapes of the tress ... And we say, “well let’s get things straightened out, let’s get this ironed out, let’s get it all squared away. ... all this wiggliness is too complicated." - Alan Watts

But neural networks are able to learn representations of fuzzy things.

"What is a dog?" is actually a very fuzzy, wiggly concept.

And so neural networks take all these things that previously couldn't be systematized, and being to make them computationally accessible.

We rightly worry about AI enabling authoritarianism and police states. But I'm excited for us to also look at the upside:

Can we enable more understanding, flexible, humane governments that don't try to fit the world in simple boxes?

I don't think that's the default outcome, but I think there might be something beautiful there to aspire towards.

I think people often don't see it this way because they're thinking of neural networks as black boxes that take X and give them Y.

But that's just scratching the surface. The really exciting thing is the representations they form, full of rich abstractions.

Related reading on the "neural networks allowing creating to abstractions we can use" idea:

Deep Learning & Human Beings - colah.github.io/posts/2015-01-…

Artificial Intelligence Augmentation: distill.pub/2017/aia/