Thread by @praeclarum

I read a research paper last night that I just can't stop thinking about.

The idea was to write a differentiable rigid body physics simulator.

I'm just blown away. I'm trying to express how big of an epiphany this has been for me, but I just can't find the words.

Let's start with this: when modeling control systems, you express them in 1 of 3 forms:

1. Linear ordinary differential equations (preferred)
2. Differential equations
3. Code

All control theory is taught assuming #1 because its the only model that can has closed-form solutions. Problem is, nothing in the world is linear, so you cheat a lot.

#2 is interesting but very hard to analyze

#3 don't even try to analyze code

A linear controller like PID (error proportional, integral, differential) is incredibly powerful because it can control most systems, but is annoyingly hard (and dangerous) to tune.

So we need models to tune our controllers against. But this meant linearizing our models and pain

When creating a controller for a complex system, say a human walking, we either split the system into many parts and solve simpler control problems resulting in predictable/bad gates.

The alternative is to use ML to learn control but it's nearly a blind search and is slow

Even with a complex model you will have hundreds (thousands) or parameters that you don't know the value of. Your model is useless unless these values match the real world. So you use an optimizer that blindly tries to guess those params (because you can't analyze code)

Here's the idea: Code the simulator using a neural network library like Torch. If expressed completely in tensors, then any value is differentiable. Which means the parameters of your model are differentiable. Which means you can do stochastic gradient decent to parameter-fit.

That use of SGD, not to find an arbitrary function, but to tune a function, embarrassingly, never occurred to me. It's a pretty obvious use, but I was so caught up in AI that I missed this amazing tool in front of me.

But we're not done. Sure I could write a simple Newtonian dynamics system in a data flow language, or maybe a MSD system, but the authors of this paper wrote an entire impulse based rigid body simulator. With collision detection. As a data flow graph.

I'm astounded.

While my mind was still being blown by that, they demonstrated that now that it's differentiable, it can be used directly in the training of a controller neural network.

The walls of the matrix are melting. The veil is being lifted.

And then the knock-out punch: Wrap that whole simulator into a recurrent structure. In programming we call this loop unrolling, and we're going to unroll time.

Now I can control the loss over time not only controlling the correctness of the controller but its time horizon/speed

It's terribly inefficient, but it doesn't matter. These networks have been designed to train millions of parameters in order to tell if an image is a hot dog or not -- they can handle physics simulation.

If you made it this far, thanks. I was trying to summarize all that into one tweet haha.

Here's the paper. It's not actually that good (explanations are lacking). But it has opened my mind more than anything else in the last few years.

arxiv.org/abs/1611.01652

It references this paper which was also need to fully crack my brain open.

journals.plos.org/plosone/articl…