More from @PatrickKidger

Patrick Kidger

@PatrickKidger

Feb 8

⚡️ My PhD thesis is on arXiv! ⚡️

To quote my examiners it is "the textbook of neural differential equations" - across ordinary/controlled/stochastic diffeqs.

w/ unpublished material:
- generalised adjoint methods
- symbolic regression
- + more!

arxiv.org/abs/2202.02435

v🧵 1/n

If you follow me then there's a decent chance that you already know what an NDE is. (If you don't, go read the introductory Chapter 1 to my thesis haha -- it's only 6 pages long.) Put a neural network inside a differential equation, and suddenly cool stuff starts happening.

2/n

Neural differential equations are a beautiful way of building models, offering:
- high-capacity function approximation;
- strong priors on model space;
- the ability to handle irregular data;
- memory efficiency;
- a foundation of well-understand theory.

3/n

Read 21 tweets

Patrick Kidger

@PatrickKidger

Sep 13, 2021

Announcing Equinox v0.1.0! Lots of new goodies for your neural networks in JAX.

-The big one: models using native jax.jit and jax.grad!
-filter, partition, combine, to manipulate PyTrees
-new filter functions
-much-improved documentation
-PyPI availability!

A thread:
1/n 🧵

First: simple models can be used directly with jax.jit and jax.grad. This is because Equinox models are just PyTrees like any other. And JAX understands PyTrees.

2/n

More complex models might have arbitrary Python types in their PyTrees -- we don't limit you to just JAX arrays.

In this case, filter/partition/combine offer a succient way to split one PyTree into two, and then recombine them.

3/n

Read 9 tweets

Patrick Kidger

@PatrickKidger

Aug 3, 2021

Announcing Equinox!

github.com/patrick-kidger…

A JAX neural network library with
- a PyTorch-like class API for model building
- whilst *also* being functional (no stored state)

It leverages two tricks: *filtered transformations* and *callable PyTrees*.

1/n🧵

First of all, I know what you're thinking. We already have e.g. Flax and Haiku (+ a few others as well).

What's new, and do we really need another?

To the best of my knowledge, Equinox overcomes some of the core technical difficulties faced in previous libraries.

2/n

We love having a PyTorch-like class API for model building.

We love having JAX-like functional programming.

But these seem like completely different paradigms, and making them work together is tricky.

3/n

Read 20 tweets

Patrick Kidger

@PatrickKidger

May 12, 2021

New paper: Neural Rough Differential Equations !

Greatly increase performance on long time series, by using the mathematics of rough path theory.

arxiv.org/abs/2009.08295
github.com/jambo6/neuralR…

Accepted at #ICML2021!

🧵: 1/n
(including a lot about what makes RNNs work)

(So first of all, yes, it's another Neural XYZ Differential Equations paper.

At some point we're going to run out of XYZ differential equations to put the word "neural" in front of.)

2/n

As for what's going on here!

We already know that RNNs are basically differential equations.

Neural CDEs are the example closest to my heart. These are the true continuous-time limit of generic RNNs:
arxiv.org/abs/2005.08926
github.com/patrick-kidger…

3/n

Read 22 tweets

Share this page!

Patrick Kidger

People who liked this thread also liked...

Try unrolling a thread yourself!

More from @PatrickKidger

Patrick Kidger

Patrick Kidger

Patrick Kidger

Patrick Kidger

Did Thread Reader help you today?

Don't want to be a Premium member but still want to support us?