Dr. Chris Rackauckas Profile picture
Mar 31, 2023 9 tweets 5 min read Read on X
#sciml #machinelearning in chemical engineering using prior scientific knowledge of chemical processes? New paper: we dive deep into using universal differential equation hybrid models and see how well gray boxes can recover the dynamics.
arxiv.org/abs/2303.13555 #julialang
For learning these cases, we used neural networks mixed with known physical dynamics, and mixed it with orthogonal collocation on finite elements (OCFEM) to receive a stable simulation simulation and estimation process.
We looked into learning reaction functions embedded within diffusion-advection equations. This is where you have spatial data associated with a chemical reaction but generally know some properties of the spatial movement, but need to learn the (nonlinear) reaction dynamics
We found in multiple instances the method could extrapolate to non-trivial behaviors outside of the training data.
This was mixed with a sparse regression for automating the discovery of the missing reaction terms. @MilesCranmer's SymbolicRegression.jl outperformed the sparse regression techniques commonly used by SINDy.
In cases where the exact polynomial was not found, we were able to show that it found missing terms in the model whose first two terms of the Taylor series matched the original one. So not an exact recovery, but clearly recovering behavior of note!
Thus the symbolic regression was finding simplified models of the phenomena! Together this shows in some non-trivial chemical engineering cases that autocompleting models via Universal Differential Equations leads to some nice results.
If you want more information on the general method, check out the original UDE paper.

arxiv.org/abs/2001.04385
If you're curious about how the training process works and the interactions of stiffness with the adjoint method, see this tweet thread:

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Dr. Chris Rackauckas

Dr. Chris Rackauckas Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @ChrisRackauckas

Apr 6, 2023
Why are biologists adopting #julialang #sciml? Performance, metaprogramming, and the development of new abstractions are improving software tools for #computationalbiology #systemsbiology #bioinformatics. Check out this new paper in Nature Methods!

nature.com/articles/s4159…
In this we detail how #julialang's core compute model gives faster code, with a detailed calculation of the effects of the #python interpreter and kernel launching costs on simulation performance. It's pretty cool how one can pen and paper calculate the 100x expected difference. Image
Julia's ecosystem has a complete set of tools for mathematical modeling (#sysbio), #bioinformatics, #machinelearning, and #datascience which we contextualize in the field of biology. Image
Read 7 tweets
Oct 18, 2022
Differentiable programming (dP) is great: train neural networks to match anything w/ gradients! ODEs? Neural ODEs. Physics? Yes. Agent-Based models? Nope, not differentiable... or are they? Check out our new paper at NeurIPS on Stochastic dP!🧵

arxiv.org/abs/2210.08572
Problem: if you flip a coin with probability p of being heads, how do you generate a code that takes the derivative with respect to that p? Of course that's not well-defined: the coin gives a 0 or 1, so it cannot have "small changes". Is there a better definition?
Its mean (or in math words, "expectation") can be differentiable! So let's change the question: is there a form of automatic differentiation that generates a program which directly calculates the derivative with respect to the mean?
Read 19 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(