🧵 Could this be the ImageNet moment for scientific AI?

Today with @PolymathicAI and others we're releasing two massive datasets that span dozens of fields - from bacterial growth to supernova!

We want this to enable multi-disciplinary foundation model research. @PolymathicAI You might ask: why would such diverse training data help AI?

Well, as we've seen over the past few years, breadth of training can lead to stronger performance. We want AI to exploit common phenomena across sciences – such as waves!

Three years ago, I started working on an easy-to-use tool for interpretable machine learning in science. I wanted it to do for symbolic regression what Theano did for deep learning.

Today, I am beyond excited to share with you the paper describing it!
arxiv.org/abs/2305.01582

1. Symbolic Regression (SR) is a supervised learning task where the space of potential models is spanned by analytic expressions. Often, the goal is to find simple yet accurate expressions that lend themselves to interpretation🔍.

2.

Come to the @icmlconf ML4Astro workshop today to see some exciting new ideas for ML approaches to astronomy! icml.cc/virtual/2022/w…

Here are five contributed papers I coauthored which will be presented today–come check them out: "Learning Galaxy Properties from Merger Trees"
Led by @AstroCKragh, this paper asks: can we predict galaxy properties by using dark matter halo merger history (i.e., the merger tree)? It turns out: yes, and quite accurately too!

ml4astro.github.io/icml2022/asset…

Could machine learning rediscover the law of gravitation simply by observing our solar system?

With our new approach, the answer is *YES*.

Led by: @PabloLemosP
With: @Niall_Jeffrey @cosmo_shirley @PeterWBattaglia
Paper: arxiv.org/abs/2202.02306
Blog: astroautomata.com/paper/rediscov… Here are two classes of problems which we can already solve:

1. Known law, unknown parameters => parameter inference.

2. Unknown law, known parameters => model discovery (eg,

https://twitter.com/MilesCranmer/status/1275241300829384704

/ arxiv.org/abs/2006.11287)

Here we look at unknown law AND unknown parameters!

Our paper demonstrating the power of Bayesian Neural Networks for planetary dynamics comes out in PNAS today!

pnas.org/content/118/40… (open access)

This paper explores a match made in heaven: chaotic systems and Bayesian neural networks.

Thread: 2/n
Chaos is synonymous with the "butterfly effect," where infinitesimal changes grow into significant differences over time.

Here's a demo with a double pendulum (from commons.wikimedia.org/w/index.php?ti…):

Here's a thread on lesser-known tools and packages that I could not live without, starting with Python.
(suggestions are very welcome!)

einops:
- github.com/arogozhnikov/e…
- Easily-interpretable reshapes + tiling + aggregations for numpy/torch/tf/etc

1/n celluloid:
- github.com/jwkvam/cellulo…
- Create matplotlib animations by looping over your plot script and calling camera.snap() each time

2/n

Very excited to share our new paper "Discovering Symbolic Models from Deep Learning with Inductive Biases"!

We describe an approach to convert a deep model into an equivalent symbolic equation.

Blog/code: astroautomata.com/paper/symbolic…
Paper: arxiv.org/abs/2006.11287

Thread👇
1/n Work w/ Alvaro Sanchez-Gonzalez and @PeterWBattaglia (@DeepMind), Rui Xu (@Princeton), @KyleCranmer (@NYUDataScience), and @DavidSpergel and @cosmo_shirley (@FlatironCCA).

2/n

Machine learning is both underused and overused in astrophysics.

What do I mean by this? 👇

1/n Overused:
Many ML applications in astrophysics aren't necessary. When applied to the wrong problem, modern ML can be overinterpreted and overextrapolated.

E.g., when deep NNs are applied to low-data low-dimensional problems.

2/n

1/10 This was a phenomenal discussion. I have many more questions than answers now but I think that's a good thing.

Here's a list of some interesting papers mentioned.

https://twitter.com/MilesCranmer/status/1250974181132513280

2/10 First, papers referenced in main tweet:
LNNs - arxiv.org/abs/2003.04630
HNNs - arxiv.org/abs/1906.01563
Graph Nets - arxiv.org/abs/1806.01261 (+ refs therein...)
Group-CNNs - proceedings.mlr.press/v48/cohenc16.p…