New paper🚨: Physics of Skill Learning

Training dynamics is complicated, but are there simple "physical laws" behind it?

We take physicists' approach of simplification and abstraction: Simple models like "spherical cows" are surprisingly effective!



🧵 arxiv.org/pdf/2501.12391 1/N We start from a motivating question: do skills learn in series or in parallel during network training? 🤔

Excited to share our new paper KAN 2.0: Kolmogorov-Arnold Networks meet Science 🚀

The problem with AI + Science is that these two disciplines use different "languages" (connectionism vs symbolism). KAN 2.0 attempts to unify them: smooth transitions from science to KAN and back. Paper:
Code:
Great collaboration with @pika7ma, @RoyWang67103904 , Wojciech Matusik, and @tegmarkarxiv.org/abs/2408.10205
github.com/KindXiaoming/p…

MLPs are so foundational, but are there alternatives? MLPs place activation functions on neurons, but can we instead place (learnable) activation functions on weights? Yes, we KAN! We propose Kolmogorov-Arnold Networks (KAN), which are more accurate and interpretable than MLPs.🧵 Paper:
Github:
Documentation: arxiv.org/abs/2404.19756
github.com/KindXiaoming/p…
kindxiaoming.github.io/pykan/

To make neural networks as modular as brains, We propose brain-inspired modular training, resulting in modular and interpretable networks! The ability to directly see modules with naked eyes can facilitate mechanistic interpretability. It’s nice to see how a “brain” grows in NN! The paper can be found here: kindxiaoming.github.io/pdfs/BIMT.pdf