OK, this website still seems to be working…so…time to share our latest preprint!
Very pleased to be able to share this one: is attention all you need to solve the Schrödinger equation? arxiv.org/abs/2211.13672
For the last several years, numerous groups have shown that neural networks can make calculations in quantum chemistry much more accurate - FermiNet, PauliNet, etc. We wrote a review article about it here:
Most work since then has only made small tweaks to these basic neural network antazes. Instead, we tried to reinvent neural network ansatzes from the ground up. The result is a model we call the Psiformer: basically, a Transformer encoder designed for quantum chemistry.
Most impressively, the bigger the system size, the bigger the improvement with the Psiformer relative to the FermiNet. On the largest system we looked at, the benzene dimer (84 electrons!) the Psiformer with VMC is more accurate than the FermiNet with *diffusion* Monte Carlo!
I really never thought I’d be an “attention is all you need” guy, but at least in this case, it seems like neural network ansatzes using self-attention are a clear improvement over prior models, and present a path to unprecedented accuracy in quantum chemical calculations.
This was work led by Ingrid von Glehn and in collaboration with @exp_n. For those at #NeurIPS2022, I’ll be speaking about this and other topics on deep learning and quantum chemistry at the #ML4PhysicalSciences workshop on Saturday!
God dammit, threading is broken. This website really is not doing well. I meant to include one more tweet showing that the Psiformer is more accurate than FermiNet on all the systems from the original FermiNet paper:
*ansatzes, not antazes. What's an antaze?
Share this Scrolly Tale with your friends.
A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.