OK, this website still seems to be working…so…time to share our latest preprint!
Very pleased to be able to share this one: is attention all you need to solve the Schrödinger equation? arxiv.org/abs/2211.13672
Very pleased to be able to share this one: is attention all you need to solve the Schrödinger equation? arxiv.org/abs/2211.13672
For the last several years, numerous groups have shown that neural networks can make calculations in quantum chemistry much more accurate - FermiNet, PauliNet, etc. We wrote a review article about it here:
https://twitter.com/jhrmnn/status/1564152388491132928
Most work since then has only made small tweaks to these basic neural network antazes. Instead, we tried to reinvent neural network ansatzes from the ground up. The result is a model we call the Psiformer: basically, a Transformer encoder designed for quantum chemistry. 
Most impressively, the bigger the system size, the bigger the improvement with the Psiformer relative to the FermiNet. On the largest system we looked at, the benzene dimer (84 electrons!) the Psiformer with VMC is more accurate than the FermiNet with *diffusion* Monte Carlo!
I really never thought I’d be an “attention is all you need” guy, but at least in this case, it seems like neural network ansatzes using self-attention are a clear improvement over prior models, and present a path to unprecedented accuracy in quantum chemical calculations.
This was work led by Ingrid von Glehn and in collaboration with @exp_n. For those at #NeurIPS2022, I’ll be speaking about this and other topics on deep learning and quantum chemistry at the #ML4PhysicalSciences workshop on Saturday!
https://twitter.com/pfau/status/1596493171714031616
God dammit, threading is broken. This website really is not doing well. I meant to include one more tweet showing that the Psiformer is more accurate than FermiNet on all the systems from the original FermiNet paper:
*ansatzes, not antazes. What's an antaze?
• • •
Missing some Tweet in this thread? You can try to
force a refresh
