Here is a thread in paper references demonstrating that current deep learning, especially transformers are amazingly powerful for symbol manipulation already:

@GuillaumeLample, @f_charton
arxiv.org/abs/1912.01412
Solving hard integrals using deep transformers

🧵1/n

https://twitter.com/Plinz/status/1502337713315274760

A paper of my team at google, improving reasoning in large theories using contrastive learning for premise selection:

arxiv.org/abs/1606.04442