Our paper "Function-Space Learning Rates" is on arXiv! We give an efficient way to estimate the magnitude of changes to NN outputs caused by a particular weight update. We analyse optimiser dynamics in function space, and enable hyperparameter transfer with our scheme FLeRM! 🧵👇 NN training learns input-output functions. But we typically understand optimisers as acting in parameter space. For example, traditional learning rates tell us how much parameters change each step, rather than the effect on the learned function.