Thread by @MoRezaK

Excited to present at #NeurIPS2019 this week!! I identify challenges in hyperparameter search when training sequential autoencoders (SAEs) on sparse data such as neural spiking, and present new methods to address them.

Wed 10:45a, East B+C #138 👨‍💼
Paper: tinyurl.com/lfads-hp-opt

We use SAEs to model dynamics underlying neural population activity. With small datasets, hyperparameters (HPs) can be critical. But SAEs are prone to a type of overfitting that is challenging to detect, making it difficult to adjust HPs automatically.

We developed coordinated dropout (CD). CD forces a network to only model shared structure underlying the data. CD first passes in a subset of samples at the input. To update network weights, CD only uses gradients from reconstruction of the complementary subset of samples.

CD allowed us to use HP search to train SAEs. We then modeled neural activity (motor cortex) from a monkey making reaching movements, and saw a substantial improvement in decoding movements, maintaining high performance even after a 5-10x reduction in data size (black).

In sum, our techniques enabled HP search on SAEs to model neural spiking activity, leading to substantially better performance on small datasets.

Come by Wednesday to learn more!
Poster: snel-repo.github.io/NeurIPS2019/po…
Wed 10:45-12:45, East Hall B+C #138