Thread by @BenjoCowley

My new work with Jonathan Pillow @jpillowtime! "High-contrast 'gaudy' images improve the training of deep neural network models of visual cortex."

We found that gaudy images can train DNNs with little data---perfect for neuro experiments!
arxiv.org/abs/2006.11412 #tweeprint

Our goal is to predict visual cortical responses from natural images. Often linear regression is used to map image features to responses b/c lack of experimental data. Here, we use a DNN (readout network) and avoid overfitting b/c our gaudy images are tailored for training!

The gaudy transformation is simple: Push pixel intensities to the extremes (either 0 or 255). We were inspired by 50-year-old active learning theory that says (under certain cases) the optimal training images are the ones that increase the variance in every pixel dimension.

We tested gaudy images with simulated visual cortical responses. Gaudy images improved training for different activation functions, # of layers, network architectures, and ground-truth networks. In particular, gaudy images were highly effective for DNNs with sigmoids.

Why are gaudy images so effective?

First, gaudy images drive responses to regions unexplored by responses to normal images.

Second, gaudy images overemphasize edges. We found that instead of transforming all pixels to gaudy, we could transform just 10% of all pixels (focusing on edges) and achieve similar performance gains.

So high-contrast edges are really important!

But Ben, you generate gaudy images *before* training. Surely choosing images *during* training will improve performance even more?

It turns out gaudy images improve performance more than active learning! This suggests gaudy images overemphasize features important for training.

This work shows gaudy images have passed the "sniff" test---they perform well in simulations.

Up next will be the "eye" test---presenting gaudy images while recording from visual cortical neurons. Stay tuned!