VAEs often require tuning a weight on the prior KL term. This is unnecessary, and simple methods for getting calibrated decoders can remove need to tune this! Some of these are known, but not widely used.
orybkin.github.io/sigma-vae/
arxiv.org/abs/2006.13202
w/ @_oleh & K. Daniilidis
orybkin.github.io/sigma-vae/
arxiv.org/abs/2006.13202
w/ @_oleh & K. Daniilidis
TL;DR is this: learn the variance of your Gaussian decoder. It will make the VAE better, and remove the need to tune KL-divergence coefficient ("beta"). There are a few choices for how to learn this variance, and the obvious choice may not be the best one.
