I am thrilled to announce our paper “Feedback Recurrent AutoEncoder” was accepted at #ICASSP2020! collaboration with Yang Yang, @TacoCohen and Jon Ryu. arxiv.org/abs/1911.04018. A quick thread.
In this paper we present a simple yet powerful idea: when using a recurrent AE to perform online lossy compression of a highly temporally correlated signal, one should feedback the state of the decoder to the encoder. We compare FRAE to many natural auto encoder designs. 
When describing each design, we argue that the ideal architecture should ensure that: (1) both encoder and decoder have recurrent connections; (2) there is feedback from decoder to encoder; (3) capable of utilizing long-term temporal correlation. FRAE satisfies all 3.
FRAE can be interpreted as a non-linear predictive coding scheme: the rec state h_t contains a summary of prev decoded frames; the encoder may take advantage of the existing info in h_t to form a code z_t+1 wich encodes only the residual info missing to reconstruct x_t+1 from h_t
We demonstrate FRAE effectiveness on spectrogram compression. We compared it to other proposed recurrent AE schemes. We also coupled FRAE with a powerful neural vocoder (WaveNet), and demonstrate both lower rate and better voice quality (P.OLQA) than Opus, an OS lowrate codec.
Finally, inspired from the Rate-Distortion AutoEncoder paper from @amir_habibian @tivaro @jmtomczak @TacoCohen, we propose a variational variant of FRAE dubbed FR-VAE, which pushed the bitrate further down through entropy coding of the latents using a prior model.
