,
9 tweets,
6 min read
Read on Twitter
Remember the #BachDoodle? We’re excited to release paper on Behind-the-Scenes design, #ML, scaling it up, and dataset of 21.6M melodies from around the world!
📜 arxiv.org/abs/1907.06637
w/ @fjord41 @ada_rob @notwaldorf @bengiswex Leon Hong @jaxcooo
📜 arxiv.org/abs/1907.06637
w/ @fjord41 @ada_rob @notwaldorf @bengiswex Leon Hong @jaxcooo
tl; dr
1/ In three days, people spent 350 years worth of time playing with the Bach Doodle, and the “harmonize” button was clicked more than 55 million times.
1/ In three days, people spent 350 years worth of time playing with the Bach Doodle, and the “harmonize” button was clicked more than 55 million times.
2/ The model Coconet 🥥 is an instance of OrderlessNADE and uses Gibbs sampling to generate the harmonizations through rewriting.
📜 Previous blogpost: g.co/magenta/coconet
📝 Paper from #ISMIR 2017: arxiv.org/abs/1903.07227
📜 Previous blogpost: g.co/magenta/coconet
📝 Paper from #ISMIR 2017: arxiv.org/abs/1903.07227
3/ We sped up Coconet from 40s to 2s in #TensorFlowJS by using dilated depth-wise separable convolutions, which requires less layers and are more accelerated then conventional convolutions. Also by fusing ops that are used in every Gibbs steps (@GreenBeanDou).
4/ We reduced the model download size to approximately 400KB through post-training weight quantization.
5/ We calibrated a speed test based on partial model evaluation time to determine if the harmonization request should be performed locally using #TensorFlowJS or sent to remote #TPU servers. @jaxcooo was able to get a nearly 50/50 split!
6/ As the model was trained on Bach Chorales, melodies outside of the soprano range is “out of distribution” and harder to harmonize.
See how Paul Davids discovers this at 6:22 via @YouTube
See how Paul Davids discovers this at 6:22 via @YouTube
7/ We found that parallel fifths and octaves were more common when user input was out of distribution, and fewer parallel fifths and octaves were correlated with positive user feedback.
8/ Checkout @notwaldorf’s amazing interactive visualizations to explore the 21.6M melodies, the dataset and also hear new harmonizations.
