Excited to announce 🎵Magenta RealTime, the first open weights music generation model capable of real-time audio generation with real-time control.

👋 **Try Magenta RT on Colab TPUs**: colab.research.google.com/github/magenta…

👀 Blog post: g.co/magenta/rt

🧵 below Magenta RT is capable of streaming generation of music audio (RTF >x1) steered by one or more style prompts (either text or audio). It is the open weights cousin of MusicFX DJ Mode and the Lyria RealTime API.

🤗 Model: huggingface.co/google/magenta…
⭐ Code: github.com/magenta/magent…

Excited to share SingSong, a system which can generate instrumental accompaniments to pair with input vocals!

📄arxiv.org/abs/2301.12662
🔊g.co/magenta/singso…

Work co-led by myself, @antoine_caillon, and @ada_rob as part of @GoogleMagenta and the broader MusicLM project 🧵 Singing is among the most intuitive ways we engage with music. We already sing along with *existing* music, but singing may also be useful as a control mechanism for music generation, allowing anyone who can sing to create *new* music with rich instrumentation.

Excited to share recent work on music transcription w/ @jwthickstun and @percyliang, appearing today at #ISMIR2022! We release Sheet Sage, a system which can transcribe music as lead sheets 🧵

📄 arxiv.org/abs/2212.01884
⭐ github.com/chrisdonahue/s…
🔊 chrisdonahue.com/sheetsage Lead sheets are scores which depict melody as notes and harmony as chord names, and are often used by musicians to perform new renditions of existing songs. Sheet Sage can convert music audio into this format w/ little setup:

> ./sheetsage.sh <ANY_URL>

⭐github.com/chrisdonahue/s…

There are 🔥 music representations lurking in Jukebox, a language model of music audio. A preview of what's to come for MIR?

Work w/ (undergrad!) Rodrigo Castellon and @percyliang. Freshly accepted @ismir2021

📜 arxiv.org/abs/2107.05677
⭐ github.com/p-lambda/jukem…

🧵[1/8] @percyliang @ismir2021 Core result: across four MIR tasks (music tagging, genre classification, key detection, emotion recognition), probing representations from Jukebox yields 30% stronger performance on average relative to probing representations from conventional pre-trained models for MIR. [2/8]