mrt2-webui

Realtime Web UI for Magenta RT2

Stream AI-generated music over WebSocket —
steer the style with text, images, video, or 🖥️ your screen.

---

What is this?

Magenta RT2 is Google's realtime music generation model. This project wraps it in a browser-based UI so you can:

• 🎵 Stream live audio directly in your browser via WebSocket
• 💬 Guide the style using natural language prompts
• 🖼️ Drop in images or video to set the mood visually
• 🖥️ Share your screen and let the music react to what's on it

---

Try it on Colab

Want to try it without installing anything? Run the demo on a free Colab GPU:

The notebook installs everything, downloads mrt2_small, and exposes the web UI through a public URL. See examples/colab_demo.ipynb.

Note: Colab is best treated as a quick demo. Audio is streamed to your browser through a public tunnel, which adds latency and jitter, so realtime playback may stutter. For true realtime, run locally on your own GPU (see below).

---

Installation

1. Set up Magenta RT2

git clone --recurse-submodules https://github.com/magenta/magenta-realtime.git
cd magenta-realtime

# macOS (Apple Silicon)
pip install -e ".[mlx]"

# Linux (NVIDIA GPU) — add the CUDA 12 JAX wheel
pip install -e "." "jax[cuda12]"

export MAGENTA_HOME=/any_folder
mrt models init
mrt models download
mrt checkpoints download

2. Install mrt2-webui

git clone https://github.com/yhs3048/mrt2-webui.git
cd mrt2-webui
pip install -r requirements.txt

3. Folder structure

Both repositories must sit side by side in the same parent directory.

anywhere/
├── magenta-realtime/     # Magenta RT2 core
└── mrt2-webui/           # This repo

MAGENTA_HOME is where model weights are stored and can be set to any path independently.

---

Usage

python webui.py

Then open your browser and navigate to http://localhost:8000.

Options

Argument	Default	Description
--port	8000	Port to serve the web UI on
--clip-device	cuda	Device for the CLIP encoder — cpu, cuda, or a specific GPU like cuda:1

Note: Ensure port 8000 (or your chosen --port) is open and accessible.

---

Acknowledgements

Built on top of Magenta RT2 by the Google Magenta team.

Developed by yhs3048 at Music and Audio Research Group (MARG), Seoul National University.

---