Magenta RealTime 2: Open Live Music Model Guide

One-sentence definition

Magenta RealTime 2 is an open-weights live music model that generates 48kHz stereo audio while responding to MIDI, text, and audio controls fast enough to be played like an instrument.

Why MRT2 matters

Most AI music demos still feel like batch jobs. You type a prompt, wait, and judge the rendered clip. MRT2 aims at a different interface: press keys, move a control, blend a prompt, and hear the model react while the session is still alive.

Google Magenta's technical post says the first Magenta RealTime model worked in chunks, which made control feel delayed. MRT2 moves to frame-level streaming, runs through an MLX-backed C++ engine on Apple Silicon, and ships with apps and Audio Unit plugins instead of only notebooks.

Launch posts

The user-facing message is consistent across Google Gemma and Google Magenta: MRT2 is not only an open audio model. It is a playable instrument surface for live control. The official Magenta thread is more detailed; the Gemma thread is the broader ecosystem announcement.

1. Official launch post

   Google Magenta introduces MRT2 as a live music model with open weights, an open source inference engine, apps, plugins, MIDI control, and low-latency MacBook playback.

   Google Magenta introduces MRT2 as a live music model with open weights, an open source inference engine, apps, plugins, MIDI control, and low-latency MacBook playback.

   — Google Magenta Project (@GoogleMagenta) June 4, 2026

2. Official launch post

   Google Gemma frames MRT2 as an open model musicians can play as an instrument using MIDI, text, and audio on a MacBook.

   Google Gemma frames MRT2 as an open model musicians can play as an instrument using MIDI, text, and audio on a MacBook.

   — Google Gemma (@googlegemma) June 4, 2026

3. Official launch post

   Google Gemma points readers to the MRT2 app and plugin download page.

   Google Gemma points readers to the MRT2 app and plugin download page.

   — Google Gemma (@googlegemma) June 4, 2026

4. Official launch post

   Google Gemma sends musicians and developers to the Google Magenta project for more experiments.

   Google Gemma sends musicians and developers to the Google Magenta project for more experiments.

   — Google Gemma (@googlegemma) June 4, 2026

Mental model: five moving parts

MRT2 is easiest to understand as a stack. The public demos sit at the top, but the useful developer surface is the line between model, inference engine, and controls.

MIDI notes / text prompt / audio prompt
        |
        v
MusicCoCa style embedding + note/drum conditioning
        |
        v
Depthformer generates SpectroStream audio tokens
        |
        v
MLX-backed C++ streaming engine on Apple Silicon
        |
        v
Standalone app / AU plugin / custom instrument

MusicCoCa maps text and audio prompts into style embeddings. SpectroStream is the codec that turns audio into tokens and back. Depthformer is the transformer that generates the token stream. The C++ engine is what makes the real-time path practical on MacBook GPUs.

Smallest local run

For musicians, the shortest path is the Mac app bundle from the MRT2 apps page. For developers, the shortest reproducible path is the Python package and CLI. The docs use `uv`, Python 3.12, and the `magenta-rt` package.

curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv --python 3.12
source .venv/bin/activate

# Apple Silicon live path
uv pip install "magenta-rt[mlx]"

# Shared resources, then one streaming model
mrt models init
mrt models download

# Generate a four-second check clip
mrt mlx generate --prompt "disco funk" --duration 4.0 --model=mrt2_small

The GitHub installation docs say MLX is the live Apple Silicon path. JAX is still available for offline, batch, or research work, including Linux setups with the appropriate JAX wheel.

Apps and plugins

The MRT2 download page matters because it shows Google is not shipping only a model checkpoint. The bundle includes standalone apps, Audio Unit plugin support, and examples that demonstrate different interfaces for the same model.

The Audio Unit plugin angle is the most practical part for DAW users. Google's install page tells users to register the plugin, refresh AU plugins in the DAW, and drag the plugin onto a MIDI track. That means the first serious tests will happen in normal music sessions, not only in research demos.

The controls are the product

MRT2 uses multiple control signals. Text and audio prompts steer style. MIDI notes steer pitch and harmony. Drum control can be used to suppress drums when the model needs to sit beside other tracks.

The technical post says note control is trained from audio and MIDI pairs, with MIDI labels inferred by MT3. It also describes two note modes: one where the model chooses attacks from held notes, and one where the user supplies onset timing. That is the difference between a model that follows a chord and a model that follows performance timing.

Latency: the real product test

Google's own post says MRT2 moved from a 2-second frame in the first version to a 40ms frame in MRT2, with average control latency under 200ms once buffers and system overhead are counted. That is the central claim.

The engineering reason is frame-level autoregression. Instead of waiting for a large chunk boundary before new control information matters, MRT2 injects conditioning at each generation step. It also uses decoder-only streaming with sliding window attention so the model can continue over time without unbounded memory.

Hardware and model choice

Start with the small model. The official docs say `mrt2_small` runs real-time on any Apple Silicon Mac, including Air models. The base model is larger and higher quality, but it has a much tighter timing budget.

The app page also says first launch may download model weights: roughly 450MB for small and 2.5GB for base. Treat those numbers as install-size planning, not runtime memory guarantees.

Architecture notes for developers

MRT2 is a codec language model. A codec language model generates discrete audio tokens, then decodes those tokens back to audio. Google's technical appendix says SpectroStream compresses 48kHz stereo audio into token frames at 25Hz, with 12 tokens per frame.

The key design choice is not only tokenization. MRT2 uses a decoder-only architecture, local sliding window attention, and attention sink embeddings to keep continuous generation stable after old context is evicted. The team also dropped explicit positional embeddings for this path, relying on causal masking and sliding attention for length generalization.

Developer paths

There are three serious developer paths. Use the app bundle if you are evaluating musical feel. Use the Python package if you are testing prompts, tokens, and offline generation. Use the C++ engine if you are building an instrument or plugin where audio callback behavior matters.

# C++ app development path from the docs
uv pip install "cmake<3.28"

cmake . -B build
cmake --build build --target hello_mrt2 -j10

./build/examples/hello_mrt2/hello_mrt2 \
  ~/Documents/Magenta/magenta-rt-v2/models/mrt2_small/mrt2_small.mlxfn \
  ~/Documents/Magenta/magenta-rt-v2/resources \
  100 \
  --prompt "ambient pads with sub bass"

For MCP.Directory readers, the interesting next step is not an MCP server yet. It is a local agent workflow around the model: generate prompt sets, test presets, catalog useful controller mappings, and create repeatable DAW session templates.

What we got wrong at first

The first read makes MRT2 sound like another open model release. That misses the point. The model only becomes different when the control loop is short enough for a musician to react to it.

We also initially treated the MacBook requirement as a simple limitation. It is a limitation, but it is also an architecture choice. Google started with Apple Silicon because MLX gives the C++ engine a predictable local GPU target and many musicians already use MacBooks. That does not remove the need for Windows and Linux paths, but it explains the launch shape.

Caveats and early friction

The launch reaction is positive, but the caveats are concrete. X replies ask for Windows, Linux, API access, and broader DAW clarity. One reply to Google Magenta asked directly about Windows; the project account answered that a broader release would be useful, but they started with Apple Silicon because the live path needs a moderately powerful GPU and MacBooks are common among musicians.

Three workflows to try

The best first workflows are small and measurable. Do not start with an album. Start with one repeatable controller setup.

Verdict

MRT2 is the most interesting when you judge it as a live instrument runtime. The open weights matter, but the bigger story is the control surface: MIDI, text, audio, MLX, C++, apps, and plugins all aimed at the same question: can a model be played?

The cautious answer is yes, for the right Mac and the right model size. The practical recommendation is simple: install the bundle, run the small model first, set audio to 48kHz, measure latency in your real DAW session, and only then decide whether the base model belongs in a live workflow.

FAQ

Glossary

Sources

This post uses first-party launch material, official Google Magenta docs, the GitHub repository, GitHub issues and PRs, Reddit discussion, and X reactions. Claims about hardware, install flow, features, and architecture are tied to the sources below.

• Google Gemma MRT2 launch thread on X
• Google Magenta MRT2 launch thread on X
• Magenta RealTime 2 apps and plugins page
• Magenta RealTime 2 technical blog
• magenta/magenta-realtime GitHub repository
• Magenta RealTime installation docs
• Magenta RealTime models and hardware docs
• GitHub issue #39: M3 Max benchmark caveat
• GitHub PR #40: audio callback safety and benchmark context
• Reddit r/aicuriosity MRT2 discussion
• Live Music Models technical report