Nemotron 3.5 Transcribe and Granary v2: promptable open ASR, and the data pipeline behind it
NVIDIA released an open-weight speech model that handles transcription, speaker labeling, translation and audio Q&A from prompts, along with the Granary v2 dataset and curation pipeline. The leaderboard gains are modest; the open, reusable data pipeline is the more useful part.
NVIDIA has released Nemotron 3.5 Transcribe and Granary v2: an open-weight speech model you steer with natural-language prompts, plus the multilingual dataset and curation pipeline used to train it. Both are on Hugging Face and ModelScope. It is a solid, incremental release, and I think the data pipeline matters more than the leaderboard numbers.
What shipped
The model. One checkpoint transcribes 40 languages, translates 31 of them to or from English, labels speakers in 13, and answers questions about the audio (English only). It takes recordings up to an hour long, labels up to eight speakers, and gives word-level timestamps. It is built for recorded audio, not streaming. A speech encoder feeds a hybrid Mamba–Transformer mixture-of-experts language model with 30B total parameters, about 4B active per token. Sortformer speaker cues enter the language model alongside the acoustic features, so speaker turns are tagged during generation rather than attached afterwards by a separate diarization step.
The data. The Granary v2 public manifest covers 463,062 hours across 48 languages, up from 25 in Granary v1. Each recording gets three transcript forms: verbatim normalized ("twenty-five euros"), inverse-normalized with disfluencies ("uh, €25"), and inverse-normalized clean ("I paid €25"), plus audio captions, speech-grounded Q&A, biasing terms and scored translations. The NeMo Curator pipeline runs audio preparation, ASR with a hallucination-gated fallback, label generation with agreement checks, and a high-fidelity filter that cut VoxPopuli from 354k hours to 111k.
The numbers, read carefully
- Open ASR Leaderboard (English): 5.33% average WER, first among open-weight models. API models are excluded from that ranking.
- Artificial Analysis: 2.8% AA-WER, tied for first among open-weight models, with about 4B active parameters against 24B for the model it ties.
- Diarization: 18.26% DER across 21 languages on David AI's leaderboard, just ahead of NVIDIA's own Nemotron 3 Diarization at 18.38%.
- Throughput: 100 hours of audio in under three minutes on one RTX PRO 6000 Blackwell GPU, using the not-yet-shipped NVFP4 version on 30-second clips at high concurrency.
The rankings are real, but the margins are small and the qualifiers matter. The throughput figure depends on a checkpoint you can't download today, and a 0.12-point DER lead over a sibling model changes nothing about deployment. The result I'd watch is accuracy per active parameter: matching a 24B-active model with about 4B active, thanks to the sparse MoE. Whether that turns into lower serving cost depends on memory for all 30B weights, precision, batching and hardware, so measure throughput and cost on your own stack before assuming it.
Why it matters in production
Some ASR problems I see in production are about output form rather than recognition: "twenty-five euros" when the billing system needs "€25". Others are recognition or attribution errors, such as a misspelled drug name or merged speaker turns in a clinical consult. Teams patch both with regex post-processing and separate diarization services that break independently. This release suggests an order to try fixes in:
- Prompt for the output form you need: verbatim or clean, translated, speaker-labeled.
- Add context terms at inference, for example "Possible terms: HbA1c, metformin. Use these terms only when supported by the audio."
- Fine-tune with NeMo Speech on data shaped by the Granary v2 pipeline, if the first two aren't enough.
In NVIDIA's medical case study, fine-tuning cut WER on held-out speakers by 45.8% relative to the best prompt. That is the right baseline, because it measures what training adds beyond the cheaper fixes. And because the pipeline that produced Granary v2's targets is open, you can build the same verbatim and clean variants, hallucination fallback and agreement filter for your own audio.
My take
Treat this as a strong open default for batch transcription of recorded multi-speaker and multilingual audio, and as a reference design for curating speech data. It doesn't stream, the quantized and on-device builds are still "coming soon," and support varies by task and language, so check the model card's task-by-language matrix first. As NVIDIA advises, spot-check names, speaker changes and audio answers against the recording. I'd pilot it on one real workload and measure how much post-processing code the prompt replaces. Download the model for that. Keep the pipeline for longer.