Nemotron 3.5 Transcribe: who said what when everyone talks at once

A film scene makes the problem legible

Howard Hawks’s 1940 comedy is an apt demonstration because its dialogue makes turn-taking a poor assumption. A transcript can contain the right sentence and still misrepresent the scene if it assigns that sentence to the wrong person. Speaker diarization answers who spoke when; speaker-attributed transcription connects that answer to the actual words.

NVIDIA says the model can label up to eight speakers. These are identities within a recording, such as S01 and S02, rather than a claim that the system knows the actors’ names. The demonstration is worth watching for how it handles interruptions, not as proof that every caption is correct.

Sources: Piotr Żelasko / NVIDIA [1] · NVIDIA / Hugging Face [3] · American Film Institute [6]

An instruction changes the transcript

Transcribe treats the desired written form as part of the task. A verbatim transcript preserves fillers and repetitions; a readable one can omit them. A list of expected names or specialised terms can guide spelling. Translation produces text in another language. Audio question answering asks the model to use the recording to answer a question. These are different outputs from the same checkpoint.

That flexibility also makes the instruction an editorial choice. Removing a hesitation may improve reading, while erasing something relevant to an interview or performance. A translation and a summary should remain recognisable as transformations of the recording. Word timestamps give the reader or editor a route back to the sound.

Language coverage depends on the task. NVIDIA lists 40 languages for transcription, 31 for translation to or from English and 13 for speaker attribution. Ukrainian is listed for transcription and English-linked translation, but its multispeaker-ASR entry is not specified. Audio question answering has been evaluated in English. The headline language count therefore does not describe every feature.

Sources: NVIDIA [4] · NVIDIA / Hugging Face [3]

Two kinds of listening feed one language model

The architecture brings together an encoder for speech content and a speaker encoder based on NVIDIA’s Sortformer. Both provide information to a language model built on Nemotron 3.5 Lightning. Speaker information reaches the model alongside the acoustic content, allowing it to produce words and speaker labels together. A separate CTC alignment component supplies word timing.

The advertised 30B/4B shorthand refers to the language-model scale. The model card counts approximately 34 billion parameters in total and 4.3 billion active per token when the audio components are included. A mixture of experts activates only part of its weights for each token; it does not turn the complete checkpoint into a four-billion-parameter download.

Sources: NVIDIA [5]

The score changes with the question

NVIDIA’s launch post reports 5.33% average word error rate on the English Open ASR leaderboard, first among open-weight entries. Its model card and technical report publish a different figure, 4.71%, for a nine-dataset comparison captured on 12 October. The documents do not establish why these figures differ. They should be kept attached to their respective tables, rather than presented as interchangeable measurements.

WER counts substitutions, deletions and insertions relative to reference words. It does not by itself tell you whether the right person received each line. For that, the report also uses cpWER, which matches speaker labels before counting word errors. The recording setup matters: on AMI meetings, NVIDIA reports 10.29% cpWER with individual headset microphones and 17.34% with a single distant microphone. The model is the same; the listening conditions are not.

Sources: NVIDIA AI [2] · NVIDIA [4] · NVIDIA [5] · NVIDIA / Hugging Face [3]

Fast processing still starts with a recording

The technical report measures up to 2,238 seconds of audio processed per second on one RTX PRO 6000 Blackwell Server Edition GPU with 96 GB of memory. That is throughput, obtained with NVFP4, DFlash2 speculative decoding and NVIDIA’s NIM serving configuration. In the same study, a single 30-second excerpt returns its transcript in 0.13 seconds. Peak throughput and single-request latency describe different operating points.

Transcribe is an offline model: it processes recorded audio. These numbers do not establish how quickly a live assistant can hear an interruption and respond. The reference BF16 configuration specifies an H100 with 80 GB, and NVIDIA says the fastest NIM serving optimizations are not yet in the open-source release. Choose deployment hardware from the documented configuration, not from the active-parameter count.

Sources: NVIDIA [5] · NVIDIA / Hugging Face [3]

A transcript needs a way back to the voice

For subtitling, interviews and recorded conversations, the useful advance is the combination of attribution, timing and control over the written form. It can make a transcript easier to navigate and revise. It also gives a concrete way to assess the system: do the words survive an interruption, do speaker labels stay stable, and can a disputed line be checked against the recording?

Those checks matter more than a smooth paragraph. An elegant transcript that moves a sentence to the wrong speaker changes the conversation it was meant to preserve.

Sources: NVIDIA [4]