NVIDIA Releases Nemotron 3.5 ASR: A 600M-Parameter Cache-Aware Streaming Model Transcribing 40 Language-Locales in Real Time
NVIDIA’s Nemotron Speech workforce has launched Nemotron 3.5 ASR. It is a 600M-parameter streaming Automatic Speech Recognition (ASR) mannequin. A single checkpoint transcribes 40 language-locales in actual time. Punctuation and capitalization are constructed in natively. The mannequin ships as open weights on Hugging Face. The license is OpenMDW-1.1. The structure is a Cache-Aware QuickConformer-RNNT.
What is Nemotron 3.5 ASR
Nemotron 3.5 ASR extends nvidia/nemotron-speech-streaming-en-0.6b to many languages. It provides prompt-based language-ID conditioning to the bottom mannequin. That lets one 600M-parameter checkpoint cowl 40 language-locales. No per-language mannequin or model-swapping is required.
The mannequin targets two workloads. The first is low-latency streaming for dwell audio. The second is high-throughput batch transcription. Output is production-ready textual content with correct casing and punctuation. No separate punctuation-restoration step is required.
(*40*)Image supply: https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b
How Cache-Aware QuickConformer-RNNT Works
The mannequin has two principal items. The first is a Cache-Aware QuickConformer encoder with 24 layers. QuickConformer is an environment friendly evolution of the Conformer structure. It makes use of linearly scalable consideration. The second piece is an RNNT (Recurrent Neural Network Transducer) decoder. RNNT emits textual content body by body as audio streams in.
The “cache-aware” design is the effectivity lever. Buffered streaming re-processes overlapping audio home windows at each step. That repeats the identical work and provides delay. This mannequin caches encoder self-attention and convolution activations as an alternative. It reuses these cached states as new audio arrives. So every audio body is processed precisely as soon as, with no overlap. Compute and end-to-end latency each drop, with out an accuracy penalty.
The Latency Knob: att_context_size
One inference setting controls the latency-accuracy tradeoff. It is the eye context measurement, att_context_size. Smaller context emits textual content sooner however sees much less future audio. Larger context raises accuracy at greater latency.
The similar checkpoint covers the total vary. Settings map to chunk sizes of 80ms, 160ms, 320ms, 560ms, and 1.12s. For instance, [56,0] provides an 80ms ultra-low-latency mode. The [56,13] setting provides 1.12s for highest accuracy. Teams decide the working level at inference time, with no retraining.
Language Detection and Coverage
The 40 language-locales embrace English, Spanish, German, and French variants. They additionally cowl Arabic, Japanese, Korean, Mandarin, Hindi, and Thai. Several different European and Nordic languages are included too.
Language conditioning works two methods. Setting target_lang to a identified locale normally provides the very best accuracy. Setting target_lang=auto lets the mannequin detect the language itself. In auto mode, it emits a language tag after terminal punctuation. One deployment can then transcribe mixed-language visitors. No separate language-ID part is required.
Comparison
| Product | Company | Access | Native streaming | Language protection | Reported latency | Pricing mannequin |
|---|---|---|---|---|---|---|
| Nemotron 3.5 ASR | NVIDIA | Open weights (OpenMDW-1.1), self-host; hosted on DeepInfra | Yes — cache-aware QuickConformer-RNNT | 40 language-locales | 80ms–1.12s, configurable at inference | Free to self-host; usage-based by way of host |
| Whisper large-v3 | OpenAI | Open weights (MIT), self-host; API | No — offline/batch | ~99 languages | Not streaming-native | Self-host free; API ~$0.006/min (batch) |
| Nova-3 | Deepgram | Closed API; on-premise/self-host (enterprise) | Yes — streaming + batch | Multilingual; +10 monolingual languages added Jan 2026 | Low-latency streaming (reported sub-300ms) | ~$0.0077/min (Nova-3 Monolingual, PAYG) |
| Universal-3 Pro Streaming | AssemblyAI | Closed API (EU endpoint out there) | Yes | 6 languages: English, Spanish, French, German, Italian, Portuguese | Sub-300ms (official); first partial ~750ms | Usage-based (PAYG) |
| Scribe v2 Realtime | ElevenLabs | Closed API | Yes | 90+ languages (99 per ElevenLabs) | ~150ms (p50) | ~$0.28/hour |
| Ursa / streaming | Speechmatics | API + on-premise + edge | Yes — streaming + batch | 50+ languages with computerized identification | Ultra-low latency (positioned) | Enterprise/utilization |
Fine-Tuning Results
Because the weights are open, groups can fine-tune for a language, area, or accent. NVIDIA printed a labored instance on Greek and Bulgarian. It fine-tuned the bottom checkpoint with the identical Cache-Aware QuickConformer-RNNT recipe. Each clip carried a target_lang tag for language conditioning. Training knowledge got here from public corpora, together with Granary, Common Voice, and FLEURS.
Results had been measured as WER on held-out FLEURS, on the 80ms setting. Greek WER fell from 35 to 24, a 32% relative enchancment. Bulgarian fell from 22 to fifteen, a 31% relative enchancment. These are uncooked WER percentages on the lowest-latency streaming mode. NVIDIA notes that evaluating at deployment latency, on held-out knowledge, provides sincere numbers.
Strengths and Considerations
Strengths:
- One 600M-parameter checkpoint covers 40 language-locales, slicing deployment sprawl.
- Cache-aware streaming processes every body as soon as, reported at 17x buffered concurrency on an H100.
att_context_sizetunes latency from 80ms to 1.12s at inference, with no retraining.- Punctuation, capitalization, and
autolanguage tagging are constructed in. - Open weights enabled a 31–32% relative WER drop on Greek and Bulgarian after fine-tuning.
Considerations:
- The mannequin handles English, however NVIDIA recommends its devoted English mannequin for English-only use.
- The 80ms mode trades some accuracy for the bottom latency.
- Japanese and Korean use CER, so cross-language error comparisons want care.
- Throughput figures are measured on H100, so outcomes on different GPUs will differ.
- The manufacturing NIM with gRPC streaming is introduced, however not but launched.
Key Takeaways
- NVIDIA’s Nemotron 3.5 ASR is an open-weights (OpenMDW-1.1), 600M-parameter streaming mannequin transcribing 40 language-locales from one checkpoint.
- Its Cache-Aware QuickConformer-RNNT design processes every audio body as soon as, reported at 17x the concurrent streams of buffered approaches on an H100.
- Latency is configurable from 80ms to 1.12s at inference by way of
att_context_size, with no retraining. - A quick fine-tune lower FLEURS WER 32% on Greek (35→24) and 31% on Bulgarian (22→15), on the 80ms setting.
- It is self-hostable and streaming-native, in contrast to closed APIs (Deepgram, AssemblyAI, ElevenLabs) or offline Whisper.
Marktechpost’s Visual Explainer
Check out the Model weights. Also, be at liberty to comply with us on Twitter and don’t neglect to affix our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to companion with us for selling your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar and so on.? Connect with us
The publish NVIDIA Releases Nemotron 3.5 ASR: A 600M-Parameter Cache-Aware Streaming Model Transcribing 40 Language-Locales in Real Time appeared first on MarkTechPost.
