NVIDIA AI Releases Nemotron 3.5 Lightning: A 30B Open MoE with 3B Active Parameters, and NeMo Switchyard Model Router
NVIDIA launched open applied sciences for constructing always-on AI brokers from methods of specialised fashions. Two artifacts shipped collectively. Nemotron 3.5 Lightning is a light-weight, customizable open mannequin constructed for high-volume agentic duties, and NeMo Switchyard is an open supply routing library that directs every step of an agent workflow to probably the most succesful and environment friendly mannequin out there. The downside each handle is structural: long-running brokers spend most of their time on software calls, end result validation, and subagent delegation, and sending each a kind of steps to a frontier reasoning mannequin provides price and latency. Lightning is a 30B mixture-of-experts mannequin with 3B energetic parameters, constructed on a hybrid Mamba-2 + MoE + Attention structure with a 1M-token context window. NVIDIA stories as much as 4x sooner output pace than similar-sized fashions, and 30% sooner completion of 10,000 PinchBench duties than Qwen3.6 35B at comparable accuracy. Many trade gamers like CrowdStrike, Harvey, CodeRabbit, Fastino Labs, and Lila Sciences are already customizing it for cybersecurity, authorized, coding, finance, and healthcare workloads.
Is it deployable?
Yes. Nemotron 3.5 Lightning is mostly out there below the permissive OpenMDW-1.1 license, with open weights, coaching information, and recipes. NVIDIA states the mannequin is prepared for industrial use.
- Which corporations: Anyone with a single trendy GPU. NVIDIA lists single-GPU deployment on 1x DGX Spark (GB10) or 1x H100. That places solo builders and seed-stage startups on the identical footing as enterprises. Mid-market groups can serve it from Baseten, Together AI, or Nebius; regulated enterprises can preserve it totally on-premises.
- Industries: Cybersecurity, authorized providers, software program engineering, monetary providers, healthcare, and life sciences all seem in NVIDIA’s named buyer set.
- Applications: Tool calling, end result validation, subagent delegation, code overview routing, log triage, contract parsing, and long-context retrieval throughout a 1M-token window.
The execution layer, not the planning layer
Long-running brokers spend most of their time on high-volume execution. Tool calls, end result validation, and subagent delegation dominate the token finances. Routing each a kind of steps to a frontier reasoning mannequin provides price and latency.
Nemotron 3.5 Lightning targets that execution layer. It is a 30B mixture-of-experts mannequin with 3B energetic parameters, constructed on a hybrid Mamba-2 + MoE + Attention structure. Context size reaches 1M tokens. Pre-training coated greater than 20 trillion tokens utilizing an NVFP4 recipe.
The mannequin is the smallest member of the Nemotron 3 household. Frontier fashions equivalent to Nemotron 3 Ultra deal with orchestration and planning, whereas Lightning handles the routine calls beneath them.
Where the pace comes from
Two mechanisms:
- First, Speculative Decoding: Multi-token prediction was baked in throughout a devoted pre-training stage, then improved with an MTP-boosting section. NVIDIA additionally ships two exterior draft fashions: DSpark, a semi-autoregressive drafter really helpful for DGX Spark and low-concurrency information middle workloads, and DFlash, which makes use of a light-weight block-diffusion mannequin.
- Second, Quantization: An NVFP4 checkpoint ships alongside BF16. The similar checkpoint serves Blackwell and Hopper natively, and extends to Ampere via W4A16 kernels.
NVIDIA stories as much as 4x output pace versus similar-sized fashions. On PinchBench, it stories 86% accuracy whereas finishing 10,000 duties 30% sooner than Qwen3.6 35B at comparable accuracy.
Published mannequin card outcomes (BF16 / NVFP4): MMLU Pro 81.94 / 81.62, GPQA Diamond 75.44 / 75.57, SWE-bench Verified 51.56 / 52.80, Terminal-Bench 2.1 24.58 / 23.46, AA-LCR 52.00 / 49.19. Recommended sampling is temperature 1.0 and top_p 0.95.
NeMo Switchyard
NeMo Switchyard is an open supply library that routes every step of an agent workflow to probably the most succesful and environment friendly mannequin out there.
It presents tuning-free routers, together with an LLM classifier with session affinity, a stage router that reads latest software exercise, and an escalation router that begins low cost and promotes on sustained issue. A tunable prefill router learns from the mannequin’s residual stream to foretell which candidate will succeed. The reference server accepts OpenAI, Anthropic, and Responses API requests.
Two printed outcomes: LangChain benchmarked 145 multi-turn agentic duties. Routing between Lightning and Claude Opus 4.8 with the escalation router lower price 74% versus a frontier-only baseline, sending 7% of calls to the frontier mannequin, at a roughly 6-point accuracy tradeoff. Cognition applied staged routing in Devin Desktop. On FrontierCode Main, routing between Opus 5 and Kimi K2.7 reached 50.6% at a $3.11 imply price, inside 2.8 factors of Opus 5 accuracy at roughly 28% decrease imply price.