NVIDIA Nemotron 3.5 Lightning: New Open-Source Agent Model, Benchmarks & How to Run It

What You'll Learn

• What Nemotron 3.5 Lightning is and why NVIDIA designed it for AI agents
• Official benchmark scores and how it compares with GPT-OSS-120b and Qwen3.6 35B
• System requirements, from a single H100 up to RTX 5090 and DGX Spark
• How to download, serve and deploy it with vLLM, Ollama and NVIDIA NIM

NVIDIA Nemotron 3.5 Lightning is the first model in the new Nemotron 3.5 lineup, released on August 11, 2026, and announced alongside the open-source NeMo Switchyard routing tool. It is an open 30B mixture-of-experts (MoE) model with 3B active parameters, built on a hybrid Mamba-2 plus Transformer MoE architecture with interleaved Mamba-2 and MoE layers and select attention layers. The weights, post-training data and recipes are public, and the release is licensed under OpenMDW-1.1, which permits commercial use, distillation and redistribution without attribution. The model targets the execution layer of long-running AI agents: high-volume tool calls, result validation and subagent delegation, where speed and token cost matter more than peak reasoning.

What Is NVIDIA Nemotron 3.5 Lightning?

Nemotron 3.5 Lightning is NVIDIA's answer to the cost problem in agentic AI. Long-running agents generate enormous token volumes because every tool call and subagent message passes through the model. NVIDIA designed Lightning as a small, efficient worker model that succeeds the Nemotron 3 Nano Omni line and is distilled from the 550B Nemotron 3 Ultra, which NVIDIA released on June 4, 2026, for long-running reasoning agents. By activating only 3B of its 30B parameters per token, the model cuts inference compute dramatically while keeping a 1M-token context window, so a single agent can hold an entire session history.

The announcement carried a clear business message. NVIDIA says Nemotron 3.5 Lightning can help enterprises save on AI token costs, and the model is available day-zero on NVIDIA's own Build API as well as DeepInfra, Baseten, OpenRouter and FriendliAI. Community GGUF quantizations from bartowski and LM Studio Community appeared within hours, so developers can run it through local tools such as Ollama and LM Studio the same day.

Key Specifications of Nemotron 3.5 Lightning

The hybrid architecture is the standout engineering choice. Mamba-2 layers keep the state-space core efficient for long sequences, MoE layers add capacity without adding active compute, and the few attention layers handle retrieval-style context mixing. NVIDIA's model card lists Hopper-class hardware from 1x H100 up to 8x H100 for datacenter deployments, while the developer blog confirms local systems including NVIDIA Jetson, GeForce RTX 5090 and DGX Spark can run it. In community testing on DGX Spark with Ollama, the model loaded in a 26GB footprint with a 262,144-token default context window.

Nemotron 3.5 Lightning Benchmarks

NVIDIA published official evaluation numbers on the NIM model card comparing the BF16 and NVFP4 checkpoints. They are strong for a 3B-active model, with MMLU Pro at 81.94, GPQA Diamond at 75.44 and SWE-bench Verified at 51.56 for the BF16 release.

Independent testing agrees. Artificial Analysis gives Nemotron 3.5 Lightning a score of 24 on its Intelligence Index, a nine-point jump over Nemotron 3 Nano (15) and level with GPT-OSS-120b. The same evaluation measured roughly 670 tokens per second on the final NVFP4 weights, the highest throughput among all compared open models and nearly twice the 386 tokens per second of Gemini 3.5 Flash-Lite. NVIDIA also reports 86% accuracy on PinchBench while finishing 10,000 agent tasks 30% faster than Qwen3.6 35B at similar accuracy.

Nemotron 3.5 Lightning vs GPT-OSS-120b vs Qwen3.6 35B

Lightning is deliberately positioned between raw intelligence and operating cost. It matches GPT-OSS-120b on the Artificial Analysis Intelligence Index while using roughly a quarter of the total parameters, and DeepInfra measures output speed at about four times that of comparable open models in the 25B-35B class. Against Qwen3.6 35B, the tradeoff is clearer: Qwen wins on raw benchmark scores such as MMLU Pro 85.63 and GPQA Diamond 83.40, but Lightning completes the same agent workloads much faster and cheaper per token.

For teams running always-on agents, the practical comparison is total cost per completed task, not a single benchmark. AI agent systems in the same class as OpenHands and other open coding agents are already adopting Lightning as the high-volume worker because tool-call accuracy stays high while token spend drops. If you are comparing foundation models for a heavy agent workload, review our roundup of the big AI model launches of the past fortnight for context on where Lightning fits in the wider race.

System Requirements: How to Run Nemotron 3.5 Lightning

One H100 is enough for production serving, and consumer-class hardware works for local experimentation. NVIDIA lists Hopper 1x-8x H100 for datacenter deployments in the model card, and the developer blog states the model runs locally on NVIDIA Jetson, GeForce RTX 5090 and DGX Spark. The NVFP4 checkpoint is the efficient choice for a single GPU because 4-bit weights cut VRAM use sharply versus the BF16 release, and the NVFP4-DSpark variant is tuned specifically for DGX Spark systems.

Community results give practical numbers. On a DGX Spark with Ollama, the model used a 26GB resident footprint at full GPU utilization with a default 262,144-token context, leaving roughly 86GB of the unified memory pool free. On an 8x H100 node, NVIDIA and partners report efficient multi-GPU serving with vLLM. On a 24GB consumer card such as an RTX 5090 you will make tradeoffs on context length and batch size, but the model is designed to fit, which is the whole point of the 3B-active design. If you need to push even larger models onto smaller GPUs, our TurboQuant guide to running 3x larger AI models on cheap hardware explains the quantization tricks that make it possible. Serving follows the standard vLLM workflow covered in our vLLM August 2026 guide, with day-zero support expected across the major serving stacks.

How to Download and Deploy Nemotron 3.5 Lightning

There are three fast paths to production. First, the hosted route: NVIDIA Build (NIM) offers a serverless API at build.nvidia.com, and DeepInfra, Baseten, OpenRouter and FriendliAI all added day-zero access on August 11, 2026, with OpenRouter even listing a free tier. Second, the self-hosted route: pull the official checkpoints from Hugging Face, where the NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 and NVFP4 models are published under the NVIDIA organization, and serve them with vLLM or the NVIDIA NeMo framework. Third, the local route: grab a community GGUF from bartowski or LM Studio Community and load it in Ollama or LM Studio on an RTX 5090, a DGX Spark or even a Jetson device.

For agent builders, NVIDIA pairs the model with NeMo Switchyard, an open-source routing tool announced on the same day. Switchyard routes different parts of an agent workload across models, so a planner can run on a larger reasoning model while Lightning handles the high-volume execution layer. NVIDIA's own example splits tool calls, result validation and subagent delegation between models in one workflow. MindStudio and Kubesimplify already published step-by-step local setup guides, and the NVIDIA developer blog and official model card remain the authoritative configuration references.

NeMo Switchyard and the Road Ahead

NeMo Switchyard is the second half of the August 11, 2026 announcement. It is an open routing layer that lets developers direct AI agent workloads across multiple models, switching between a commander model and fast worker models as the task changes. This matters because agent economics are dominated by the execution layer, and routing is how enterprises cut cost without cutting quality. MSI announced support for both Lightning and Switchyard on its XpertStation WS300 and EdgeXpert platforms the same day, and Fastino Labs released finance and healthcare models post-trained on Lightning entirely using an agent, evidence that the fine-tuning ecosystem is already active.

Looking further ahead, Reuters reported on August 11, 2026, that NVIDIA is developing Nemotron 4, a trillion-parameter open model family intended to rival the strongest open-weight systems. Lightning is the efficient worker in that strategy, and Nemotron 4 would be the frontier-scale flagship. For now, the 3.5 lineup starts with Lightning and the model card confirms more checkpoints in the family, so teams should expect a fast release cadence.

Conclusion

Nemotron 3.5 Lightning is the strongest open-weight efficiency play NVIDIA has shipped in the 30B class. It matches GPT-OSS-120b on Artificial Analysis's Intelligence Index at roughly a quarter of the total parameters, delivers near-670 tokens per second on NVFP4 weights, and runs from a Jetson-class device up to an 8x H100 node, all under a permissive OpenMDW-1.1 license. Independent evaluation, including Artificial Analysis's benchmark team, treats the model as a specialized worker rather than a general reasoning flagship, and that is exactly the niche NVIDIA is targeting: the high-volume execution layer of always-on AI agents.

The verdict for developers is simple. If you serve long-running agents and token cost is your bottleneck, download the NVFP4 checkpoint and benchmark it against your current worker model this week, because the zero-competition window on tooling around this release is closing fast. With NeMo Switchyard for routing, day-zero API support and community GGUF builds already live, the fastest path from download to production is shorter than for any previous Nemotron release.