NVIDIA Nemotron 3 Super (120B-A12B)

2026-03-11 · Publisher : NVIDIA · Domain : Artificial Intelligence

NVIDIA releases Nemotron 3 Super, an open model with 120 billion parameters, 12 billion of them active, a hybrid Mamba-Transformer LatentMoE designed for agentic reasoning with a one-million-token context.

### Context Three months after Nemotron 3 Nano, Super is the announced tier for multi-agent systems. NVIDIA presents it as delivering more than 5 times the throughput of the previous Super generation. NVIDIA compares it to Qwen3.5-122B and GPT-OSS-120B, two open models. ### What the model brings - LatentMoE: tokens are compressed before reaching the experts, which allows, according to NVIDIA, 4 times more experts for the same inference cost. - Multi-token prediction (MTP): up to 3 times real-time speedup on structured generations, according to the technical blog. - Native pretraining in NVFP4 (4-bit) optimized for Blackwell. - Reinforcement learning across 21 environment configurations, with more than 1.2 million trajectories. - Release of the datasets (10 trillion curated unique pretraining tokens, 15 million coding problems, about 40 million post-training samples) and of the full pipeline on GitHub. ### Architecture and specifications 120 billion parameters (12 billion active), Mamba-2 architecture, MoE and attention with MTP, context up to 1 million tokens. Pretraining on more than 25 trillion tokens (knowledge cutoff June 2025), post-training through February 2026. Languages: English, French, German, Italian, Japanese, Spanish, Chinese. License: NVIDIA Nemotron Open Model License. Openness: weights (base, BF16, FP8, NVFP4), a large part of the data and the recipes (LoRA SFT, GRPO, DAPO, vLLM, SGLang and TensorRT-LLM cookbooks); about 10 of the 37 RL datasets are published according to the blog, so the data is only partially open. ### Cost and access | Item | Detail | |---|---| | Official API | Available on build.nvidia.com, Perplexity, OpenRouter and several hosting providers; NVIDIA pricing per million tokens not disclosed | | Minimum hardware | 8 x H100 80 GB (HF model card); 2 x B200 or B300 for the BF16 version | | Weights | Base, BF16, FP8 and NVFP4 on March 10, 2026 (Hugging Face) | ### Benchmarks | Benchmark (published by NVIDIA) | Nemotron 3 Super | Qwen3.5-122B-A10B | GPT-OSS-120B | |---|---|---|---| | MMLU-Pro | 83.73 | 86.70 | 81.00 | | AIME25 | 90.21 | 90.36 | 92.50 | | GPQA | 79.23 | 86.60 | 80.10 | | LiveCodeBench | 81.19 | 78.93 | 88.00 | | SWE-Bench (OpenHands) | 60.47 | 66.40 | 41.9 | | Arena-Hard-V2 | 73.88 | 75.15 | 90.26 | Source: Hugging Face model card nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16, NVIDIA measurements. Both comparators are open models. NVIDIA also cites 85.6% on PinchBench (OpenClaw agents), without detailing the comparisons in the excerpt reviewed. ### Versus closed models The vendor's table includes no closed model, and no comparison with a closed model was found in the sources reviewed. Against Qwen3.5-122B, Super trails on MMLU-Pro (83.73 versus 86.70), GPQA (79.23 versus 86.60) and SWE-Bench (60.47 versus 66.40), but leads on LiveCodeBench (81.19 versus 78.93). Against GPT-OSS-120B, it leads on SWE-Bench (60.47 versus 41.9) and trails on Arena-Hard-V2 (73.88 versus 90.26). The one-million-token context is a distinctive axis: on RULER at 1 million, the model card gives 91.75 for Super versus 91.33 for Qwen3.5-122B and 22.30 for GPT-OSS-120B. The main asset remains the openness of the weights and data, with a license allowing commercial use. ### Key takeaways Nemotron 3 Super brings a 120B model with 12B active parameters and a long context, with weights, data and recipes published. It is not the leader of its category on every test, but its openness and throughput target the self-hosting of agents.

Tags : LLM, Open Source, MoE, NVIDIA, Nemotron 3, Poids, données et recettes ouverts, NVIDIA Nemotron Open Model License