NVIDIA Nemotron 3 Ultra: 550B Parameters, 300 Tokens Per Second, and Still Not Beating China's Best Open Model

Architecture - Why 550B Parameters at 55B Active Makes Sense

Nemotron 3 Ultra uses a latent mixture-of-experts (MoE) architecture. The model has 550 billion total parameters spread across many specialized expert sub-networks, but only activates approximately 55 billion parameters per token at inference time. This is the same architecture approach used by Mixtral, DeepSeek-V3, and Kimi K2 - the insight is that not all of a model's knowledge is relevant for every token, so routing each token to the most relevant experts gives you better effective utilization than a dense model of equal active parameter count.

The practical result: inference cost is governed by active parameter count (55B), not total parameter count (550B). This is why Nemotron 3 Ultra can deliver 300+ output tokens per second while costing approximately 30% less per inference than leading alternatives. Running the model requires datacenter GPUs - a 550B parameter model cannot run on consumer hardware regardless of architecture. NVIDIA is deploying it through NIM microservices on build.nvidia.com, making API access available without the need to run your own inference cluster.

The model is designed specifically for complex reasoning, planning, and agentic workflows - AI systems that plan, execute, and iterate on multi-step tasks with minimal human oversight. NVIDIA positioned it not as a general-purpose chat model but as infrastructure for enterprise agentic applications. The Nemotron 3 family now has three tiers: Nano (lightweight tasks, weights published with 2.5T pre-training tokens), Super (120B parameters, March 2026, mid-range enterprise), and Ultra (550B, June 4, complex reasoning and agentic work).

Benchmarks - Where It Leads and Where It Trails

The Intelligence Index of 48 is the key figure from Artificial Analysis, who evaluated the model at NVIDIA's request before the Computex announcement. The 12-point gap over Nemotron 3 Super is large for this benchmarking landscape. The speed advantage over Chinese open models is real and measurable: at 300+ tokens per second on a pre-release DeepInfra endpoint, Nemotron 3 Ultra runs 3-6x faster than Kimi K2.6 and comparable Chinese models - a significant advantage for latency-sensitive enterprise applications.

The honest limitation: Kimi K2.6 still leads the overall open-weight intelligence ranking despite its lower speed. Nemotron 3 Ultra wins on speed and US-jurisdiction open-weights deployment; it does not win on raw reasoning capability against the best Chinese open models. For teams that require open-weights models on US-controlled infrastructure (defense, finance, regulated health), Nemotron 3 Ultra has no peer. For teams that just want the best open-weights reasoning capability regardless of origin, Kimi K2.6 is still the benchmark leader.

Availability and How to Access It

Nemotron 3 Ultra ships June 4, 2026 - three days after the Computex announcement. Access options:

The Bigger Picture - NVIDIA's Open Model Strategy

NVIDIA has been in the model business longer than most people realize. The Nemotron family has accumulated over 50 million downloads across all variants in the year leading up to April 2026. But Nemotron 3 Ultra represents a strategic escalation: this is NVIDIA explicitly positioning itself as a full-stack AI platform company, not just a chip vendor that happens to release a model occasionally.

The Nemotron Coalition is part of this strategy - a group of eight AI labs including Mistral AI and Perplexity that NVIDIA assembled in March 2026 to co-develop open frontier models on DGX Cloud infrastructure. Coalition members get early access to NVIDIA's training infrastructure; NVIDIA gets co-development credibility and distribution into the coalition members' user bases. Nemotron 4, already confirmed as under development, is the next product of this coalition model.

The open-weights strategy also addresses a competitive threat from China that NVIDIA is uniquely positioned to see. Chinese labs have flooded the open ecosystem with strong models - from roughly 1.2% of global open-model usage in late 2024 to around 30% by end of 2025, per Decrypt. For US enterprises and developers that need capable open-weights models but require US-jurisdiction infrastructure, Nemotron 3 Ultra is NVIDIA's answer. The question is whether capability parity with Chinese open models is achievable on the same cost curve, and Nemotron 3 Ultra's Intelligence Index of 48 vs Kimi K2.6's #1 ranking suggests the gap is not yet closed.

Frequently Asked Questions