Qwen3-Next

2025-09-11 · Publisher : Alibaba · Domain : Artificial Intelligence

Alibaba Qwen introduces Qwen3-Next-80B-A3B, an ultra-sparse MoE with 80 billion parameters (3 billion active) and hybrid attention, released in Instruct and Thinking versions under Apache 2.0.

### Context With Qwen3 and Qwen3-2507, the family has been growing toward ever larger MoEs and longer contexts. Qwen3-Next goes the other way: reducing the cost per token at comparable quality by changing the architecture. The Thinking model is compared with Gemini 2.5 Flash Thinking. ### What the model brings - Hybrid attention: three Gated DeltaNet layers (linear attention) for every one classic gated attention layer, which speeds up long-context processing. - Very highly sparse MoE: 512 experts, 10 active plus 1 shared expert, for 3 billion activated parameters out of 80. - Multi-token prediction (MTP) and stability optimizations (zero-centered normalization with weight decay). - The base model outperforms Qwen3-32B-Base with 10% of the training cost and 10 times the inference throughput beyond 32,000 tokens, according to the vendor. - Reinforcement learning stabilized with GSPO, a method described in a dedicated post by the Qwen team. ### Architecture and specifications 80 billion parameters in total, 3 billion active, 48 layers (12 blocks of 3 Gated DeltaNet + MoE layers followed by 1 Gated Attention + MoE layer), hidden dimension 2048, pre-training on 15 trillion tokens. Native context of 262,144 tokens, extensible to 1,010,000. Two variants: Instruct (direct mode) and Thinking (reasoning only). Apache 2.0 license. Knowledge cutoff: not disclosed. ### Cost and access | Item | Value | |---|---| | Weights | Free, Apache 2.0 | | Alibaba Cloud API, Instruct and Thinking (current pricing consulted) | $0.15 input / $1.2 output per million tokens | | Required hardware | Not disclosed (transformers, vLLM, SGLang support) | ### Benchmarks Thinking, Hugging Face model card (vendor measurements). | Benchmark | Gemini 2.5 Flash Thinking | Qwen3-32B Thinking | Qwen3-235B-A22B-Thinking-2507 | Qwen3-Next-80B-A3B-Thinking | |---|---|---|---|---| | AIME25 | 72.0 | 72.9 | 92.3 | 87.8 | | HMMT25 | 64.2 | 51.5 | 83.9 | 73.9 | | LiveCodeBench v6 | 61.2 | 60.6 | 74.1 | 68.7 | | GPQA | 82.8 | 68.4 | 81.1 | 77.2 | | Arena-Hard v2 | 56.7 | 48.4 | 79.7 | 62.3 | | BFCL-v3 | 68.6 | 70.3 | 71.9 | 72.0 | Source: Hugging Face model card for Qwen3-Next-80B-A3B-Thinking, vendor measurements. For Instruct, the card compares only Qwen models: Arena-Hard v2 at 82.7 versus 79.2 for Qwen3-235B-A22B-Instruct-2507. ### Versus closed models The only closed model in the vendor's comparison is Gemini 2.5 Flash Thinking. Qwen3-Next-Thinking outperforms it on AIME25 (87.8 versus 72.0), HMMT25, LiveCodeBench v6 (68.7 versus 61.2), Arena-Hard v2 and BFCL-v3, and stays behind on GPQA (77.2 versus 82.8). The model is not compared with top-tier closed models (o3, Gemini 2.5 Pro) in the card. Its appeal is mainly economic: 3 billion active parameters and a claimed throughput 10 times higher beyond 32,000 tokens, with an API price of $0.15 for input. These throughput gains are vendor claims, not independently verified. ### Key takeaways Qwen3-Next is a technical milestone: a hybrid, ultra-sparse architecture, the first step of the Qwen3-Next series according to the vendor. It aims to make long context cheaper to serve.

Tags : LLM, Open Source, Alibaba, Qwen, MoE, Apache 2.0, Architecture