QwQ-32B

2025-03-06 · Publisher : Alibaba · Domain : Artificial Intelligence

Alibaba Qwen releases QwQ-32B, a 32 billion parameter reasoning model under Apache 2.0, trained with reinforcement learning and presented as comparable to DeepSeek-R1 (671 billion parameters).

### Context In early 2025, DeepSeek-R1 showed that long reasoning obtained through reinforcement could rival OpenAI's closed reasoning models. QwQ-32B responds with a much smaller model, able to run on few GPUs. The blog cites o1-mini and the distilled versions of DeepSeek-R1 as references. ### What the model brings - Two-stage reinforcement training from a cold start: first mathematics and code with verifiable rewards (accuracy verifier, code execution server), then general capabilities with a reward model and rule-based verifiers. - Announced parity with DeepSeek-R1 (671 billion parameters, of which 37 billion active) for 32 billion parameters. - Built-in agent capabilities: tool calling and adapting reasoning based on environment feedback. - Open weights under Apache 2.0, also accessible through Qwen Chat. ### Architecture and specifications Dense model with 32.5 billion parameters, 64 layers, 131,072 token context (Hugging Face card), Qwen2.5 architecture (RoPE, SwiGLU, RMSNorm, QKV bias). Languages and cutoff date: not disclosed in the sources consulted. Apache 2.0 license, commercial use allowed. Weights on Hugging Face and ModelScope. ### Cost and access | Item | Value | |---|---| | Weights | Free, Apache 2.0 | | Alibaba Cloud API (qwq-plus, current price consulted via the pricing page, not the launch price) | $0.8 input / $2.4 output per million tokens | | Required hardware | Not disclosed by the vendor | ### Benchmarks The blog presents the comparisons (AIME24, LiveCodeBench, LiveBench, IFEval, BFCL) only as an image chart, with no values in text; the Hugging Face card does not give any either. No numeric table is therefore reproduced here. The models compared by the vendor are DeepSeek-R1, its distilled Qwen-32B and Llama-70B versions, and o1-mini. ### Versus closed models The only closed model cited in the vendor's comparison is OpenAI's o1-mini; the vendor states that QwQ-32B surpasses or matches it on the series of tests presented, without the figures being available in text. The central claim is parity with DeepSeek-R1 with about twenty times fewer total parameters. For an open-versus-closed choice, this means that a reasoning model can run on one or a few GPUs, under a permissive license. Without verified figures in text, the exact size of the gap with the closed reasoning models of March 2025 is not documented here. ### Key takeaways The blog specifies that both reinforcement stages were run on a base model already rich in world knowledge, which according to the vendor explains the effectiveness of the method. For a company, the practical point is the combination of the Apache 2.0 license, the 32 billion parameter size and agent capability. QwQ-32B is the milestone where Qwen demonstrates that large-scale reinforcement on a strong base model compensates for size. It is a reasoning model that can be deployed in-house for a modest hardware cost.

Tags : LLM, Open Source, Alibaba, Qwen, Raisonnement, Apache 2.0