Qwen2.5-Omni

2025-03-27 · Publisher : Alibaba · Domain : Artificial Intelligence

Alibaba Qwen releases Qwen2.5-Omni-7B, an end-to-end multimodal open-weights model: it perceives text, images, audio and video and responds in text or in speech synthesized as a continuous stream.

### Context Voice and video assistants had until then relied on separate toolchains (speech recognition, LLM, synthesis). The blog compares Qwen2.5-Omni with the Qwen2.5-VL-7B and Qwen2-Audio models and with Gemini 1.5 Pro. ### What the model brings - Thinker-Talker architecture: the Thinker (Transformer decoder with audio and image encoders) understands and produces text, the Talker (dual-track autoregressive decoder) generates speech as a stream. - TMRoPE (Time-aligned Multimodal RoPE), a position encoding that synchronizes video and audio timestamps. - Real-time interaction with chunked input and immediate output. - End-to-end voice instruction following, close to text-based following according to the vendor (MMLU, GSM8K). - Claimed state-of-the-art performance on OmniBench. ### Architecture and specifications 7B version (about 10.7 billion parameters in total on Hugging Face, Thinker and Talker included), then 3B version published on April 30, 2025. Licenses: Apache 2.0 for the 7B, “Qwen Research” license for the 3B. Theoretical minimum GPU memory in BF16 for the 7B: 31.11 GB (15 s video), 41.85 GB (30 s), 60.19 GB (60 s), to be multiplied by at least 1.2 in practice according to the card. Disabling the Talker saves about 2 GB. Context and knowledge cutoff: not verified. Technical report available on arXiv. ### Cost and access Weights on Hugging Face, ModelScope, DashScope and GitHub; demo on Qwen Chat. API pricing not verified for this entry. ### Benchmarks Excerpts from the Hugging Face model card (vendor measurements). | Benchmark | Gemini 1.5 Pro | GPT-4o-mini | Qwen2.5-VL-7B | Qwen2.5-Omni-7B | |---|---|---|---|---| | OmniBench (average) | 42.91% | not published | not published | 56.13% | | MMMU (val) | not published | 60.0 | 58.6 | 59.2 | | MathVista (testmini) | not published | 52.5 | 68.2 | 67.9 | | MMStar | not published | 54.8 | 63.9 | 64.0 | Source: Hugging Face Qwen2.5-Omni-7B card, comparisons chosen by the vendor. ### Versus closed models The only closed models in the table are Gemini 1.5 Pro (OmniBench 42.91% versus 56.13%) and GPT-4o-mini (MMStar 54.8 versus 64.0; MathVista 52.5 versus 67.9), where Qwen2.5-Omni-7B is ahead. On MMMU, GPT-4o-mini keeps a slight advantage (60.0 versus 59.2). The vendor claims vision performance close to Qwen2.5-VL-7B, confirmed by the table. The closed models cited are from a generation prior to March 2025 (Gemini 1.5 Pro) or small models, which limits the scope of the comparison. The main benefit is having a self-hostable voice and video model, with a footprint of 30 to 60 GB of GPU depending on video length. ### Key takeaways Qwen2.5-Omni opens the category of open end-to-end omni models with streaming voice output. The published comparison remains limited to older or lightweight closed references.

Tags : LLM, Open Source, Alibaba, Qwen, Multimodal, Audio, Apache 2.0