ERNIE 4.5

2025-06-30 · Publisher : Baidu · Domain : Artificial Intelligence

Baidu opens the entire ERNIE 4.5 family under the Apache 2.0 license: ten models, from a 0.3-billion dense model to multimodal MoEs with 47 billion active parameters, the largest totaling 424 billion.

### Context ERNIE models were previously offered as closed services. In 2025, after DeepSeek-R1 and the wave of Chinese open models, Baidu in turn opens the ERNIE 4.5 family on June 30, a few days after Tencent. The closed reference compared is GPT-4.1, the open reference DeepSeek-V3-0324 and Qwen3-235B-A22B. ### What the model brings - Heterogeneous multimodal MoE: text and vision are trained together, with modality-isolated routing, a router orthogonality loss and a multimodal balancing loss, so that one modality does not degrade the other. - Training efficiency: heterogeneous hybrid parallelism, FP8 precision and fine-grained recomputation, with 47% model FLOPs utilization (MFU) on 2,016 NVIDIA H800 GPUs for the largest text model. - Near-lossless quantization at 4 bits and 2 bits: 4 GPUs of 80 GB at 4 bits, or one 141 GB GPU at 2 bits, for the largest model; announced throughput of 56,000 input tokens and 18,000 output tokens per second per H800 node. - Modality-specific post-training: SFT, direct preference optimization (DPO) or a modified reinforcement method (UPO); the vision models handle modes with and without thinking. - Baidu claims that ERNIE-4.5-300B-A47B outperforms DeepSeek-V3 (671 billion) on 22 of 28 benchmarks, and that ERNIE-4.5-21B-A3B outperforms Qwen3-30B with about 30% fewer parameters (vendor claims). ### Architecture and specifications - Family: 10 variants, including ERNIE-4.5-300B-A47B (text), ERNIE-4.5-21B-A3B (text), ERNIE-4.5-0.3B (dense), ERNIE-4.5-VL-424B-A47B and ERNIE-4.5-VL-28B-A3B (vision-language). - ERNIE-4.5-300B-A47B: 300 billion parameters in total, 47 billion active, 54 layers, 64 text experts (8 active) and 64 vision experts (8 active), 131,072-token context. - Languages: English and Chinese. Training data and knowledge cutoff: not disclosed in the sources consulted. - License: Apache 2.0 (free commercial use). Weights: PyTorch (-PT) and PaddlePaddle (-Paddle) versions on Hugging Face, GitHub and Baidu AI Studio. ### Cost and access No API pricing was consulted for ERNIE 4.5. For self-hosting the largest model, the technical report indicates 4 GPUs of 80 GB with 4-bit quantization or 1 GPU of 141 GB with 2-bit quantization. | Item | Information | | --- | --- | | Official Baidu API | Pricing not consulted | | Largest model, 4 bits | 4 GPUs of 80 GB | | Largest model, 2 bits | 1 GPU of 141 GB | | License | Apache 2.0 | ### Benchmarks | Benchmark | ERNIE-4.5-300B-A47B | GPT-4.1 | DeepSeek-V3-0324 (open) | Qwen3-235B-A22B (open) | | --- | --- | --- | --- | --- | | MMLU | 86.5 | 90.2 | 86.5 | 85.5 | | MMLU-Pro | 78.4 | 80.7 | 81.2 | 72.9 | | SimpleQA | 45.9 | 40.2 | 27.3 | 12.4 | | ChineseSimpleQA | 77.1 | 64.0 | 72.0 | 63.0 | | MATH-500 | 96.4 | 92.8 | 94.0 | 91.2 | | AIME 2024 | 54.8 | 48.1 | 59.4 | 40.1 | | AIME 2025 | 35.1 | 36.7 | 47.7 | 24.7 | | LiveCodeBench | 38.8 | 40.5 | 45.4 | 36.1 | | IFEval | 88.0 | 87.4 | 82.3 | 83.2 | | Multi-IF | 76.6 | 70.8 | 66.9 | 70.2 | Source: ERNIE 4.5 technical report, table 5 (June 29, 2025). Scores measured by Baidu. For vision in thinking mode (table 8 of the same report), OpenAI o1 scores 71.4 on MathVista, 77.7 on MMMU and 81.0 on DocVQA, versus 78.9, 70.0 and 93.1 for ERNIE-4.5-VL-424B-A47B. ### Versus closed models Against GPT-4.1, ERNIE-4.5-300B-A47B is ahead on factual knowledge (SimpleQA 45.9 versus 40.2; ChineseSimpleQA 77.1 versus 64.0), in mathematics on MATH-500 (96.4 versus 92.8) and on instruction following (Multi-IF 76.6 versus 70.8). It is behind on MMLU (86.5 versus 90.2), MMLU-Pro (78.4 versus 80.7), AIME 2025 (35.1 versus 36.7) and LiveCodeBench (38.8 versus 40.5). Baidu writes that the results in mathematics and code are comparable to those of GPT-4.1 and DeepSeek-V3-0324, and that progress remains possible on difficult tasks. In vision with thinking, the largest model surpasses OpenAI o1 on MathVista, DocVQA and ChartQA but remains behind on MMMU (70.0 versus 77.7). GPT-4.1 is a non-reasoning model from spring 2025: the table says nothing about closed reasoning models. No pricing is available to compare costs. The open choice makes sense through the Apache 2.0 license across the whole range, the presence of small models that can be used in-house and the announced 2-bit and 4-bit quantization. ### Key takeaways ERNIE 4.5 is Baidu's entry into open models: a complete family under Apache 2.0, with vendor results close to GPT-4.1 and DeepSeek-V3-0324. Its strength is the range, which goes from 0.3B to the multimodal MoE, and the quantization and deployment tooling published with it.

Tags : LLM, Open Source, Baidu, ERNIE, Apache 2.0, MoE, Multimodal