MiMo-7B

2025-04-30 · Publisher : Xiaomi · Domain : Artificial Intelligence

Xiaomi releases MiMo-7B, its first family of open-weights language models (MIT): a 7-billion-parameter model trained for reasoning, at the level of o1-mini in mathematics and code.

Xiaomi releases MiMo-7B, its first family of open-weights language models (MIT): a 7-billion-parameter model trained for reasoning, at the level of o1-mini in mathematics and code. ### Context In early 2025, open reasoning models that succeeded at reinforcement learning in code and mathematics mostly relied on large bases, around 32 billion parameters, according to Xiaomi's report. DeepSeek-R1 had just shown that an open model could rival OpenAI's closed reasoning models. Xiaomi, known for its smartphones, enters the field of open language models with this model and seeks to prove that a small model can reason as well as a larger one. ### What the model brings - A model designed for reasoning from pre-training: improved data pipeline to increase the density of reasoning patterns, plus synthetic reasoning data, over about 25 trillion tokens in three mixing stages. - Multi-token prediction (MTP) added as a training objective, with an MTP layer that also serves for speculative decoding: Xiaomi reports an acceptance rate of about 90%. - Reinforcement learning on 130,000 mathematics and code problems verifiable by rules, with correctness-only rewards to avoid reward hacking, and a code reward weighted by test difficulty. - Dedicated training infrastructure (Seamless Rollout Engine): Xiaomi announces training 2.29 times faster and validation 1.96 times faster. - Four checkpoints released: Base, SFT, RL-Zero (RL from the base) and RL (RL from the SFT), useful for research. - Update of May 30, 2025 (MiMo-7B-RL-0530): SFT set raised from about 500,000 to 6 million examples and RL window extended from 32,000 to 48,000 tokens; AIME 2024 goes from 68.2 to 80.1 according to the model card. ### Architecture and specifications - Size: 7 billion parameters, dense model; pre-training on about 25 trillion tokens. - Modalities: text. Languages, knowledge cutoff and context length: not disclosed in the model card consulted. - License: MIT (commercial use allowed, with no threshold or geographic restriction). - Weights: MiMo-7B-Base, MiMo-7B-SFT, MiMo-7B-RL-Zero, MiMo-7B-RL on Hugging Face and ModelScope. ### Cost and access No API pricing is published in the sources consulted for this model. The model is intended for self-hosting; the minimum hardware is not specified by the vendor. | Item | Information | | --- | --- | | Official API | Pricing not disclosed | | Self-hosting | 7-billion-parameter model; required hardware not disclosed | | License | MIT | ### Benchmarks | Benchmark | MiMo-7B-RL | OpenAI o1-mini | GPT-4o-0513 | Claude 3.5 Sonnet 1022 | QwQ-32B-Preview (open) | | --- | --- | --- | --- | --- | --- | | MATH-500 (Pass@1) | 95.8 | 90.0 | 74.6 | 78.3 | 90.6 | | AIME 2024 (Pass@1) | 68.2 | 63.6 | 9.3 | 16.0 | 50.0 | | AIME 2025 (Pass@1) | 55.4 | 50.7 | 11.6 | 7.4 | 32.4 | | LiveCodeBench v5 (Pass@1) | 57.8 | 53.8 | 32.9 | 38.9 | 41.9 | | GPQA Diamond (Pass@1) | 54.4 | 60.0 | 49.9 | 65.0 | 54.5 | | MMLU-Pro (EM) | 58.6 | 80.3 | 72.6 | 78.0 | 52.0 | | IF-Eval (Prompt Strict) | 61.0 | 84.8 | 84.3 | 86.5 | 40.4 | Source: Hugging Face model card of MiMo-7B-RL. Scores measured by Xiaomi, the scores of the closed models being taken from the same table in the card; no third-party measurement. ### Versus closed models On the mathematics and code tests in the table, MiMo-7B-RL surpasses o1-mini: 68.2 versus 63.6 on AIME 2024, 55.4 versus 50.7 on AIME 2025 and 57.8 versus 53.8 on LiveCodeBench v5. It is clearly ahead of GPT-4o and Claude 3.5 Sonnet, which are not reasoning models, on mathematics (for example 68.2 versus 9.3 and 16.0 on AIME 2024). However, it remains behind on general knowledge and instruction following: 58.6 on MMLU-Pro versus 80.3 for o1-mini, and 61.0 on IF-Eval versus 84.8; the gap on GPQA Diamond is more modest (54.4 versus 60.0). The closed models compared in the card date from 2024; this is not OpenAI's most recent reasoning generation at that date. For a company, the interest lies in the size: a 7-billion-parameter model under the MIT license can be hosted internally for targeted reasoning tasks, provided one accepts weaker knowledge coverage. ### Key takeaways MiMo-7B shows, with the vendor's figures as support, that a small, well-trained model can match o1-mini on mathematics and code. It is a proof-of-concept milestone: the MIT license and the four checkpoints make it a base for research and specialization, not a general-purpose model.

Tags : LLM, Open Source, Xiaomi, MiMo, MIT, Raisonnement, 7B