Llama 4 (Scout, Maverick, Behemoth in preview)

2025-04-05 · Publisher : Meta · Domain : Artificial Intelligence

Meta releases Llama 4 Scout and Llama 4 Maverick, its first open-weights, natively multimodal mixture-of-experts models, and previews Behemoth, a 2-trillion-parameter model still in training.

### Context Llama 3.3 (70B, December 2024) was the last dense milestone of the family. In April 2025, the best closed models were GPT-4.5, Claude 3.7 Sonnet and Gemini 2.5 Pro, and Chinese open models (DeepSeek V3) set the tone on the open-weights side. Meta wanted to regain the advantage with an MoE architecture and a very long context. ### What the model brings - Mixture-of-experts architecture: only 17 billion parameters are active per token, whichever the model (Scout: 16 experts; Maverick: 128 experts). - Native multimodality through early fusion: text and images are processed in the same model (text and image input, text output). - Scout announces a 10-million-token window; Maverick 1 million according to its model card. - Scout fits on a single H100 GPU with on-the-fly int4 quantization according to the model card; Maverick is published in BF16 and FP8. - Behemoth (288 billion active parameters, about 2 trillion in total, 16 experts) serves as a teacher model, with no weights published at the time of the announcement. ### Architecture and specifications | Model | Active / total | Experts | Context | Status | |---|---|---|---|---| | Scout | 17B / 109B | 16 | 10M (announcement) | Weights published | | Maverick | 17B / 400B | 128 | 1M (model card) | Weights published | | Behemoth | 288B / about 2,000B | 16 | not disclosed | Preview, not published | Supported languages: 12 (including French, English, German, Spanish, Arabic, Hindi). Knowledge cutoff date: August 2024. Training: about 22 trillion tokens for Maverick (model card), more than 30 trillion for the family according to Meta; Maverick consumed 2.38 million H100 hours. License: Llama 4 Community License (non-exclusive, free for commercial and research use) with a specific authorization to request from Meta beyond 700 million monthly active users, the "Built with Llama" notice and the "Llama" prefix in the name of derivatives being mandatory; the agreement is with Meta Platforms Ireland for users in the EEA and Switzerland. Weights on Hugging Face (meta-llama) and llama.com. ### Cost and access | Item | Value | |---|---| | Official Meta API at announcement | Pricing not disclosed at announcement | | Scout at third-party hosts | $0.15 input / $0.40 output per million tokens (recorded by The Decoder, April 2025) | | Maverick at third-party hosts | $0.24 input / $0.77 output per million tokens (same source) | | Scout self-hosting | 1 H100 GPU in int4 (model card) | ### Benchmarks | Benchmark (instruction-tuned models) | Scout | Maverick | |---|---|---| | MMMU | 69.4 | 73.4 | | MMLU Pro | 74.3 | 80.5 | | GPQA Diamond | 57.2 | 69.8 | | LiveCodeBench | 32.8 | 43.4 | Source: Llama 4 model card (GitHub meta-llama/llama-models), scores published by the publisher. The announcement's comparison tables against closed models are images that I could not re-read figure by figure: no GPT-4o or Gemini 2.0 Flash scores are therefore reproduced here. Third parties: on the Artificial Analysis Intelligence Index of the time, Maverick scored 49 points (ahead of Claude 3.7 Sonnet, behind DeepSeek V3) and Scout 36 (GPT-4o mini level) according to The Decoder. On Fiction.LiveBench at 120,000 tokens, Maverick reaches 28.1% accuracy against 90.6% for Gemini 2.5 Pro, and Scout 15.6%. Behemoth, figures announced by Meta and relayed by secondary sources: MATH-500 95.0 (Gemini 2.0 Pro 91.8; Claude 3.7 Sonnet 82.2), GPQA Diamond 73.7 (GPT-4.5 71.4; Claude 3.7 Sonnet 68.0). ### The LMArena controversy Meta announced an Elo score of 1417 on LMArena for Maverick for an "experimental chat version". The model tested was named Llama-4-Maverick-03-26-Experimental, distinct from the released version (Llama-4-Maverick-17B-128E-Instruct). LMArena stated on April 8, 2025 that Meta's interpretation of its policy did not match what it expects from providers, and updated its rules. Meta replied that it was a variant optimized for conversation, tested among others. On April 11, the public version ranked 32nd (TechCrunch); with style control, the experimental version dropped from 2nd to 5th place (The Decoder). In January 2026, Yann LeCun told the Financial Times that the results had been "a little bit fudged" (remarks reported by secondary sources). ### Versus closed models At launch, Meta positions Maverick ahead of GPT-4o and Gemini 2.0 Flash, and comparable to DeepSeek V3 with less than half the active parameters: these are the publisher's announcements. Third-party measurements are more nuanced: 49 points on the Artificial Analysis index, below DeepSeek V3, and a sharp drop on long context against Gemini 2.5 Pro (28.1% versus 90.6% at 120,000 tokens). The closed reference model at that date was Gemini 2.5 Pro (announced on March 25, 2025), not compared by Meta. The price gap remains the main argument: a few tens of cents per million tokens at hosts, with no license fee below 700 million users. For an open-versus-closed choice, the license is not OSI: it imposes attribution and a user threshold. ### Key takeaways Llama 4 marks Meta's shift to multimodal MoE and a long context, but the reliability of its announcements (LMArena, 10M context) has been contested. Behemoth was not published: Meta postponed it (May 2025, press) and then changed strategy.

Tags : LLM, Open Source, Meta, Llama, MoE, Multimodal, Llama 4 Community License