Tülu 3 405B

2025-01-30 · Publisher : Ai2 · Domain : Artificial Intelligence

Ai2 releases Tülu 3 405B, a model derived from Llama 3.1 405B and post-trained with the open Tülu 3 recipe (SFT, DPO, RLVR), which Ai2 claims rivals DeepSeek V3 and GPT-4o.

### Context Tülu 3 had been released in November 2024 in 8B and 70B sizes. Ai2 applies the same recipe to the largest Llama model, presenting it as the first application of fully open post-training recipes to a model of this size. The aim is to show that open post-training can compare with that of closed models. ### What the model brings - A three-stage post-training: supervised fine-tuning, direct preference optimization (DPO) and reinforcement learning with verifiable rewards (RLVR). - Ai2 observes that RLVR improves the MATH score more at 405B than at 70B and 8B, which it relates to the findings of the DeepSeek-R1 report. - A performance claimed as competitive with or better than DeepSeek V3 and GPT-4o, and better than Llama 3.1 405B Instruct and Nous Hermes 3 405B. - Documented infrastructure: 32 nodes (256 GPUs), inference with 16-way tensor parallelism and 240 additional GPUs for training. - Publication of the code (open-instruct), the weights, the data and the evaluation method. ### Architecture and specifications 405 billion parameters, Meta's Llama 3.1 405B base model. License: Llama 3.1 Community License (HF metadata), with its own restrictions, unlike the Apache 2.0 of the OLMo models. Openness: post-training (data, code, recipes) is fully open, but the pre-training of the Llama base is not; the model is therefore not fully open in the OLMo sense. Context and data cutoff: not recorded. Weights on Hugging Face, with SFT (January 10), DPO (January 28) and final variants. ### Cost and access | Item | Detail | |---|---| | Official API | No Ai2 API for this model | | Hardware | Ai2 used 256 GPUs for training; self-hosting requirement not disclosed (405 billion parameters, roughly 810 GB in BF16, calculated) | ### Benchmarks Ai2 presents a comparison table with DeepSeek V3, GPT-4o and Llama 3.1 405B Instruct, but it is only distributed as an image on the page: no figure could be extracted from the text. The scores appear in the arXiv report 2411.15124. Source: Ai2 post of January 30, 2025, comparison published by Ai2, figures not recorded. ### Versus closed models Ai2 claims performance competitive with or better than GPT-4o and DeepSeek V3, without the figures having been verified here. GPT-4o was then a closed OpenAI model; the gap is therefore supported only by the vendor's wording. The notable point is methodological: the post-training recipe, the data and the code are open, which makes it possible to reproduce or adapt the process. Dependence on the Llama license and on Meta's pre-training limits sovereignty over the base model. The self-hosting cost of a 405-billion-parameter model remains very high. ### Key takeaways Tülu 3 405B demonstrates the value of open post-training at very large scale, but remains a Llama-based model under the Llama license.

Tags : LLM, Open Source, Poids ouverts, Ai2, Tülu 3, Post-entraînement ouvert, Llama 3.1 Community License