OpenAI o1 (Strawberry)

2024-09-12 · Publisher : OpenAI · Domain : Artificial Intelligence

OpenAI launches the o1 series, models trained with reinforcement learning to think before they answer. o1-preview and o1-mini ship in ChatGPT and the API; o1 reaches 83% on AIME 2024 versus 13% for GPT-4o.

What changes

  • A new paradigm: the model produces a long internal chain of thought before answering, and its performance improves with the thinking time it is given (test-time compute).
  • Math: 74.4% on AIME 2024 with a single sample and 83.3% with consensus over 64 samples for o1, versus 9.3% and 13.4% for GPT-4o.
  • First model to surpass PhD-level human experts on GPQA Diamond (77.3% for o1); 89th percentile on Codeforces.
  • Two models available on day one: o1-preview (USD 15 input and USD 60 output per million tokens) and o1-mini, 80% cheaper and geared toward code.
  • The raw chain of thought is hidden from the user, who only sees a model-generated summary.

OpenAI introduces o1, a new series of models trained with reinforcement learning to reason before answering. Two models open on day one, o1-preview and o1-mini, while the published scores also cover o1, the next version then in development. OpenAI resets the counter to 1: the series is now called OpenAI o1. ### In detail - The model produces a long internal chain of thought: it breaks the problem down, spots and corrects its mistakes, and changes approach when needed. - Performance improves with reinforcement learning training compute and with thinking time at inference, a new scaling axis beyond pretraining. - A massive gap with GPT-4o on reasoning tasks: competition math, programming, science. - The raw chain of thought stays hidden; the user sees a model-generated summary. - Safety: on one of OpenAI's hardest jailbreaking tests, o1-preview scores 84 out of 100 versus 22 for GPT-4o. ### Specifications - o1-preview: 128,000-token context, 32,768 output tokens at most, knowledge cutoff October 2023, text only (model page). - o1-mini: a faster model, 80% cheaper, effective at coding, 128,000-token context. - ChatGPT Plus and Team from launch, with 30 messages per week for o1-preview and 50 for o1-mini; API restricted to usage tier 5, at 20 requests per minute, without function calling, streaming or system messages. ### Launch pricing | Model (USD per million tokens) | Input | Output | | --- | --- | --- | | o1-preview | 15 | 60 | | o1-mini | 3 | 12 | ### Benchmarks published by the vendor Scores published by the vendor (pass@1 unless noted). | Benchmark | GPT-4o | o1-preview | o1 | | --- | --- | --- | --- | | AIME 2024 | 9.3% | 44.6% | 74.4% | | AIME 2024 (consensus over 64) | 13.4% | 56.7% | 83.3% | | Codeforces (Elo) | 808 | 1,258 | 1,673 | | GPQA Diamond | 50.6% | 73.3% | 77.3% | | MATH | 60.3% | 85.5% | 94.8% | | MMLU | 88.0% | 90.8% | 92.3% | ### Why it is a turning point o1 opens the era of reasoning models: capability no longer depends only on model size but also on the compute spent on each question. The whole industry will follow this path. ### Key takeaways Less versatile than GPT-4o at launch (no web browsing or file uploads), o1 stands out on hard problems in science, math and programming.

Links

Tags : LLM, OpenAI, Raisonnement, o1, o1-preview, o1-mini, Chaîne de pensée