New term: reasoning model

2024-09-12 · Publisher : OpenAI · Domain : Artificial Intelligence

A reasoning model is trained to "think" before answering by producing a chain of thought. Test-time compute is the extra computation this takes, which became a new lever for performance.

### Definition A **reasoning model** is a large language model trained, notably through reinforcement learning, to produce a chain of thought before its final answer. **Test-time compute** is the computation spent at answer time: the more is granted, the better the results on hard problems. ### Origin On September 12, 2024, OpenAI introduced o1-preview and o1-mini in the post "Learning to reason with LLMs": "We've developed a new series of AI models designed to spend more time thinking before they respond." OpenAI added that o1's performance "consistently improves with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute)". ### What the term covers - On the 2024 AIME exams, GPT-4o solved 12% of problems on average, o1 74% with a single sample. - Reasoning tokens are hidden in the API but billed as output tokens. - Qwen QwQ, Gemini Flash Thinking, DeepSeek R1 and then Claude with extended thinking adopted the approach in the following months. ### Criticism and limits Reasoning is expensive, slows responses down and brings nothing on simple tasks. Whether the displayed chain of thought faithfully reflects the model's actual computation is still debated, and so is the word "reasoning" itself. ### Key takeaway Since o1, answer quality is also tuned by how long the model is allowed to think, with an explicit trade-off between accuracy, cost and latency.

Tags : Vocabulaire, Raisonnement, Inférence, Coûts