GPT-4o (Omni)

2024-05-13 · Publisher : OpenAI · Domain : Artificial Intelligence

OpenAI unveils GPT-4o ("o" for "omni"), a single model trained across text, vision and audio that answers by voice in 320 ms on average, matches GPT-4 Turbo at half the price and becomes free in ChatGPT.

OpenAI unveils GPT-4o ("o" for "omni"), a single model trained end to end across text, vision and audio, which answers by voice with a latency close to that of a human conversation and becomes available for free in ChatGPT. ### In detail - A single network handles every modality: no more pipeline of transcription, text model and then speech synthesis, which lost tone, multiple speakers and background noise. - Voice responses in as little as 232 ms, 320 ms on average, compared with 2.8 s (GPT-3.5) and 5.4 s (GPT-4) for the previous Voice Mode. - GPT-4 Turbo level on English text and code, clearly better in other languages, in vision and in audio. - In the API: 2x faster, half the price and 5x higher rate limits than GPT-4 Turbo. - A GPT-4-level model opened to free ChatGPT users. ### Specifications - At launch: text and image input, text output, in ChatGPT and the API. The new Voice Mode arrives in alpha for ChatGPT Plus in the following weeks; audio and video in the API first for a small group of partners. - 128,000-token context, knowledge cutoff October 2023 (model page). - New tokenizer: 1.1x fewer tokens in French, up to 4.4x fewer in Gujarati. - Risk rated "Medium" at most under the Preparedness Framework, on persuasion; more than 70 external red-teaming experts. ### Launch pricing | Model (USD per million tokens) | Input | Output | | --- | --- | --- | | gpt-4o-2024-05-13 | 5 | 15 | ### Benchmarks published by the vendor Scores published by the vendor (simple-evals suite, 0-shot). | Benchmark | GPT-4o | GPT-4 Turbo (2024-04-09) | | --- | --- | --- | | MMLU | 88.7% | 86.5% | | GPQA | 53.6% | 49.1% | | MATH | 76.6% | 72.2% | | HumanEval | 90.2% | 87.6% | | MGSM | 90.5% | 88.6% | | DROP (F1) | 83.4 | 85.4 | ### Why it is a turning point GPT-4o makes voice a credible interface: perception of tone, expressive voice (laughter, singing), real-time translation. It also brings the high end to everyone by making it free in ChatGPT and half the price in the API. ### Key takeaways OpenAI's first "omni" model, it sets the standard for native multimodality. Audio capabilities were opened gradually after the announcement.

Tags : LLM, OpenAI, Multimodal, GPT-4, GPT-4o, Voix, Temps réel