New term: context rot
2025-07-14 · Publisher : Chroma · Domain : Artificial Intelligence
Context rot is the drop in a model's reliability as its context grows longer, even on simple tasks and well below its maximum window size.
What changes
- A large context window no longer means a well-used context.
- Needle-in-a-haystack tests are no longer enough to judge long-context performance.
- Summarizing, pruning and starting over from a clean context become basic habits.
- The term gives context engineering a concrete rationale.
### Definition **Context rot** describes the loss of answer quality as the number of input tokens grows, especially when the context piles up distractions, dead ends or irrelevant information. ### Origin The phrase first appeared on June 18, 2025 in a Hacker News comment by user Workaccount2, picked up the same day by Simon Willison: "Maybe you can call it context rot, where as context grows and especially if it grows with lots of distractions and dead ends, the output quality falls off rapidly." On July 14, 2025, Chroma (Kelly Hong, Anton Troynikov, Jeff Huber) used it as the title of a technical report, "Context Rot: How Increasing Input Tokens Impacts LLM Performance", which established the term: that is the date chosen here. ### What the term covers - Chroma evaluated 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3, and concluded: "Models do not use their context uniformly; instead, their performance grows increasingly unreliable as input length grows." - Degradation worsens when the question is semantically distant from the target information and when distractors are present. - The experiment code is published so the results can be reproduced. ### Criticism and limits Chroma sells a vector database and has an interest in promoting targeted retrieval; its experiments are nonetheless reproducible. The size of the effect varies across models and tasks. ### Key takeaway Putting everything into the context is not a strategy: a short, relevant context beats a full window.
Links
Tags : Vocabulaire, Contexte, Qualité, Évaluation