Cloudflare: a configuration file twice its normal size blocks part of the Internet for nearly six hours

2025-11-18 · Publisher : Cloudflare · Domain : Cloud & DevOps

On 18 November 2025, a permissions change in a ClickHouse database doubles the size of a bot management file. Cloudflare's proxy crashes: 5xx errors on the CDN for 5 hours 46 minutes, with Access, Turnstile and Workers KV affected.

### Background On 18 November 2025 at 11:20 UTC, **Cloudflare** suffered a major outage that lasted until 17:06 UTC (**5 hours 46 minutes**). In its post-mortem, the company describes the causal chain: a permissions change on a **ClickHouse** database, deployed at 11:05 UTC, altered the behaviour of a query. The generation of the **bot management feature file** queried metadata without filtering by database, which returned duplicate columns and **doubled** the file size. It exceeded the 200-feature memory limit of the FL2 proxy, whose code did not handle the error: "thread fl2_worker_thread panicked: called Result::unwrap() on an Err value". ### Key facts - Services affected: CDN and security services (HTTP 5xx errors), Turnstile, Workers KV, the dashboard, Cloudflare Access (authentication failures) and email security (temporary loss of IP reputation). - Remediation: stopping propagation of malformed files at 14:24 UTC, redeploying the last known good file at 14:30, forced restart of the core proxy. - Follow-ups: harden the ingestion of configuration files, expand global kill switches, prevent error reports from consuming too many resources, review failure modes of proxy modules. ### Why it matters Cloudflare is a shared entry point for a very large number of sites and services: when it fails, unrelated applications fail together. The incident also shows that an automatically generated configuration file must be treated as untrusted input. For IT leaders, it is the argument for documenting dependence on a CDN or an identity provider, and for planning a tested bypass path. ### Takeaway An automatically generated configuration file became unvalidated input that took down the proxy of one of the largest delivery networks.

Tags : Incident, Edge, Cloud, Sécurité