Google Cloud outage: an error in Service Control takes down more than 50 services

2025-06-12 · Publisher : Google Cloud · Domain : Cloud & DevOps

On 12 June 2025, a faulty quota policy crashes Service Control, Google Cloud's API management component, in all regions. More than 50 products return 503 errors for about three hours.

What changes

  • Global outage on 12 June 2025, from 10:49 am to about 1:49 pm (Pacific time).
  • An invalid quota policy triggered a null pointer in Service Control, in a crash loop.
  • More than 50 products returned 503 errors, including BigQuery, Cloud Run and Workspace.
  • Cloudflare, whose Workers KV relies on a third-party provider, was also hit for 2 hours 28 minutes.

### Background On 12 June 2025, from 10:49 am to about 1:49 pm (Pacific time), **Google Cloud** suffered a global outage. According to its incident report, a new quota policy feature added to **Service Control** (the component that checks permissions and quotas for API calls) lacked proper error handling and feature-flag protection. An invalid quota update, propagated globally, contained blank fields; they exercised a code path that triggered a null pointer dereference, and the binaries went into a crash loop. ### Key facts - More than 50 products returned 503 errors, including BigQuery, Compute Engine, Cloud Storage, Cloud Run, IAM and Google Workspace services (Gmail, Drive, Docs, Meet, Calendar). - A "red button" disabling the faulty path was activated about 40 minutes after the start, launching recovery in most regions; us-central1 had a longer recovery. - Google commits to modularizing Service Control, enforcing feature flags on critical binaries, improving error handling, introducing exponential backoff and communicating better. - Cloudflare suffered a 2 hour 28 minute outage the same day: its Workers KV service relies in part on a third-party cloud provider, whose outage directly hit it (Cloudflare does not name Google in its post). ### Why it matters The outage shows that a cross-cutting dependency (here, API checks) breaks all regions at once: geographic redundancy does not protect against a global software defect. For IT leaders, it is also a reminder of dependency chains: a third-party service that relies on a hyperscaler goes down with it. Mapping these dependencies is part of the continuity plan. ### Takeaway A global configuration change broke a hyperscaler's control plane, and third-party services with it.

Links

Tags : Incident, Google Cloud, Hyperscaler, Cloud