Scalr incident
Google Cloud Storage outage is impacting Scalr services.
Scalr experienced a major incident on June 12, 2025 affecting Scalr Platform and Scalr Worker, lasting 1h 55m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jun 12, 2025, 06:22 PM UTC
We are currently investigating this issue.
- investigating Jun 12, 2025, 06:38 PM UTC
Google cloud storage is currently experiencing issues.
- investigating Jun 12, 2025, 06:49 PM UTC
Google Cloud storage is experiencing a widespread outage and not accessible.
- investigating Jun 12, 2025, 06:49 PM UTC
We are continuing to investigate this issue.
- identified Jun 12, 2025, 07:22 PM UTC
Google Cloud has identified and is mitigating the issue. They have confirmed the issue has recovered in some locations.
- identified Jun 12, 2025, 07:22 PM UTC
Google Cloud has identified and is mitigating the issue. They have confirmed the issue has recovered in some locations. Scalr is still seeing intermittent issues.
- monitoring Jun 12, 2025, 08:43 PM UTC
According to our monitoring, less than 1% of requests are currently affected by the Google incident. Majority of the users should see a full recovery at this point.
- resolved Jun 12, 2025, 09:17 PM UTC
This incident has been resolved.
- postmortem Jun 17, 2025, 01:10 PM UTC
On Thursday, June 12, 2025, starting at approximately 2:00 PM EST, our customers experienced service errors and degraded performance. Full service was restored for all users by 5:15 PM EST. This disruption was caused by a major global outage across Google Cloud services, which our platform relies on. According to Google's post-incident analysis, a recent update to their internal "Service Control" system contained a bug. This bug was triggered by a routine policy change, causing a critical component of their API infrastructure to fail worldwide. This failure resulted in the errors our customers experienced. The detailed RCA can be found here: [https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW](https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW) We apologize for the significant impact this had on your operations and appreciate your understanding. The Scalr Team