Google Workspace incident
Google Classroom experiencing elevated latency issue
Google Workspace experienced a minor incident on June 9, 2025 affecting Classroom, lasting 1h 53m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jun 09, 2025, 11:28 PM UTC
Description: We are experiencing an Classroom beginning at Monday, 2025-06-09 15:00 PDT Our engineering team continues to investigate the issue. We will provide an update by Monday, 2025-06-09 17:00 PDT with current details. We apologize to all who are affected by the disruption. Diagnosis: The impacted users would experience increased latency while using Classroom. Workaround: None at this time.
- investigating Jun 09, 2025, 11:51 PM UTC
Description: Mitigation work is currently underway by our engineering team. We do not have an ETA for mitigation at this point. We will provide more information by Monday, 2025-06-09 17:50 PDT. Diagnosis: The impacted users would experience increased latency while using Classroom. Workaround: None at this time.
- resolved Jun 10, 2025, 12:12 AM UTC
Description: The issue with Classroom has been resolved for all affected users as of Monday, 2025-06-09 17:00 PDT. The issue was due to a recent change in the Spanner Database which prevented many users from being able to access Classroom. Our engineers have mitigated the issue by rolling back the recent release to the previous version. We will publish an analysis of this incident once we have completed our internal investigation. We thank you for your patience while we worked on resolving the issue. Diagnosis: The impacted users experienced increased latency while using Classroom. Workaround: None at this time.
- resolved Jun 10, 2025, 04:02 AM UTC
Affected services and features: Google Classroom Regions/Zones: Global Description: Google Classroom users experienced elevated latency for a duration of 1 hour and 53 minutes. The incident was triggered by a schema change applied to Google's globally distributed database. This schema change caused a surge in utilization across multiple zones, reaching the maximum utilization at 100%. As a result, a backend service relying on the database experienced a significant drop in availability and performance, impacting Classroom users. The incident was mitigated by rolling back the schema change. Google will complete a detailed Incident report in the following days that will provide a full root cause. Customer impact: Impacted users experienced increased latency while using Google Classroom.
- resolved Jun 12, 2025, 12:39 PM UTC
Incident Report Summary On Monday, 9 June 2025, at 15:07 US/Pacific, Google Classroom experienced increased latency for a duration of 1 hour 53 minutes. We deeply regret the disruption our Google Classroom users experienced. We know you rely on our services, and we fell short of meeting your expectations. We are taking immediate action to improve our platform's reliability and prevent future disruptions. Root Cause The incident was triggered by a schema change applied to Google's globally distributed database, which stores and serves a majority of Classroom data. This schema change caused a surge in CPU utilization across multiple clusters, reaching the maximum utilization at 100 percent. As a result, a backend service relying on the database experienced a significant drop in availability and performance, impacting Google Classroom users. Remediation and Prevention Google engineers were alerted to the outage via a monitoring alert on June 9 at 15:07 US/Pacific and immediately started an investigation. The issue was mitigated on 9 June at 17:00 US/Pacific once the schema change was fully rolled back. Google is committed to preventing a repeat of this issue in the future and is completing the following actions: Update Google Classrooms database release process efficiency to use incremental rollout, to aid in identifying potential impact and prevent a recurrence. Implement proactive identification for schema changes with potential wide-reaching impact prior to rollout. Google is committed to quickly and continually improving our technology and operations to prevent service disruptions. We appreciate your patience and apologize again for the impact to your organization. We thank you for your business. Detailed Description of Impact Google Classroom On 9 June 2025, from 15:07 to 17:00 US/Pacific Google Classroom users experienced increased latency wherein the application may have failed to load for an extended period. Google Classroom experienced a significant drop in availability and performance for approximately ~0.1% of users for a duration of 1 hour and 53 minutes. Most requests made to the database experienced latency or unavailability during the incident time.