DoIT incident

Wrong billing data related to Google Cloud Platform's Compute Engine, where CUDS are applied to projects.

Minor Resolved View vendor source →

DoIT experienced a minor incident on May 23, 2024 affecting Cloud Cost Analytics and Anomaly Detection, lasting 52d 23h. The incident has been resolved; the full update timeline is below.

Started
May 23, 2024, 03:21 PM UTC
Resolved
Jul 15, 2024, 02:41 PM UTC
Duration
52d 23h
Detected by Pingoru
May 23, 2024, 03:21 PM UTC

Affected components

Cloud Cost AnalyticsAnomaly Detection

Update timeline

  1. investigating May 23, 2024, 03:21 PM UTC

    We are currently experiencing a delay in the delivery of credits data, which has triggered various anomalies across our customer base. This issue has been identified as a problem on Google's end. Specifically, it is affecting anomaly detection on GCP projects that contain Compute Engine Committed Use Discounts (CUDs). We are in contact with Google and will provide updates as soon as they resolve the issue and the data is corrected. We apologize for any inconvenience this may cause and appreciate your patience.

  2. identified May 24, 2024, 11:50 AM UTC

    Google has acknowledged the issue and is currently addressing it. We appreciate your patience while Google is working on finding a solution.

  3. identified May 28, 2024, 03:35 PM UTC

    Google has escalated this issue and is actively addressing its root cause. The next anticipated update is expected on May 29th. We are grateful for your continued patience as we collaborate with Google to address this issue.

  4. identified May 30, 2024, 06:27 AM UTC

    Google has confirmed that they are still working on delivering a solution. The next anticipated update is expected on May 31st. We appreciate your ongoing patience as we work with Google to resolve this issue.

  5. identified Jun 03, 2024, 09:03 AM UTC

    We are working closely with Google to escalate and resolve this issue. Thank you for your ongoing patience as we continue to work on this.

  6. identified Jun 17, 2024, 03:26 PM UTC

    We are actively engaged in resolving this issue with Google. Your patience is greatly valued as we actively collaborate with Google to swiftly address and resolve this issue.

  7. identified Jul 01, 2024, 07:03 AM UTC

    Google has stated that more time is required to address this issue. We deeply appreciate your patience as we work in close collaboration with Google to promptly address and resolve this matter.

  8. resolved Jul 15, 2024, 02:41 PM UTC

    Google has successfully resolved the underlying issue with a comprehensive fix. Thank you for your patience as we worked with Google to deliver the fix.