Apigee incident

Runtime errors in us-west

Major Resolved View vendor source →

Apigee experienced a major incident on April 15, 2023 affecting API Services, lasting 2d 2h. The incident has been resolved; the full update timeline is below.

Started
Apr 15, 2023, 03:58 PM UTC
Resolved
Apr 17, 2023, 06:03 PM UTC
Duration
2d 2h
Detected by Pingoru
Apr 15, 2023, 03:58 PM UTC

Affected components

API Services

Update timeline

  1. identified Apr 15, 2023, 03:58 PM UTC

    We have identified an issue with DNS failures on Apigee Edge in US-West that may result in failed runtime calls. Teams are actively working on remediation.

  2. identified Apr 15, 2023, 05:08 PM UTC

    We are continuing our work to identify the root cause of the issue implement a mitigation. In the meantime, we have failed over customer who have multi-regional setups that we have identified as impacted to their alternate regions.

  3. monitoring Apr 15, 2023, 06:10 PM UTC

    The team has rolled out a mitigation, and we are monitoring recovery in us-west. Once we have confidence the region is stable and the mitigation is completed (expected within the next 2 hours), we will begin re-enabling us-west for customers who have been failed out of that region.

  4. resolved Apr 17, 2023, 06:03 PM UTC

    This issue has been resolved following the mitigation rollout. All DNS resolutions are stable in the us-west region, and runtime issues are confirmed to be resolved.