Tyk experienced a critical incident on April 26, 2023 affecting Controller at aws-us-east-1, lasting 2h 59m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Apr 26, 2023, 07:10 AM UTC
A database upgrade in the aws-us-east-1 zone on a third party provider led to the database becoming unhealthy. All of the Control Plane deployments are impacted as are Tyk Cloud capabilities in this zone. We are working with the provider to mitigate this issue. API traffic in this zone should not be affected. Please avoid re-deploying or restarting the Edge deployments in this zone until the incident is resolved.
- identified Apr 26, 2023, 08:28 AM UTC
The database is now healthy and the Control Plane deployments are being gradually restored. We will update once everything is operating normally.
- monitoring Apr 26, 2023, 09:07 AM UTC
All of the Control Plane deployments in aws-us-east-1 are restored. We're monitoring the system while the database provider completes the procedures on their end.
- resolved Apr 26, 2023, 10:09 AM UTC
The database provider has confirmed the successful completion of the restoration process. At this point, there are no observed adverse effects on our side. We will be conducting a postmortem shortly.