gaiia software incident
Intermittent connectivity issue from AWS Cognito
gaiia software experienced a major incident on December 13, 2024 affecting Customer Portal, lasting 53m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- monitoring Dec 13, 2024, 03:14 AM UTC
We are seeing intermittent connectivity issues from AWS, specifically from the Cognito web service. This can impact end-customers' ability to log into their Client Portal. AWS has declared the issue on their status page and are investigating. https://health.aws.amazon.com/health/status
- monitoring Dec 13, 2024, 03:24 AM UTC
AWS has identified the root cause: "Dec 12 7:17 PM PST We can confirm increased error rates for Amazon Cognito in the US-EAST-1 Region. We believe we have isolated the root cause to one of two potential issues. We are actively working in parallel to mitigate the issue, while continuing to verify the root cause. During this time, customers may be experiencing 400 Errors, receiving the error 'TooManyRequestsException'. We will provide an update within 30 minutes, or sooner if we have additional information to share. At this stage, we expect to see signs of recovery within the next 60 minutes."
- monitoring Dec 13, 2024, 03:45 AM UTC
Update from AWS: "Dec 12 7:37 PM PST We are beginning to see signs of recovery. Customers may continue to receive ‘TooManyRequestsException“ while we work toward full resolution. We recommend continuing to retry any failed requests and will provide more information shortly. We expect full recovery within the next 30 minutes."
- monitoring Dec 13, 2024, 04:07 AM UTC
AWS Cognito is now fully operational, and the Client Portal is functioning normally.
- resolved Dec 13, 2024, 04:08 AM UTC
This incident has been resolved.