Cronofy experienced a minor incident on June 13, 2023 affecting Scheduler and API and 1 more component, lasting 1h 49m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- monitoring Jun 13, 2023, 07:44 PM UTC
We have seen elevated errors in our US data center, these appear to be related to an open issue in AWS us-east-1: https://health.aws.amazon.com/health/home#/account/dashboard/open-issues?eventID=arn:aws:health:us-east-1::event/MULTIPLE_SERVICES/AWS_MULTIPLE_SERVICES_OPERATIONAL_ISSUE/AWS_MULTIPLE_SERVICES_OPERATIONAL_ISSUE_798F8_DDDD18AFADD&eventTab=details The impact appears low at present, but given it is at the infrastructure level it could quickly become significant and so we have opened this incident.
- monitoring Jun 13, 2023, 08:05 PM UTC
The impact on service in our US data center still appears to be minimal. AWS are resolving the underlying issue in us-east-1 and we will continue to monitor until their incident is closed.
- monitoring Jun 13, 2023, 08:41 PM UTC
The issue with AWS Security Token Service in us-east-1 has become worse for roughly the past 15 minutes. This is mainly impacting scheduling tasks from triggering as they are not able to authenticate properly within the US data center to perform their tasks. This will lead to degraded service in tasks such as the polling of Apple calendars. The impact of this has been minimized for core services as we prevented them from scaling in earlier in the incident.
- monitoring Jun 13, 2023, 09:09 PM UTC
AWS are reporting many services as fully recovered in us-east-1 and we have observed stable service in our US data center since around 21:45 UTC. We will continue to monitor but expect to resolve this incident soon.
- resolved Jun 13, 2023, 09:33 PM UTC
AWS's us-east-1 region, where our US data center is hosted, experienced an issue affecting some services Cronofy's platform relies upon. The impact to service was low, at its peak resulting in a small degradation in performance within our US data center and a handful of server errors being returned by the service. This incident has been resolved.