Chartmetric experienced a major incident on December 24, 2021 affecting Web Application and API, lasting 3h 8m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Dec 24, 2021, 01:45 AM UTC
We're aware of an issue causing certain internal and external API calls to time out or take unusually long and are currently investigating.
- investigating Dec 24, 2021, 01:48 AM UTC
We've localized the issue to our API servers and are continuing to investigate.
- identified Dec 24, 2021, 02:10 AM UTC
We've identified the issue is related to a recent update and are reloading our servers now.
- monitoring Dec 24, 2021, 02:18 AM UTC
Our API servers appear to be responding normally once again. We will continue monitoring to ensure stability.
- identified Dec 24, 2021, 02:25 AM UTC
The previous solution seems to have only worked temoporarily; we're continuing to develop a permanent fix.
- identified Dec 24, 2021, 02:34 AM UTC
We've developed and are in the process of publishing a new patch.
- identified Dec 24, 2021, 02:53 AM UTC
The previous fix was ineffectual and we are working on another. We're certain now that it is a networking issue.
- identified Dec 24, 2021, 03:18 AM UTC
We're preparing to roll back to yesterday's version of the application as a temporary fix so we can investigate this problem more deeply without extending this outage further.
- monitoring Dec 24, 2021, 03:26 AM UTC
The rollback appears to have fully restored functionality. We are continuing to monitor the situation closely to ensure stability.
- resolved Dec 24, 2021, 04:54 AM UTC
We've determined stability has been reestablished.