CentralReach experienced a critical incident on December 7, 2021 affecting CentralReach Platform, lasting 20h 26m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Dec 07, 2021, 03:44 PM UTC
Our team has identified a confirmed Amazon Web Services (AWS) issue that is impacting the CentralReach platform and preventing users from logging in. We are monitoring the issue closely and will provide more updates as soon as possible.
- investigating Dec 07, 2021, 04:11 PM UTC
We are continuing to investigate this issue.
- investigating Dec 07, 2021, 04:30 PM UTC
We are continuing to monitor this issue.
- identified Dec 07, 2021, 04:51 PM UTC
Amazon Web Services (AWS) reports they have identified the issue which is affecting the CentralReach Platform and other software providers. AWS states they are actively working toward recovery.
- identified Dec 07, 2021, 05:07 PM UTC
Amazon Web Services (AWS) reports that all teams are engaged and are actively working on addressing the issue; however they do not have an ETA to recovery at this time.
- identified Dec 07, 2021, 05:40 PM UTC
We are continuing to monitor this issue.
- identified Dec 07, 2021, 06:26 PM UTC
Amazon Web Services (AWS) reports that their service team is still engaged and troubleshooting the issue affecting the CentralReach Platform and other software providers, but they do not yet have an ETA for resolution.
- identified Dec 07, 2021, 06:53 PM UTC
We are continuing to monitor this issue. Amazon Web Services (AWS) reports they do not have an estimate for full recovery yet.
- identified Dec 07, 2021, 07:21 PM UTC
Amazon Web Services (AWS) reports that they are continuing to work on a fix for this issue.
- identified Dec 07, 2021, 08:05 PM UTC
Amazon Web Services (AWS) reports they do not have an ETA for full recovery at this time, and that all AWS teams are engaged and actively working on resolving the issue.
- identified Dec 07, 2021, 08:43 PM UTC
We are continuing to monitor this issue.
- identified Dec 07, 2021, 09:00 PM UTC
Amazon Web Services (AWS) reports that they continue to work on a number of different mitigation and resolution actions. They do not have an ETA for full recovery.
- identified Dec 07, 2021, 09:28 PM UTC
We are continuing to monitor this issue.
- identified Dec 07, 2021, 10:05 PM UTC
Amazon Web Services (AWS) reports that they are seeing some services being recovered, but CentralReach and other software providers are still waiting on a resolution to restore access.
- identified Dec 07, 2021, 10:34 PM UTC
Amazon Web Services (AWS) reports that they are now beginning to see significant recovery for AWS service availability. They are continuing to closely monitor the issue, as CentralReach and other software providers are still waiting on a full recovery.
- identified Dec 07, 2021, 11:07 PM UTC
We are continuing to monitor this issue.
- identified Dec 07, 2021, 11:48 PM UTC
We are continuing to monitor this issue. Users may be able to log in, but issues related to the ongoing AWS disruption are continuing. If users choose to log in, please use caution and save frequently.
- identified Dec 08, 2021, 01:01 AM UTC
Amazon Web Services (AWS) reports that the root network device issues are resolved, but they are now working towards recovery of impaired services. CentralReach is continuing to monitor and advises users to be cautious and save frequently if they choose to log in.
- identified Dec 08, 2021, 03:23 AM UTC
Amazon Web Services (AWS) reports that they are still working to bring all impaired services online. CentralReach is continuing to monitor and advises users to be cautious and save frequently if they choose to log in.
- monitoring Dec 08, 2021, 11:33 AM UTC
Amazon Web Services (AWS) has implemented a fix and we are monitoring the results within the CentralReach platform.
- resolved Dec 08, 2021, 12:11 PM UTC
This incident has been resolved.