Technolutions incident
Service interruption to databases on ARES
Technolutions experienced a notice incident on November 18, 2022 affecting Slate, lasting 39m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 18, 2022, 07:03 PM UTC
We are currently investigating this issue.
- investigating Nov 18, 2022, 07:21 PM UTC
Normal operations have been restored to databases on the ARES cluster. We continue to investigate the underlying cause of the earlier interruption and will provide an update in the near future.
- monitoring Nov 18, 2022, 07:21 PM UTC
Earlier this afternoon, at approximately 1:30pm Eastern Time, the ARES database cluster lost connectivity to its disks as a result of an intermittent network interruption in the AWS infrastructure. This resulted in databases on this cluster being unavailable while network connectivity was restored. All databases are operating within normal ranges at this time, and we are continuing to monitor the ARES cluster. Due to the ARES cluster hosting a primary configuration database for Slate, this event may have also impacted some operations with databases on other clusters. These other clusters already contain replicas of the configuration data, and we will be accelerating our scheduled plan to utilize these replicas more fully to minimize any potential for future impact.
- resolved Nov 19, 2022, 06:16 AM UTC
This incident has been resolved.