Civitas Learning experienced a minor incident on May 28, 2024 affecting Degree Map and Data Warehouse (Explore) and 1 more component, lasting 3d. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 28, 2024, 12:52 PM UTC
We are currently investigating an issue with workflows timing out. We will provide an update as soon as we have more details.
- investigating May 28, 2024, 05:32 PM UTC
We are continuing to investigate this issue and will provide another update soon.
- identified May 28, 2024, 09:00 PM UTC
We have identified an issue and currently working on a fix. We will provide another update as soon as we have more details.
- identified May 28, 2024, 11:50 PM UTC
Our team continues to work on a fix. We will provide more updates when they become available.
- identified May 29, 2024, 01:26 PM UTC
We have partially resolved the issue and are continuing to work towards full resolution. We will provide another update once we have more details.
- identified May 29, 2024, 08:48 PM UTC
We are continuing to work on a fix for this issue.
- identified May 30, 2024, 01:15 PM UTC
Fixes put in place have resulted in successful workflows for the majority of the extracts and builds. There are a few builds that are still in process or failed, and we are continuing our troubleshooting efforts to ensure all builds are successful. We will provide an update this afternoon on the status of the few builds that have not succeeded.
- monitoring May 30, 2024, 08:58 PM UTC
A fix has been implemented and we are monitoring the results. We will provide another update tomorrow after workflows complete their normal processing.
- resolved May 31, 2024, 01:21 PM UTC
Workflows are performing as expected. The problem was identified as fault with a load balancer. The issue has been resolved and we've added additional safety measures to capture issues of this type in the future.