CSIU experienced a major incident on November 16, 2020, lasting 20d 15h. The incident has been resolved; the full update timeline is below.
Update timeline
- investigating Nov 16, 2020, 11:35 PM UTC
We are aware of an issue affecting some Student Information System users. We are now investigating this issue and will provide a status update as soon as we are able to determine a cause.
- monitoring Nov 17, 2020, 01:01 AM UTC
A fix has been implemented to migrate the affected resources to redundant hardware and we are monitoring the results. Some underlying issues with a host server have been encountered and we have a case open with the vendor to get to the bottom of it. We will update when we feel confident it is fully resolved.
- investigating Nov 17, 2020, 01:58 PM UTC
We are aware that the issue has occurred again and are investigating. We will provide an update as soon as we are able to determine the cause.
- monitoring Nov 17, 2020, 03:39 PM UTC
We are back to a operational state for all customers. We apologize for the inconvenience as we continue to investigate the underlying cause. More information will be provided as we are able.
- monitoring Nov 20, 2020, 03:55 PM UTC
We have continued to operate with no further disruptions over the past three days. We continue to dialog with our vendors to get a final resolution and will update once more when we feel confident this is fully resolved.
- resolved Dec 07, 2020, 02:49 PM UTC
This incident has been resolved.
- postmortem Dec 07, 2020, 02:49 PM UTC
The issue was triggered by a problem with the server replication product used by the CSIU, which caused one of the SQL Servers to become unresponsive. After detailed consultation with our hardware and software vendors, we feel confident that the conditions that caused this issue have been resolved. Thanks for your patience while we resolved this issue.