Owens CC incident

Employee and Student access to College Systems

Major Resolved View vendor source →

Owens CC experienced a major incident on January 30, 2023, lasting 13h 17m. The incident has been resolved; the full update timeline is below.

Started
Jan 30, 2023, 10:23 PM UTC
Resolved
Jan 31, 2023, 11:41 AM UTC
Duration
13h 17m
Detected by Pingoru
Jan 30, 2023, 10:23 PM UTC

Update timeline

  1. identified Jan 30, 2023, 10:23 PM UTC

    We are experiencing an issue impacting several College Systems like VPN, Ozone, Intranet and Office 365 applications including Employee Email. Blackboard Learn and MS Teams are not affected by this outage. ITS team is working on resolving the issue and hopes to have services restored later tonight (1/30/2023). If you have any questions regarding this unexpected outage, please contact the Owens Community College Help Desk at 567.661.7120.

  2. monitoring Jan 30, 2023, 11:33 PM UTC

    We have corrected the issue with employee access and are watching the correction while it processes through our student population. As soon as we know more we will update.

  3. monitoring Jan 30, 2023, 11:52 PM UTC

    We expect all students to be corrected by 10:00pm tonight. We will continue to monitor the event.

  4. resolved Jan 31, 2023, 11:41 AM UTC

    This incident has been resolved.

  5. postmortem Feb 03, 2023, 01:30 PM UTC

    In the afternoon of Monday, January 30th Owens experienced an issue that resulted in the removal of the security roles assigned to Student, Faculty, Employee and Owens Associates. As these roles were removed, access granted based on these roles was lost. Microsoft’s O365 licensing is based on these roles and access to email, Teams, etc. was removed. Since our email accounts were unlicensed, inbound emails received an undeliverable message from Microsoft as the accounts were suspended and no email rules would process. Our Owens Service Desk also was impacted by this event as many projects such as; Access Store, S.P.A.C.E. Committee, ITS Project Request and Facilities Maintenance Requests are limited to Employee and Owens Associates. Any website access grant by the security role, such as the access to intranet, Ozone cards, etc. would have been blocked as well. Once the cause was identified, we were able to start corrective actions at 6:30pm Monday night. Microsoft licensing was processed by midnight and all accounts were relicensed at that time. We had to work with our vendor Atlassian to finalize the correction with the Owens Service Desk. Access was restored Thursday, February 2nd at 10:00pm. As of Friday, February 3rd any impact from this event has been corrected.