Bucks County Community College incident

4/20/2023 - Systems Outage - Microsoft 365 Applications

Major Resolved View vendor source →

Bucks County Community College experienced a major incident on April 20, 2023 affecting Microsoft 365 Applications, lasting 4h 41m. The incident has been resolved; the full update timeline is below.

Started
Apr 20, 2023, 03:33 PM UTC
Resolved
Apr 20, 2023, 08:14 PM UTC
Duration
4h 41m
Detected by Pingoru
Apr 20, 2023, 03:33 PM UTC

Affected components

Microsoft 365 Applications

Update timeline

  1. investigating Apr 20, 2023, 03:33 PM UTC

    Microsoft is reporting that some users may be unable to access all or some of the 365 apps. Below is their email *********************************************************************************************** Published Time: 4/20/2023 10:33:55 AM Title: Some users may be unable to view or access Microsoft 365 apps or services User impact: Some users may be unable to view or access Microsoft 365 apps or services More info: Impacted services may include but are not limited to: - Microsoft 365 Online apps - Users may be unable to access Microsoft 365 web apps, such as Excel Online. Additionally, the search bar may not appear in any Office Online service. - Microsoft Teams - Users may be unable to access the Microsoft Teams admin center. - SharePoint Online - Users may be unable to view the settings gear, search bar and waffle. - Microsoft Planner - Users may be unable to access Microsoft 365 web apps through Microsoft Planner. Current status: We determined that a section of caching infrastructure is performing below acceptable performance thresholds, causing calls to gather user licensing information to bypass the cache and go directly to Azure Active Directory infrastructure resulting in high resource utilization, resulting in throttling and impact. We’re analyzing forensics within the caching infrastructure to identify the cause of the issue. Scope of impact: Impact is specific to some users who are served through the affected infrastructure. Start time: Thursday, April 20, 2023, at 12:38 AM UTC ******************************************************************************************** We are monitoring the situation and will report as additional information becomes available. Thanks, Help Desk and Services Center

  2. resolved Apr 20, 2023, 08:14 PM UTC

    Microsoft is reporting the issue with the 365 apps has been resolved. Things appear to have been back for a while but they just issued the all clear. Thanks for your patience, Help Desk and Services Center