Teem incident

S2 - Users unable to sign into Teem via Outlook Plugin

Major Resolved View vendor source →

Teem experienced a major incident on May 9, 2023 affecting Exchange Sync, lasting 9h 47m. The incident has been resolved; the full update timeline is below.

Started
May 09, 2023, 04:58 PM UTC
Resolved
May 10, 2023, 02:45 AM UTC
Duration
9h 47m
Detected by Pingoru
May 09, 2023, 04:58 PM UTC

Affected components

Exchange Sync

Update timeline

  1. investigating May 09, 2023, 04:58 PM UTC

    We are currently investigating an issue where users aren't able to access Teem through the Outlook Plugin. We are asking that you please use app.teem.com to access the application for now. We will provide another update at 1pm MST.

  2. investigating May 09, 2023, 07:33 PM UTC

    We are continuing to investigate this issue.

  3. investigating May 09, 2023, 09:46 PM UTC

    Our engineering team continues to investigate this issue and are working on finding the root cause. We appreciate your patience.

  4. monitoring May 09, 2023, 10:01 PM UTC

    The issue with Users unable to sign into Teem via Outlook Plugin has been identified and a fix has been implemented. We are moving into a Monitoring phase for the next couple of hours. We will provide another update at 6pm MST. Thank you for your continued patience as we worked to resolved this issue.

  5. monitoring May 09, 2023, 10:01 PM UTC

    We are continuing to monitor for any further issues.

  6. resolved May 10, 2023, 02:45 AM UTC

    This incident has been resolved.

  7. postmortem May 23, 2023, 06:41 PM UTC

    **Teem Detailed Root Cause Analysis – Severity 2 – May 9, 2023** **Unable to Access Teem via Outlook Plugin** **Description:** On May 9, 2023, multiple customers began reporting the inability to access Teem through the Outlook Plugin. When trying to access Teem through the plugin, customers were presented with the following error, “"Unfortunately, we were unable to associate your external account with Teem. Contact your administrator for help." **Type of Event:** Service Disruption **Services\\Modules Impacted:** Production **Timeline:** On May 9, 2023, at 9am, MST an initial report was submitted to customer support about the inability to access Teem through the Outlook Plugin. Around the 11 o’clock hour, additional customers began to report the same issue and all customers were notified that we are currently investigating the issue via Status Page. Around 4:01pm MST, the issue was identified and implemented by our engineering team and our Status Page was updated to a Monitoring Phase. Customers began to confirm the resolution and there were no additional reports to Teem Support. Our Status Page was updated from Monitoring to Resolved at 8:45pm, MST. **Total Duration of Event:** 11hrs 45mins **Root Cause Analysis:** The team identified an internal process that involved certificate updates. **Preventative Action:** Internal teams are actively researching preventative analysis. However, if this issue happens in the short term, another internal process has been put in place to quickly resolve this issue.