Mattermost incident

Increased time for new cloud sign-ups

Minor Resolved View vendor source →

Mattermost experienced a minor incident on January 5, 2021 affecting Sign-Up, lasting 2h 11m. The incident has been resolved; the full update timeline is below.

Started
Jan 05, 2021, 06:16 PM UTC
Resolved
Jan 05, 2021, 08:27 PM UTC
Duration
2h 11m
Detected by Pingoru
Jan 05, 2021, 06:16 PM UTC

Affected components

Sign-Up

Update timeline

  1. investigating Jan 05, 2021, 06:16 PM UTC

    We are currently experiencing an increased time for customers signing up to create their cloud workspaces

  2. identified Jan 05, 2021, 07:01 PM UTC

    We have identified the issue, doing some additional checks to confirm it.

  3. monitoring Jan 05, 2021, 08:04 PM UTC

    A fix has been implemented and we will keep monitoring the situation

  4. resolved Jan 05, 2021, 08:27 PM UTC

    The issue has been resolved. A post-mortem will be posted within the next days.

  5. postmortem Jan 08, 2021, 02:19 PM UTC

    An internal post-mortem was conducted and the issue was related to internal DNS resolution. Performance degradation of our internal DNS servers was responsible for slow network responses and increased timeout errors. The issue was resolved and actions are taken to improve system monitoring and avoid similar issues in the future.