Rocket.Chat incident

EUC workspaces inaccessible

Minor Resolved View vendor source →

Rocket.Chat experienced a minor incident on September 12, 2023 affecting Rocket.Chat, lasting 9h 32m. The incident has been resolved; the full update timeline is below.

Started
Sep 12, 2023, 07:29 AM UTC
Resolved
Sep 12, 2023, 05:01 PM UTC
Duration
9h 32m
Detected by Pingoru
Sep 12, 2023, 07:29 AM UTC

Affected components

Rocket.Chat

Update timeline

  1. investigating Sep 12, 2023, 07:29 AM UTC

    Initial status of Investigating

  2. identified Sep 12, 2023, 08:42 AM UTC

    DB outage, now recovering

  3. identified Sep 12, 2023, 10:05 AM UTC

    Some issues with Kubernetes cluster

  4. monitoring Sep 12, 2023, 10:59 AM UTC

    Workspaces recovering

  5. monitoring Sep 12, 2023, 11:27 AM UTC

    Suspending all trials while recovering customers

  6. monitoring Sep 12, 2023, 11:47 AM UTC

    All customers recovered and are fully operational

  7. resolved Sep 12, 2023, 05:01 PM UTC

    All customer workspaces are fully operational