Metabase incident

Issue with Metabase Cloud

Critical Resolved View vendor source →

Metabase experienced a critical incident on November 17, 2022 affecting Metabase Cloud Platform, lasting 26m. The incident has been resolved; the full update timeline is below.

Started
Nov 17, 2022, 06:47 PM UTC
Resolved
Nov 17, 2022, 07:14 PM UTC
Duration
26m
Detected by Pingoru
Nov 17, 2022, 06:47 PM UTC

Affected components

Metabase Cloud Platform

Update timeline

  1. investigating Nov 17, 2022, 06:47 PM UTC

    We identified an issue with our Cloud. We're currently investigating

  2. resolved Nov 17, 2022, 07:14 PM UTC

    We have restored the service. We will provide a description of the issue in the next few days.

  3. postmortem Nov 28, 2022, 11:55 PM UTC

    While provisioning new K8S clusters in our AWS environment, DNS and load balancer related configurations were accidentally overwritten with wrong parameters, preventing ingress network access to all Metabase Cloud instances. We have restored the correct configurations and added additional process controls and automated checks to prevent this from happening in the future.