Lightdash incident

Performance degraded on dashboards and scheduled deliveries

Major Resolved View vendor source →

Lightdash experienced a major incident on February 28, 2024 affecting Lightdash Cloud (US) and Lightdash Cloud (EU), lasting 2h 7m. The incident has been resolved; the full update timeline is below.

Started
Feb 28, 2024, 04:34 PM UTC
Resolved
Feb 28, 2024, 06:42 PM UTC
Duration
2h 7m
Detected by Pingoru
Feb 28, 2024, 04:34 PM UTC

Affected components

Lightdash Cloud (US)Lightdash Cloud (EU)

Update timeline

  1. identified Feb 28, 2024, 04:34 PM UTC

    Dashboards and scheduled deliveries are experiencing degraded performance (>1 minute) load times due to a service outage with our feature flag provider (https://status.posthog.com) We're currently removing any calls to this service to return Lightdash to normal. Please subscribe for updates.

  2. identified Feb 28, 2024, 05:06 PM UTC

    We're currently deploying a fix to all customers, once this is deployed we'll begin monitoring the fix to ensure we've restored services to normal.

  3. identified Feb 28, 2024, 05:09 PM UTC

    We are continuing to work on a fix for this issue.

  4. monitoring Feb 28, 2024, 05:54 PM UTC

    We've deployed a fix to all customers and are monitoring performance closely. Users are reporting normal loading times.

  5. resolved Feb 28, 2024, 06:42 PM UTC

    This incident has been resolved.