Memfault incident

Delays in processing data

Major Resolved View vendor source →

Memfault experienced a major incident on July 19, 2024 affecting API & Dashboard and Ingress, lasting 14h 51m. The incident has been resolved; the full update timeline is below.

Started
Jul 19, 2024, 08:29 AM UTC
Resolved
Jul 19, 2024, 11:21 PM UTC
Duration
14h 51m
Detected by Pingoru
Jul 19, 2024, 08:29 AM UTC

Affected components

API & DashboardIngress

Update timeline

  1. investigating Jul 19, 2024, 08:29 AM UTC

    We're experiencing an elevated level of ingestion errors and are currently looking into the issue. The dashboard and API may become unavailable.

  2. investigating Jul 19, 2024, 08:30 AM UTC

    We are continuing to investigate this issue.

  3. identified Jul 19, 2024, 08:58 AM UTC

    The issue has been identified and a fix is being implemented.

  4. identified Jul 19, 2024, 08:59 AM UTC

    We are continuing to work on a fix for this issue.

  5. monitoring Jul 19, 2024, 09:07 AM UTC

    A fix has been implemented and we are monitoring the results.

  6. monitoring Jul 19, 2024, 04:25 PM UTC

    Data ingestion is approximately 30 minutes delayed. We will provide an update in two hours.

  7. monitoring Jul 19, 2024, 06:45 PM UTC

    We're still behind in processing data by about 30 minutes. We're continuing to work through the backlog.

  8. resolved Jul 19, 2024, 11:21 PM UTC

    We've caught up with data processing and this incident has been resolved.