Memfault experienced a major incident on March 13, 2025 affecting API & Dashboard and Ingress, lasting 7h 53m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 13, 2025, 01:09 PM UTC
We're experiencing an elevated level of API errors and are currently looking into the issue. The dashboard and API may become unavailable. We are behind in processing some data but will catch up.
- investigating Mar 13, 2025, 01:12 PM UTC
We are continuing to investigate the issue. The dashboard is failing to load for some users.
- identified Mar 13, 2025, 01:38 PM UTC
We've resolved our database pressure and the app now loads without error. We're working on processing queued ingress data and will work to process the backlog of data we've queued. No data has been lost during this incident.
- monitoring Mar 13, 2025, 03:24 PM UTC
We're back operating normally. We have a backlog of data (particularly Linux MAR files and MCU Trace Events) which we will complete processing in the next few hours. No data has been lost.
- monitoring Mar 13, 2025, 06:10 PM UTC
We're continuing to run normally while we process queued data. All data is up to date other than these two systems: - MCU Trace Events - Linux Memfault Archive files We'll continue to monitor and we'll update when we've processed all backlogged data.
- resolved Mar 13, 2025, 09:03 PM UTC
This incident has been resolved and we've caught up with all data.