Pinpoint incident

Production Service Disruption — Database Storage

Critical Resolved View vendor source →

Pinpoint experienced a critical incident on May 13, 2026 affecting Pinpoint, lasting 35m. The incident has been resolved; the full update timeline is below.

Started
May 13, 2026, 06:29 AM UTC
Resolved
May 13, 2026, 07:04 AM UTC
Duration
35m
Detected by Pingoru
May 13, 2026, 06:29 AM UTC

Affected components

Pinpoint

Update timeline

  1. investigating May 13, 2026, 06:29 AM UTC

    We are currently experiencing a service disruption caused by an issue with our primary database. While users are still able to log in and view data in Pinpoint, actions that involve saving or updating information are temporarily unavailable. Our team has identified the root cause and is actively working to restore full functionality. We will provide further updates as the situation progresses.

  2. monitoring May 13, 2026, 06:54 AM UTC

    A fix has been implemented and we are monitoring the results.

  3. monitoring May 13, 2026, 06:58 AM UTC

    We are continuing to monitor for any further issues.

  4. resolved May 13, 2026, 07:04 AM UTC

    This incident has been resolved.

  5. postmortem May 13, 2026, 02:16 PM UTC

    **Duration:** Approximately 4 hours of intermittent disruption \(03:00 – 06:51 UTC\), with two distinct impact windows. ## What Happened Pinpoint experienced an unexpected service disruption in the early hours of 13 May. The incident had two impact windows: an initial period of unavailability between approximately 03:00 and 04:00 UTC, and a follow-on period between approximately 05:39 and 06:51 UTC during which users attempting to modify certain types of records \(predominantly scorecard submissions\) encountered errors. No data was lost or corrupted, and no background jobs were delayed. ## Timeline \(UTC\) * 03:00 Initial impact begins; application unavailable. * 04:00 Initial impact resolves; engineering team investigating. * 05:39 Second impact window begins; errors on a subset of record updates. * 06:29 Status page warning posted. * 06:51 Second impact window resolves. * 07:04 Incident marked as resolved. ## Why It Happened The disruption was caused by an unexpected interaction between a routine internal database operation and an active background process. This combination of events is an outlier we had not previously encountered or anticipated. The root cause has been fully investigated and is understood. ## What We're Doing About It * **Additional Monitoring** We have put additional monitoring in place to detect this specific class of interaction much earlier, before it can cause customer impact. * **Increased Database Capacity** We have provisioned additional database capacity to further reduce the likelihood of recurrence. _We sincerely apologise for the disruption and appreciate your patience._