Service Unavailability

Update timeline

1. investigating Feb 12, 2026, 02:05 PM UTC

   We are currently experiencing a full service outage. All requests are returning 503 errors. Our engineering team is actively investigating and working to restore service as quickly as possible.

2. investigating Feb 12, 2026, 02:09 PM UTC

   We are currently experiencing a major service disruption. Most requests are returning 503 errors, though limited service is available through a single server. Our engineering team is actively investigating and working to restore full capacity.

3. identified Feb 12, 2026, 02:18 PM UTC

   We are continuing to recover from a service disruption that caused 503 errors. Additional servers have been brought back online and service is improving. Our engineering team is working to restore full capacity.

4. monitoring Feb 12, 2026, 02:26 PM UTC

   Service has been mostly restored following an earlier outage. Some users may experience minor intermittent slowness. Our team continues to monitor as we return to full normal operation.

5. monitoring Feb 12, 2026, 02:46 PM UTC

   Service has stabilised following the earlier outage. Our servers are back online and performing as expected. Our team will continue to monitor the situation.

6. resolved Feb 12, 2026, 03:02 PM UTC

   The earlier service outage has been resolved. Our servers are back online and operating normally. We apologise for the disruption and are taking steps to prevent this from recurring.

7. postmortem Feb 13, 2026, 10:53 AM UTC

   **Date:** 12 February 2026 **Duration:** Approximately 30 minutes \(14:00 - 14:35 UTC\), with a brief secondary disruption during recovery. ### What Happened Pinpoint experienced a service outage beginning at approximately 14:00 UTC on 12 February. Most users were unable to access the platform and received 504 timeout errors, though partial capacity meant some requests were still being served. Full service was restored by approximately 14:35 UTC. Some outbound emails were delayed due to background workers also being affected. All queued emails were delivered by approximately 16:15 UTC. No data was lost or corrupted. ### Timeline \(UTC\) * **14:00** - Automated monitoring alerts triggered for service unavailability. * **14:02** - Engineering team began investigating. * **14:08** - Root cause identified as a network configuration conflict introduced by a third-party infrastructure provider. * **14:20** - Initial fix applied; service stabilising. * **14:22** - Third-party provider's automation reinstated the conflicting configuration, causing a brief secondary disruption. * **14:30** - Fix reapplied with additional safeguards to prevent recurrence. * **14:35** - Full service restored. * **16:15** - All delayed outbound emails delivered. ### Why It Happened A third-party infrastructure provider applied an automated network configuration change to our servers. The change was overly broad and conflicted with our own networking configuration, causing the majority of inbound traffic to be dropped. During initial recovery, the provider's automation re-applied the conflicting change before we implemented stronger protections. ### What We're Doing About It * **Immediate Safeguards:** Protections are in place to prevent this specific change from affecting our infrastructure again. * **Removing the Dependency:** We are accelerating work to bring the affected networking functionality fully in-house, removing the third-party dependency entirely. We sincerely apologise for the disruption and appreciate your patience.