Sardine incident

Degraded API Latency

Resolved after 1 hour 5 minutes, on May 22, 2026.

postmortem May 26, 2026, 10:56 PM UTC

#### Summary Our cache service experienced intermittent latency issues on May 22, 2026. Service has been fully restored. #### What Happened Our cache infrastructure experienced three distinct periods of elevated latency: * 22:23-22:31 UTC * 22:46-22:52 UTC * 23:00-23:05 UTC The system partially recovered between incidents but experienced cascading failures before full restoration at 23:05 UTC #### Why It Happened The incident began with an unusual spike in cache sets and cache deletes that stressed the caching infrastructure. #### What We're Doing About It * Implementing infrastructure improvements to prevent similar incidents resulting from noisy neighbor pattern * Enhancing cache monitoring, alerting and run books * Working with SME’s from our cloud provider to identify and address any contributing factors We apologize for the disruption and appreciate your patience.

Started
May 22, 2026, 10:00 PM UTC
Resolved
May 22, 2026, 11:05 PM UTC
Duration
1h 5m
Detected by Pingoru
May 22, 2026, 10:00 PM UTC

Update timeline

  1. resolved May 22, 2026, 11:31 PM UTC

    From 10:22 UTC to to 11:05 UTC elevated latency could have affected some API requests.

  2. postmortem May 26, 2026, 10:56 PM UTC

    #### Summary Our cache service experienced intermittent latency issues on May 22, 2026. Service has been fully restored. #### What Happened Our cache infrastructure experienced three distinct periods of elevated latency: * 22:23-22:31 UTC * 22:46-22:52 UTC * 23:00-23:05 UTC The system partially recovered between incidents but experienced cascading failures before full restoration at 23:05 UTC #### Why It Happened The incident began with an unusual spike in cache sets and cache deletes that stressed the caching infrastructure. #### What We're Doing About It * Implementing infrastructure improvements to prevent similar incidents resulting from noisy neighbor pattern * Enhancing cache monitoring, alerting and run books * Working with SME’s from our cloud provider to identify and address any contributing factors We apologize for the disruption and appreciate your patience.

Your product runs on more than Sardine

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →