Upstash incident

Fly.io Upstash Redis service distruption (FRA region)

Major Resolved View vendor source →

Upstash experienced a major incident on May 11, 2026, lasting 3h 16m. The incident has been resolved; the full update timeline is below.

Started
May 11, 2026, 03:05 PM UTC
Resolved
May 11, 2026, 06:22 PM UTC
Duration
3h 16m
Detected by Pingoru
May 11, 2026, 03:05 PM UTC

Update timeline

  1. investigating May 11, 2026, 03:05 PM UTC

    Some databases may experience increased latency or timeouts in Fly.io’s FRA region.

  2. investigating May 11, 2026, 03:07 PM UTC

    We are continuing to investigate the issue.

  3. investigating May 11, 2026, 05:19 PM UTC

    We are working with Fly team to investigate the root cause.

  4. resolved May 11, 2026, 06:22 PM UTC

    The incident has been resolved. We are working with Fly team on RCA.

  5. postmortem May 15, 2026, 12:49 PM UTC

    On May 12th and 13th at various times, a subset of Upstash Redis instances on [Fly.io](http://Fly.io) experienced intermittent hangs and elevated error rates. The Redis process would stall inside a logging syscall — alive but not making progress — which made the issue hard to spot from our usual telemetry. After investigating with Fly's team, we identified the root cause as a bad interaction between a recent guest kernel update on Fly's newer machines and an upstream Cloud Hypervisor bug \([cloud-hypervisor#7672](https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7672)\) affecting log writes from inside the VM. We mitigated by disabling the affected logging paths, and Fly has since rolled out a hypervisor-side patch, fully resolving the issue. No data was lost. Sorry for the disruption.