Hatchet incident

[us-west-2] Increased latency on single shard

Minor Resolved View vendor source →

Hatchet experienced a minor incident on July 29, 2026 affecting API (cloud.hatchet.run) and API (Hatchet API), lasting 31m. The incident has been resolved; the full update timeline is below.

Started
Jul 29, 2026, 07:56 AM UTC
Resolved
Jul 29, 2026, 08:27 AM UTC
Duration
31m
Detected by Pingoru
Jul 29, 2026, 07:56 AM UTC

Affected components

API (cloud.hatchet.run)API (Hatchet API)

Update timeline

  1. investigating Jul 29, 2026, 07:56 AM UTC

    We're monitoring increased latency on a single shared cloud shard in our us-west-2 region experienced elevated database lock contention, which is causing higher API and scheduling latency and some intermittent task-dispatch delays for tenants on that shard. Tasks are continuing to be ingested and assigned.

  2. resolved Jul 29, 2026, 08:27 AM UTC

    We saw elevated database load on our shared production environment beginning around 05:00 UTC, which caused some multi-step workflows to experience scheduling delays and increased API/dashboard latency for affected tenants. We reduced connection pressure on the shared database and latency has returned to normal. We’re continuing to monitor closely.