Is Hatchet down?

On August 22, 2026, starting around 12:00 UTC, the shared us-west-2 cloud environment experienced elevated database CPU and elevated scheduling latency. A high volume of concurrent workflow runs with stacked concurrency controls on a shared shard caused lock contention and delayed task processing for tenants on that environment. We took action to throttle the work driving saturation and constraining scheduler capacity while the system recovered. Database load and scheduling latency returned to normal later the same afternoon. No data loss occurred.

On August 22, 2026, starting around 12:00 UTC, one shared shard in the us-west-2 cloud environment experienced elevated database CPU and elevated scheduling latency. A high volume of concurrent workflow runs with stacked concurrency controls caused lock contention and delayed task processing for tenants on that shard. Other shared shards in us-west-2, shared environments in other regions, and dedicated customer shards were not affected. Took steps to throttle the workload and reduce contention. Database load and scheduling latency on the affected shard returned to normal later the same afternoon. No data loss occurred.

We're monitoring increased latency on a single shared cloud shard in our us-west-2 region experienced elevated database lock contention, which is causing higher API and scheduling latency and some intermittent task-dispatch delays for tenants on that shard. Tasks are continuing to be ingested and assigned.

We saw elevated database load on our shared production environment beginning around 05:00 UTC, which caused some multi-step workflows to experience scheduling delays and increased API/dashboard latency for affected tenants. We reduced connection pressure on the shared database and latency has returned to normal. We’re continuing to monitor closely.

Between approximately 05:00–10:00 UTC on July 24, the shared cloud shard in our us-west-2 region experienced elevated database lock contention, which caused higher API and scheduling latency and some intermittent task-dispatch delays for tenants on that shard. This was isolated to a single shard in us-west-2 (cloud). Other regions and shards were not affected. We have no evidence of dropped workload, and all assignments were eventually completed. We are continuing to investigate and monitor.