Hive incident

Hive API increased error rates

Major Resolved View vendor source →

Hive experienced a major incident on September 13, 2023, lasting —. The incident has been resolved; the full update timeline is below.

Started
Sep 13, 2023, 02:49 PM UTC
Resolved
Aug 29, 2023, 04:00 PM UTC
Duration
Detected by Pingoru
Sep 13, 2023, 02:49 PM UTC

Update timeline

  1. resolved Sep 13, 2023, 02:49 PM UTC

    A heavy resource operation which blocked a subset of servers responding within a 60 second threshold resulted in increased API 500 level responses on a subset of 0.5-1.5% of public API request responses. The issue has been identified and a hotfix resulting in isolation of the issue has been deployed. Furthermore, a configuration change which more quickly quarantines unhealthy servers has been deployed to ensure more immediate quarantine of unhealthy servers.