Search errors for runway-served stores (18:00–18:05 UTC)
Timeline · 1 update
- resolved Sep 25, 2026, 06:00 PM UTC
Type: Incident Duration: 5 minutes Affected Components: Collections / Search Sep 25, 18:00:33 GMT+0 - Investigating - Search requests for stores served by our runway search API are returning an error instead of results. We are investigating. Sep 25, 18:05:15 GMT+0 - Resolved - Between 18:00 and 18:05 UTC, search requests for stores served by our runway search API returned an error instead of results. Cause: during clean-up of a decommissioned cluster, a DNS record that the search service still used to reach its search index was removed automatically by an operator on the old cluster. The record was re-created at 18:05 UTC and search recovered immediately. Stores on the realtime API were not affected. We are moving the search service onto an internal path so a DNS change cannot affect it again, and adding paging to the synthetic check that detected this.