Investigating Search Latency

Update timeline

1. investigating Sep 25, 2025, 04:16 PM UTC

   We are currently investigating increased latency in our in-app search (including alert search and incident search). Newly created resources may not be immediately searchable. Search results in-app may be slower to load than normal. Search results may be returned more slowly for certain API endpoints, including: GET /api/v1/incidents, GET /api/v1/services, GET /api/v1/groups, GET /api/v1/users, and GET /api/v1/audits. Our team is actively working on resolving this issue and will provide updates as soon as more information becomes available.

2. monitoring Sep 25, 2025, 04:49 PM UTC

   We identified the root cause of the issue and have implemented a fix. Latencies have returned to normal and we are monitoring them going forward.

3. resolved Sep 25, 2025, 09:13 PM UTC

   Sep 25, 14:13 PDT Resolved - This incident has been resolved. Sep 25, 09:49 PDT Monitoring - We identified the root cause of the issue and have implemented a fix. Latencies have returned to normal and we are monitoring them going forward. Sep 25, 09:16 PDT Investigating - We are currently investigating increased latency in our in-app search (including alert search and incident search). Newly created resources may not be immediately searchable. Search results in-app may be slower to load than normal. Search results may be returned more slowly for certain API endpoints, including: GET /api/v1/incidents, GET /api/v1/services, GET /api/v1/groups, GET /api/v1/users, and GET /api/v1/audits. Our team is actively working on resolving this issue and will provide updates as soon as more information becomes available.

4. resolved Sep 25, 2025, 09:13 PM UTC

   This incident has been resolved.