Honeycomb.io incident

Querying issues

Minor Resolved View vendor source →

Honeycomb.io experienced a minor incident on January 21, 2025 affecting ui.honeycomb.io - US1 Querying, lasting 1d 2h. The incident has been resolved; the full update timeline is below.

Started
Jan 21, 2025, 10:10 PM UTC
Resolved
Jan 23, 2025, 12:21 AM UTC
Duration
1d 2h
Detected by Pingoru
Jan 21, 2025, 10:10 PM UTC

Affected components

ui.honeycomb.io - US1 Querying

Update timeline

  1. investigating Jan 21, 2025, 10:10 PM UTC

    Some US customers are seeing queries slow to succeed and/or failing. We are investigating the cause of the slowness and the query failures.

  2. investigating Jan 22, 2025, 01:31 AM UTC

    We are still investigating a transient failure in our querying system; We have observed no lost data at this time, but querying continues to be impacted for a subset of users

  3. investigating Jan 22, 2025, 02:09 AM UTC

    We are still investigating a transient failure in our querying system; We continue to see no lost data, and querying should be back for all previously affected users.

  4. identified Jan 22, 2025, 03:54 PM UTC

    We have identified a fix for the querying issues observed yesterday as well as their underlying cause, and are currently executing it. The system is presently stable and usable.

  5. monitoring Jan 22, 2025, 09:54 PM UTC

    We've pushed out a fix to relevant query subsystems. We're monitoring and will continue to do so, but we are fully functional.

  6. resolved Jan 23, 2025, 12:21 AM UTC

    This incident has been resolved.