- Detected by Pingoru
- May 21, 2026, 11:22 PM UTC
- Resolved
- May 22, 2026, 04:42 PM UTC
- Duration
- 17h 20m
Affected: ui.honeycomb.io - US1 Queryingui.eu1.honeycomb.io - EU1 Querying
Timeline · 3 updates
-
investigating May 21, 2026, 11:22 PM UTC
We have identified and are working to resolve an issue that is causing query results to return inconsistent results.
-
investigating May 22, 2026, 12:49 AM UTC
Impact from this issue has concluded. For a period of time (different between US and EU instances, noted below), queries spanning data older than the most recent 2 hours with certain GROUP / WHERE clauses returned inconsistent results. Data was never lost, and reruns of the same queries will show the previously missing data. US impact times: 21:50 UTC - 23:20 UTC EU impact times: 20:50 UTC - 23:20 UTC
-
resolved May 22, 2026, 04:42 PM UTC
This is an administrative update, marking the incident as Resolved. There has been no further impact since 23:20UTC on May 21 2026.
Read the full incident report →
- Detected by Pingoru
- May 08, 2026, 08:00 AM UTC
- Resolved
- May 11, 2026, 04:37 PM UTC
- Duration
- 3d 8h
Affected: ui.honeycomb.io - US1 Activity Log
Timeline · 7 updates
-
identified May 08, 2026, 08:43 AM UTC
Activity log delay is rising, and is more than an hour behind due to a MySQL replica failover.
-
monitoring May 08, 2026, 10:13 AM UTC
Backfilling of the past 2h30m of data is now in progress and should complete shortly.
-
monitoring May 08, 2026, 10:21 AM UTC
We believe that the activity log should now be caught up.
-
monitoring May 08, 2026, 10:22 AM UTC
We are continuing to monitor for any further issues.
-
monitoring May 08, 2026, 10:40 AM UTC
All activity log streams are now caught up, except for the query runs table, which should have all data since the start of the incident backfilled by 1600 PDT.
-
monitoring May 08, 2026, 08:46 PM UTC
Activity Log recovered at 07:00 UTC-07:00 today (about 6.5h ago). All streams are caught up.
-
resolved May 11, 2026, 04:37 PM UTC
This incident has been resolved.
Read the full incident report →
- Detected by Pingoru
- May 01, 2026, 07:51 PM UTC
- Resolved
- May 01, 2026, 01:00 PM UTC
- Duration
- —
Timeline · 1 update
-
resolved May 01, 2026, 07:51 PM UTC
The database failover that caused query issues earlier in the EU (https://status.honeycomb.io/incidents/n855d8kzp32y) has also had knock-on effects on our Activity Log beta feature. Due to recovery issues on the data replication flow used for this mechanism, activity log events will be missing from 12:45 UTC until 19:45 UTC, representing a 5 hour gap in events. We have identified configuration parameters that will be adjusted to reduce the likelihood of such losses in the future.
Read the full incident report →
- Detected by Pingoru
- May 01, 2026, 01:22 PM UTC
- Resolved
- May 01, 2026, 12:30 PM UTC
- Duration
- —
Timeline · 1 update
-
resolved May 01, 2026, 01:22 PM UTC
At 12:50, our main database underwent an automated failover. This failover led to issues with some of our query engine connection pools, and caused queries to fail for roughly 10 minutes before self resolving.
Read the full incident report →
- Detected by Pingoru
- Apr 30, 2026, 03:03 PM UTC
- Resolved
- Apr 30, 2026, 04:37 PM UTC
- Duration
- 1h 33m
Affected: ui.honeycomb.io - US1 Trigger & SLO Alertingui.eu1.honeycomb.io - EU1 Trigger & SLO Alerting
Timeline · 3 updates
-
identified Apr 30, 2026, 03:03 PM UTC
We discovered that links in Slack triggers are not working and have identified the cause and are in process of implementing a fix
-
monitoring Apr 30, 2026, 03:07 PM UTC
The fix has been implemented and links on Slack triggers are functional again.
-
resolved Apr 30, 2026, 04:37 PM UTC
We are confident the issue is fully resolved.
Read the full incident report →
- Detected by Pingoru
- Apr 20, 2026, 04:56 PM UTC
- Resolved
- Apr 20, 2026, 05:02 PM UTC
- Duration
- 6m
Affected: ui.eu1.honeycomb.io - EU1 Activity Log
Timeline · 2 updates
-
identified Apr 20, 2026, 04:56 PM UTC
Activity Log events are interrupted, starting from around 15:51UTC. Customers may experience some data lost since the beginning of the incident.
-
resolved Apr 20, 2026, 05:02 PM UTC
Activity Log is back to operational, and most events should be recovered.
Read the full incident report →
- Detected by Pingoru
- Apr 15, 2026, 07:04 PM UTC
- Resolved
- Apr 15, 2026, 07:25 PM UTC
- Duration
- 20m
Affected: ui.honeycomb.io - US1 Activity Logui.eu1.honeycomb.io - EU1 Activity Log
Timeline · 2 updates
-
identified Apr 15, 2026, 07:04 PM UTC
Activity Log is experiencing a delay in processing events. We are working on remediating the issue.
-
resolved Apr 15, 2026, 07:25 PM UTC
Activity Log is ingesting events again. We had no loss in data.
Read the full incident report →
- Detected by Pingoru
- Apr 14, 2026, 09:36 PM UTC
- Resolved
- Apr 14, 2026, 10:37 PM UTC
- Duration
- 1h
Affected: ui.eu1.honeycomb.io - EU1 Activity Log
Timeline · 3 updates
-
identified Apr 14, 2026, 09:36 PM UTC
Activity Log events may be delayed, starting from around 19:55UTC. We're actively working on remediation.
-
monitoring Apr 14, 2026, 10:08 PM UTC
A fix has been implemented and we are monitoring recovery.
-
resolved Apr 14, 2026, 10:37 PM UTC
All activity log data is now live and up to date.
Read the full incident report →
- Detected by Pingoru
- Apr 10, 2026, 01:41 PM UTC
- Resolved
- Apr 10, 2026, 05:24 PM UTC
- Duration
- 3h 42m
Affected: api.honeycomb.io - US1 Event Ingest
Timeline · 7 updates
-
identified Apr 10, 2026, 01:41 PM UTC
We have confirmed reports of some teams getting issues ingesting their data. We have identified a probable source behind this behavior and are currently trying to correct it.
-
identified Apr 10, 2026, 01:46 PM UTC
We have further narrowed down that only teams on Classic environments are currently seeing issues. We are figuring out mitigation mechanisms at the moment.
-
identified Apr 10, 2026, 01:55 PM UTC
We've identified a manual fix and are deploying that by hand to each affected environment, while also investigating a systemic solution.
-
monitoring Apr 10, 2026, 02:19 PM UTC
We've applied the manual fix to nearly all affected datasets (less than 1% remaining). The systemic solution is still in progress.
-
monitoring Apr 10, 2026, 02:32 PM UTC
The manual fix has been applied to all environments.
-
monitoring Apr 10, 2026, 03:21 PM UTC
We are continuing to work on the systemic solution and monitor. We'll leave the incident open until we're able to confirm the issue won't recur.
-
resolved Apr 10, 2026, 05:24 PM UTC
We can confirm that the fix was applied uniformly to all impacted and potentially impacted environments; no teams should see ingest errors related to this issue anymore.
Read the full incident report →
- Detected by Pingoru
- Apr 09, 2026, 05:43 PM UTC
- Resolved
- Apr 09, 2026, 07:00 PM UTC
- Duration
- 1h 16m
Timeline · 1 update
-
resolved Apr 09, 2026, 05:43 PM UTC
We have detected and remediated an issue around our SLO processing pipeline. From 17:00 to 17:15 UTC, events were being delayed, and alerts may likewise have been a few minutes late. Processing has caught up and is currently within normal operational ranges.
Read the full incident report →
- Detected by Pingoru
- Apr 09, 2026, 11:52 AM UTC
- Resolved
- Apr 10, 2026, 07:10 PM UTC
- Duration
- 1d 7h
Affected: ui.honeycomb.io - US1 Trigger & SLO Alertingui.honeycomb.io - US1 Activity Log
Timeline · 6 updates
-
identified Apr 09, 2026, 11:52 AM UTC
We have identified the cause and are working to resolve the issue.
-
identified Apr 09, 2026, 08:24 PM UTC
We have identified an issue with triggers and SLOs and are actively investigating. Customers may experience delays when creating or updating triggers and SLOs. A fix for the activity log delay has been implemented and is in the process of recovering. Customers may continue to see activity log unavailability while this completes.
-
identified Apr 09, 2026, 09:03 PM UTC
We have identified an issue with triggers and SLOs and have implemented a fix. Customers should no longer see degraded performance for triggers or SLOs. A fix for the activity log delay has been implemented and is in the process of recovering. Customers may continue to see activity log unavailability while this completes.
-
identified Apr 09, 2026, 11:43 PM UTC
SLOs experienced a period of additional processing delays. A fix has been deployed and recovery is ongoing.
-
identified Apr 10, 2026, 12:05 AM UTC
SLO processing has fully recovered.
-
resolved Apr 10, 2026, 07:10 PM UTC
All activity log data is now live and up to date.
Read the full incident report →
- Detected by Pingoru
- Apr 09, 2026, 10:12 AM UTC
- Resolved
- Apr 09, 2026, 10:12 AM UTC
- Duration
- —
Affected: ui.honeycomb.io - US1 Queryingui.honeycomb.io - US1 Trigger & SLO Alerting
Timeline · 1 update
-
resolved Apr 09, 2026, 10:12 AM UTC
We experienced a partial querying outage from approximately 8:58 - 9:18 UTC. This outage may have resulted in some Trigger and SLO alert failures for the duration of the outage. The cause of the outage has been identified and mitigated.
Read the full incident report →
- Detected by Pingoru
- Mar 18, 2026, 06:07 PM UTC
- Resolved
- Mar 18, 2026, 06:18 PM UTC
- Duration
- 11m
Affected: ui.honeycomb.io - US1 App Interface
Timeline · 3 updates
-
investigating Mar 18, 2026, 06:07 PM UTC
We are currently investigating this issue.
-
identified Mar 18, 2026, 06:10 PM UTC
We have identified the issue and are working on a fix.
-
resolved Mar 18, 2026, 06:18 PM UTC
The issue has been identified and we have shipped a fix.
Read the full incident report →
- Detected by Pingoru
- Mar 12, 2026, 11:55 AM UTC
- Resolved
- Mar 12, 2026, 01:39 PM UTC
- Duration
- 1h 43m
Affected: ui.honeycomb.io - US1 Queryingui.honeycomb.io - US1 Trigger & SLO Alerting
Timeline · 3 updates
-
identified Mar 12, 2026, 11:55 AM UTC
We are investigating elevated error rates affecting queries in our US1 region. Our team is actively working on a resolution. We will provide updates as we learn more.
-
identified Mar 12, 2026, 01:17 PM UTC
We have rolled back a code change to our query engine, but our triggers service has not recovered. We are investigating.
-
resolved Mar 12, 2026, 01:39 PM UTC
This incident has been resolved.
Read the full incident report →
- Detected by Pingoru
- Mar 10, 2026, 01:50 PM UTC
- Resolved
- Mar 10, 2026, 02:14 PM UTC
- Duration
- 24m
Affected: api.honeycomb.io - US1 Event Ingest
Timeline · 3 updates
-
investigating Mar 10, 2026, 01:50 PM UTC
We are currently seeing elevated errors in connecting to some of our databases, which results in low amounts of Ingest traffic being rejected. We are looking into correcting the situation.
-
monitoring Mar 10, 2026, 01:56 PM UTC
We have stabilized the cluster and are monitoring the state while investigating to understand what triggered the degradation. Overall, 0.04% traffic was rejected with an error due to this degradation during the 20 minute window we were impaired.
-
resolved Mar 10, 2026, 02:14 PM UTC
This incident is now resolved. We are also noting that some other parts of honeycomb (the query UI) may have seen spurious failures as well during the impacted period (13:22—13:44 UTC).
Read the full incident report →
- Detected by Pingoru
- Feb 18, 2026, 11:07 PM UTC
- Resolved
- Feb 18, 2026, 11:07 PM UTC
- Duration
- —
Affected: ui.eu1.honeycomb.io - EU1 Querying
Timeline · 1 update
-
resolved Feb 18, 2026, 11:07 PM UTC
Between 22:10 and 22:40 UTC on Feb18, 2026 Honeycomb experienced a partial query outage in our EU environment. The incident has been resolved, and we are continuing to investigate the contributing factors.
Read the full incident report →
- Detected by Pingoru
- Feb 10, 2026, 08:18 PM UTC
- Resolved
- Feb 10, 2026, 08:30 PM UTC
- Duration
- 11m
Affected: ui.honeycomb.io - US1 Queryingui.eu1.honeycomb.io - EU1 Querying
Timeline · 2 updates
-
investigating Feb 10, 2026, 08:18 PM UTC
We are aware of elevated errors in trigger evaluations and queries, and are working to resolve it
-
resolved Feb 10, 2026, 08:30 PM UTC
This incident has been resolved.
Read the full incident report →
- Detected by Pingoru
- Jan 22, 2026, 11:27 PM UTC
- Resolved
- Jan 22, 2026, 11:42 PM UTC
- Duration
- 15m
Affected: ui.honeycomb.io - US1 Queryingui.honeycomb.io - US1 Trigger & SLO Alerting
Timeline · 2 updates
-
investigating Jan 22, 2026, 11:27 PM UTC
We are currently investigating what appears to be a transient issue related to querying and triggers
-
resolved Jan 22, 2026, 11:42 PM UTC
This appears to have been a minor transient network issue; we do not expect this to recur
Read the full incident report →
- Detected by Pingoru
- Jan 15, 2026, 03:46 PM UTC
- Resolved
- Jan 15, 2026, 03:46 PM UTC
- Duration
- —
Timeline · 1 update
-
resolved Jan 15, 2026, 03:46 PM UTC
We observed a period of increased latency accompanied by a higher-than-normal error rate in some event ingest requests in the US. The service has since stabilized and is operating normally. We are continuing to monitor the system and are evaluating improvements to reduce the impact of similar conditions in the future. We apologize for the disruption and appreciate your patience.
Read the full incident report →
- Detected by Pingoru
- Jan 06, 2026, 10:14 PM UTC
- Resolved
- Jan 07, 2026, 07:54 PM UTC
- Duration
- 21h 40m
Affected: ui.honeycomb.io - US1 Queryingui.honeycomb.io - US1 Activity Logui.eu1.honeycomb.io - EU1 Queryingui.eu1.honeycomb.io - EU1 Activity Log
Timeline · 6 updates
-
identified Jan 06, 2026, 10:14 PM UTC
We are seeing a spike in querying errors and the Activity Log has delayed ingest of new activity. We have identified the issue and are working to resolve
-
identified Jan 06, 2026, 10:37 PM UTC
We have resolved the issue which impacted querying but are still working to get Activity Log up to date
-
identified Jan 06, 2026, 11:05 PM UTC
We are currently expecting to lose the last three hours of data for the beta activity log feature as part of remediation of this issue.
-
identified Jan 07, 2026, 01:17 AM UTC
We have restored basic activity log functionality, though some delays remain, and we will analyze further tomorrow during business hours.
-
identified Jan 07, 2026, 07:53 PM UTC
We have stabilized activity log. We have confirmed activity log entries lost from 4:54pm-7:04pm ET in the US and 9:43pm-11:55pm GMT in the EU. Event ingest was unimpacted. All systems are currently operating normally
-
resolved Jan 07, 2026, 07:54 PM UTC
This incident has been resolved.
Read the full incident report →
- Detected by Pingoru
- Dec 05, 2025, 08:49 PM UTC
- Resolved
- Dec 17, 2025, 03:47 PM UTC
- Duration
- 11d 18h
Affected: api.eu1.honeycomb.io - EU1 Event Ingestui.eu1.honeycomb.io - EU1 Queryingui.eu1.honeycomb.io - EU1 Trigger & SLO Alertingui.eu1.honeycomb.io - EU1 Activity Log
Timeline · 29 updates
Read the full incident report →
- Detected by Pingoru
- Nov 25, 2025, 10:51 PM UTC
- Resolved
- Nov 27, 2025, 12:14 AM UTC
- Duration
- 1d 1h
Affected: ui.honeycomb.io - US1 Activity Log
Timeline · 5 updates
-
identified Nov 25, 2025, 10:51 PM UTC
We have identified the cause and are working to resolve the issue.
-
identified Nov 26, 2025, 05:02 AM UTC
We are continuing to work on a fix for this issue.
-
monitoring Nov 26, 2025, 05:07 AM UTC
Due to issues with replication streams, the activity log will have a gap in its data. We estimate that a little over 6 hours of content will be missing, from 20:27 UTC until 2:43 UTC. We do not expect missing data to be backfilled. No other Honeycomb data or feature has been impacted. We are currently working on catching up on late data after this window.
-
monitoring Nov 26, 2025, 05:24 PM UTC
We are continuing to work on catching up late data from yesterday night's replication issues and expect to complete this process in approximately 6 hours.
-
resolved Nov 27, 2025, 12:14 AM UTC
As of 23:00 UTC, all systems are back to synchronized.
Read the full incident report →
- Detected by Pingoru
- Oct 20, 2025, 02:16 PM UTC
- Resolved
- Oct 21, 2025, 03:41 PM UTC
- Duration
- 1d 1h
Affected: ui.honeycomb.io - US1 Queryingui.honeycomb.io - US1 Trigger & SLO Alerting
Timeline · 8 updates
-
monitoring Oct 20, 2025, 02:16 PM UTC
Due to constrained EC2 instance capacity in the AWS us-east-1 region, we are choosing to allocate the capacity we have to incoming events and telemetry. As such, customers may see over 5 minute delays in our processing of - SLOs - Service Maps We do not expect a degradation of our core ingest -> query flow, and we do not expect triggers to be impacted
-
monitoring Oct 20, 2025, 03:14 PM UTC
SLO processing has recovered. Service Maps continues to be degraded but historical data can be queried.
-
monitoring Oct 20, 2025, 03:30 PM UTC
The networking issues in us-east-1 is affecting our query engine - customers may see errors or delays when running queries as a result.
-
monitoring Oct 20, 2025, 05:39 PM UTC
We’re currently observing that querying is seeing signs of recovery. We are continuing to monitor the situation
-
monitoring Oct 20, 2025, 08:06 PM UTC
AWS is starting to show signs of recovery. Querying remains partially impacted and may take longer to return results. We are continuing to monitor the situation.
-
monitoring Oct 20, 2025, 11:08 PM UTC
Service maps are up and running and errors have resolved. Querying, triggers and SLOs are continuing to improve and customers may see delays as functionality continues to improve. We are continuing to monitor the situation.
-
monitoring Oct 21, 2025, 12:11 AM UTC
Honeycomb core functionality is operational. Service maps are up and running and errors have resolved. Querying, triggers and SLOs are continuing to improve and customers may see delays as functionality continues to improve. This is our final update for the night unless the situation degrades. We will continue to monitor the situation.
-
resolved Oct 21, 2025, 03:41 PM UTC
This incident has been resolved. At no point did we lose any customer data that hit our load balancers. Querying has been stable for hours, and all features that were degraded are functional.
Read the full incident report →
- Detected by Pingoru
- Oct 20, 2025, 07:24 AM UTC
- Resolved
- Oct 20, 2025, 10:04 AM UTC
- Duration
- 2h 40m
Affected: ui.honeycomb.io - US1 Queryingui.honeycomb.io - US1 Trigger & SLO Alerting
Timeline · 6 updates
Read the full incident report →
- Detected by Pingoru
- Oct 08, 2025, 02:26 PM UTC
- Resolved
- Oct 08, 2025, 03:24 PM UTC
- Duration
- 57m
Timeline · 3 updates
-
identified Oct 08, 2025, 02:26 PM UTC
We are currently experiencing issues with our configuration with one of our AI providers. Features such as Query Assistant and our Calculated Field assistant are not functional, and some other parts of Honeycomb Intelligence may be partially degraded at this moment.
-
monitoring Oct 08, 2025, 03:10 PM UTC
We have made changes to our configuration and errors rates seem to have gone back to normal. We are keeping an eye on things to know the features remain stable.
-
resolved Oct 08, 2025, 03:24 PM UTC
The system has remained stable, we do believe everything to now be in order.
Read the full incident report →
