Halcyon experienced a major incident on October 24, 2025 affecting REST API and Webhooks and 1 more component, lasting 12h 43m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 24, 2025, 06:44 AM UTC
Halcyon has identified performance degradation in the ingest processing pipeline and is investigating
- identified Oct 24, 2025, 08:17 AM UTC
Root cause has been identified and the team is working to address the problem.
- identified Oct 24, 2025, 08:51 AM UTC
We are continuing to work on a fix for this issue.
- identified Oct 24, 2025, 12:48 PM UTC
Team is continuing to take actions to restore operations. Until then, event ingestion, webhooks/alerting, new device registration, and some API operations in our US region will be severally degraded. Cloud detections for our ransomware protections, Atlas API, are still online.
- monitoring Oct 24, 2025, 01:29 PM UTC
We are currently recovering. Some systems may still be slow or partially inoperable, such as updating webhooks, receiving webhooks, and alerts/events showing in the console.
- monitoring Oct 24, 2025, 02:36 PM UTC
Data is still being processed successfully, but there may be a delay in receiving alert/event webhooks or assets/alerts/events showing in the console or API.
- monitoring Oct 24, 2025, 05:25 PM UTC
We have tracked the cause of the problem and are investigating to prevent it from happening again in the near future. Sending webhooks have been temporarily disabled as we investigate, but will still be sent once we are fully operational.
- monitoring Oct 24, 2025, 05:59 PM UTC
We have re-enabled all services, webhook sending, and the display of alerts/events should be processing again. We are monitoring for any further issues.
- monitoring Oct 24, 2025, 06:40 PM UTC
All services are operational and we are monitoring for any further issues.
- resolved Oct 24, 2025, 07:28 PM UTC
All systems are operational and working as expected.