Red Canary incident
Alert Ingestion Delays Causing Failing Status Checks
Resolved after 3 hours 2 minutes, affecting SentinelOne and Microsoft and 1 more, on February 10, 2025.
We have successfully deployed a fix for the alert ingestion issue and alerts are now being ingested. Detections may still be delayed while we process the backlog of failed alerts. In addition, status checks that appeared to fail during this incident will automatically resolve; no customer action is required. We will continue to monitor the situation and will close this incident once all backlogged alerts have been ingested.
Affected components
Update timeline
- identified Feb 10, 2025, 08:00 PM UTC
We have identified an issue causing some alerts to fail ingestion, which may result in failing status checks for certain customers. Our team is actively testing a fix and assessing its impact. No alert data has been lost and all failed alerts will be reprocessed once the fix is implemented. Please note that detections based on these alerts may be delayed until ingestion is fully restored. We appreciate your patience and will provide further updates as they become available.
- monitoring Feb 10, 2025, 08:41 PM UTC
We have successfully deployed a fix for the alert ingestion issue and alerts are now being ingested. Detections may still be delayed while we process the backlog of failed alerts. In addition, status checks that appeared to fail during this incident will automatically resolve; no customer action is required. We will continue to monitor the situation and will close this incident once all backlogged alerts have been ingested.
- monitoring Feb 10, 2025, 08:42 PM UTC
We have successfully deployed a fix for the alert ingestion issue and alerts are now being ingested. Detections may still be delayed while we process the backlog of failed alerts. In addition, status checks that appeared to fail during this incident will automatically resolve; no customer action is required. We will continue to monitor the situation and will close this incident once all backlogged alerts have been ingested.
- resolved Feb 10, 2025, 11:02 PM UTC
This incident has been resolved and all backlogged alerts processed.
Your product runs on more than Red Canary
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services