Panther Labs experienced a major incident on November 16, 2022 affecting Alert Delivery, lasting 5h 17m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Nov 16, 2022, 08:11 PM UTC
Panther has identified an issue that prevents alerts from being delivered to Slack destinations using webhooks. Our Engineering team has developed a fix for this issue and we plan to deploy it to impacted customer accounts soon. We will update this incident as our investigation and remediation process continues.
- resolved Nov 17, 2022, 01:28 AM UTC
We have deployed the fix for this issue to affected customer accounts. New alerts should now be sent properly to Slack destinations, and the incident is resolved. To check for alerts that failed during this time and manually resend them, please follow the instructions below: 1. Open the Panther Console and navigate to Alerts. 2. Click the name of an alert that shows "Destination Delivery Failed". 3. Next to "Alert delivery failed", click "Show History". 4. Click the retry delivery button.