Dead Man's Snitch Outage History

Started Nov 22, 2025, 05:12 PM UTC· resolved Nov 23, 2025, 12:03 AM UTC

1. investigating Nov 22, 2025, 05:12 PM UTC

   We are experiencing spurious timeouts receiving check-ins for Snitches. The root cause is related to maintenance a service provider is performing on our backend message bus. You may see occasional timeouts though those check-ins should still be processed normally. We are currently investigating a way to work around the timeouts.

2. monitoring Nov 22, 2025, 08:20 PM UTC

   Maintenance on our primary message bus is ongoing with our service provider however timeouts have stabilized. We're continuing to monitor the system and get a better understanding of the problem. We're also discussing possible changes to mitigate similar issues in the future but we expect it will be a larger project to plan and implement.

3. resolved Nov 23, 2025, 12:03 AM UTC

   Maintenance on our primary message bus is done and all systems are back to normal. This kind of maintenance shouldn't be causing timeouts or delayed processing and we will be digging into it to ensure it doesn't occur again.

Read the full incident report →

Started Oct 20, 2025, 05:19 PM UTC· resolved Oct 20, 2025, 07:18 PM UTC

1. monitoring Oct 20, 2025, 05:19 PM UTC

   Email alerts are currently backed up due to upstream issues with our service provider (Amazon SES). Integrations (such as Slack and Webhooks) are unaffected though depend on the state of those services.

2. monitoring Oct 20, 2025, 06:06 PM UTC

   We've worked through the backlog of email alerts as Amazon SES has recovered for us. We are still monitoring the situation as AWS continues to work through issues with multiple services.

3. monitoring Oct 20, 2025, 07:18 PM UTC

   We are continuing to monitor for any further issues.

4. resolved Oct 20, 2025, 07:18 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jul 29, 2025, 04:30 AM UTC· resolved Jul 29, 2025, 10:30 AM UTC

1. resolved Jul 29, 2025, 11:12 AM UTC

   The certificate for https://releases.deadmanssnitch.com expired early this morning (US Eastern) which left downloads unavailable for roughly 6 hours. We're sorry for the issue and any trouble it caused. The root cause was that we missed migrating a DNS record used to verify the certificate when migrating to a new DNS provider earlier this year.

Read the full incident report →

Started Apr 08, 2025, 12:12 AM UTC· resolved Apr 08, 2025, 03:26 AM UTC

1. identified Apr 08, 2025, 12:12 AM UTC

   We have received reports of false alerts being sent out for some short interval Snitches. The issue is related to maintenance an upstream provider is performing on our main message bus service that is causing some check-ins to be processed out of order. We've taken an initial step to mitigate the issue but have more work to do to fully address the issue.

2. monitoring Apr 08, 2025, 01:02 AM UTC

   We are continuing to monitor the situation as maintenance on our primary message bus continues at our upstream provider. We're currently discussing options for a longer term fix but we expect that fully solving the issue will take longer than it will take for the maintenance to complete.

3. resolved Apr 08, 2025, 03:26 AM UTC

   Our primary message bus is still undergoing maintenance and we have implemented all the mitigations that we can without making larger changes. Taking that into consideration and the fact that we have not seen a disruption which would cause out of order processing within the last hour we've decided to close this incident.

Read the full incident report →

Started Jan 28, 2025, 02:39 PM UTC· resolved Jan 28, 2025, 03:39 PM UTC

1. identified Jan 28, 2025, 02:39 PM UTC

   The system we use to send alerts and emails is currently backlogged due an upstream service issue. We're

2. identified Jan 28, 2025, 02:48 PM UTC

   We believe we've identified the issue which has been causing one job (sending iOS push notifications) to block our queue. We're working to deploy a fix now.

3. monitoring Jan 28, 2025, 02:57 PM UTC

   We've deployed a fix for the issue and our worker system has worked through all of the queued jobs. We're monitoring to make sure the system is stable and will be reviewing steps to avoid this in the future.

4. resolved Jan 28, 2025, 03:39 PM UTC

   Alerts and emails are once again being sent in a timely manor. We've identified the root cause as a job which had a missing timeout which was causing issues due to an upstream connection issue. This highlighted that the monitoring we have in place for our worker system needs to be reviewed as we didn't receive an alert that there was an issue.

Read the full incident report →