Singlewire Software incident

Channel Delivery Slowdowns and Failures

Minor Resolved View vendor source →

Singlewire Software experienced a minor incident on August 13, 2026 affecting Android Push Notifications and Emergency Calling and 1 more component, lasting 1d. The incident has been resolved; the full update timeline is below.

Started
Aug 13, 2026, 08:47 PM UTC
Resolved
Aug 14, 2026, 09:41 PM UTC
Duration
1d
Detected by Pingoru
Aug 13, 2026, 08:47 PM UTC

Affected components

Android Push NotificationsEmergency CallingiOS Push NotificationsSinglewire IDP AuthenticationEmail NotificationsPhone Call NotificationsSMS NotificationsWebEx Teams NotificationsMicrosoft TeamsOn-Premises Notifications

Update timeline

  1. investigating Aug 13, 2026, 08:47 PM UTC

    We've detected slowdowns and potential failures in notification delivery across many channels. Those channels include: - SMS messages - Phone calls - Emails - Push notifications to mobile apps - All on-premises devices, including speakers This may also impact authentication and general platform access. Our engineers are working to understand the scope of the issue and find a fix. We'll provide an update when we know more.

  2. monitoring Aug 13, 2026, 10:07 PM UTC

    We've failed over on the impacted service, and we believe service should be restored at this point. We'll continue to monitor overnight and provide an update in the morning when we have more information about the root cause of the service disruption.

  3. monitoring Aug 14, 2026, 01:54 PM UTC

    We're continuing to monitor to ensure we've completely recovered, and we'll give another update before 2PM CDT.

  4. monitoring Aug 14, 2026, 06:40 PM UTC

    Service has been restored, and we believe we've addressed the root cause. However, because of the intermittent nature of the issue, we're going to continue to monitor for the rest of the business day.

  5. resolved Aug 14, 2026, 09:41 PM UTC

    This incident has been resolved.

  6. postmortem Aug 24, 2026, 05:07 PM UTC

    On August 13th, we observed a few isolated occurrences of queries to our in-memory data store timing out. While we don't have a definitive root cause for the issue, we have been able to simulate similar cases with high query volume in our testing environments, and are making the following remediations to both hopefully prevent this issue from occurring again as well as mitigate it in the event that it does. First, we'll be upgrading the instance class of the machines this database runs on, which uses a more modern AWS hypervisor and processor architecture. Additionally, we'll be bolstering certain commands with retry and circuit breaker logic, such that in the rare case that this does happen again, the command will be immediately retried to a point, while allowing the queries to be skipped entirely if the database is truly unavailable. Lastly, we'll be tweaking our slow query logs in order to ensure we keep an eye on performance of this resource so that we can continue to tune our queries and ensure they don't place an undue amount of stress on this database. We apologize for any inconvenience this may have caused.