Pulsara incident

Delay in Push Notification Delivery

Critical Resolved View vendor source →

Pulsara experienced a critical incident on March 13, 2020 affecting Third Party Mobile Push Notifications - US, lasting 23m. The incident has been resolved; the full update timeline is below.

Started
Mar 13, 2020, 08:35 PM UTC
Resolved
Mar 13, 2020, 08:58 PM UTC
Duration
23m
Detected by Pingoru
Mar 13, 2020, 08:35 PM UTC

Affected components

Third Party Mobile Push Notifications - US

Update timeline

  1. investigating Mar 13, 2020, 08:47 PM UTC

    We are actively investigating delays in push notification delivery.

  2. identified Mar 13, 2020, 08:50 PM UTC

    We have identified an issue with one of our third party vendors. We are actively working on a fix for this. Currently, no push notification are being sent.

  3. monitoring Mar 13, 2020, 08:57 PM UTC

    Push notifications have now been restored. We are currently monitoring this event and will have more details to follow.

  4. resolved Mar 13, 2020, 08:58 PM UTC

    This incident has been resolved.

  5. postmortem Mar 13, 2020, 10:49 PM UTC

    Today, at roughly 14:35 MST our internal monitoring systems alerted us of push notification failures. At 14:38 MST our engineering team discovered that the root cause of this incident was a failure with one of our third party vendors. During this time, no push notifications on either Android or iOS were sent. Pulsara video calls and all other data components continued to work during this time. Our development team continued to monitor this event and prepared backup alternatives to deploy to our production environment if needed. At 14:55 MST our third party vendor resumed working status and push notifications began to work again. At this time, all new push notifications were sent as expected. As of 14:55 MST, Pulsara was completely back up and running at 100%.