Zipline incident

An upstream provider's outage is affecting our ability to deliver push notifications

Major Resolved View vendor source →

Zipline experienced a major incident on December 7, 2021 affecting Notifications and Files, lasting 11h 52m. The incident has been resolved; the full update timeline is below.

Started
Dec 07, 2021, 05:02 PM UTC
Resolved
Dec 08, 2021, 04:55 AM UTC
Duration
11h 52m
Detected by Pingoru
Dec 07, 2021, 05:02 PM UTC

Affected components

NotificationsFiles

Update timeline

  1. identified Dec 07, 2021, 05:02 PM UTC

    An upstream provider is encountering errors that are affecting Zipline's ability to deliver push notification to mobile devices. The rest of Zipline is fully functional and we are actively monitoring the situation.

  2. identified Dec 07, 2021, 05:20 PM UTC

    Push notifications to mobile devices are currently being delayed and will be delivered after the incident is resolved.

  3. identified Dec 07, 2021, 09:43 PM UTC

    Our upstream provider is continuing to work on a number of different mitigation and resolution actions. They've identified the root cause of this issue. While there are some early signs of recovery, there remains no ETA for full recovery yet. User and Team alignments processing is also delayed due to this incident. We have received the files successfully and they will be processed once the incident is resolved.

  4. monitoring Dec 07, 2021, 11:42 PM UTC

    Our upstream provider has mitigated the underlying issue and Zipline is fully operational. All delayed mobile push notifications have been delivered and we are processing the user and team alignment files.

  5. resolved Dec 08, 2021, 04:55 AM UTC

    This incident has been resolved.