Delays on payload targeting ingestion and audience generation

Update timeline

1. investigating Jun 27, 2019, 05:05 PM UTC

   We are currently experiencing delays on payload targeting ingestion and audience generation. This is being investigated by our engineers.

2. identified Jun 27, 2019, 05:43 PM UTC

   The root cause has been identified in a downstream provider. Our provider have acknowledged the issue and is currently working to resolve it.

3. identified Jun 27, 2019, 05:56 PM UTC

   We are continuing to work on a fix for this issue.

4. identified Jun 27, 2019, 06:30 PM UTC

   We are continuing to monitor the status of our 3rd party provider, but have initiated an alternative resolution process to restore functionality.

5. identified Jun 27, 2019, 07:39 PM UTC

   Due to the extend outage from our provider we have implement an alternative solution. However this will take some time to become available. We are monitoring the situation with our provider in the hope of resolving this sooner.

6. monitoring Jun 27, 2019, 08:50 PM UTC

   We have implement a workaround to the on-going provider outage and are continuing to monitor the recovery.

7. monitoring Jun 27, 2019, 09:47 PM UTC

   Push Notifications have now recovered. Payload Targeting audience updates are still delayed. We continue to monitor our downstream provider for full service recovery

8. resolved Jun 27, 2019, 10:34 PM UTC

   This incident has been resolved. Our downstream provider has resolved their outage and we have disabled the alternative solution. There remains a delay on update to Payload Targeting update however this should clear up shortly.