CrossEngage incident

Data Processing Delay

Minor Resolved View vendor source →

CrossEngage experienced a minor incident on May 1, 2024 affecting Audience campaign dispatch and API v1.0 and 1 more component, lasting 1d. The incident has been resolved; the full update timeline is below.

Started
May 01, 2024, 10:32 AM UTC
Resolved
May 02, 2024, 10:45 AM UTC
Duration
1d
Detected by Pingoru
May 01, 2024, 10:32 AM UTC

Affected components

Audience campaign dispatchAPI v1.0SegmentationRealtime campaign dispatchAPI v2.0Web trackingStories Dispatch

Update timeline

  1. identified May 01, 2024, 10:32 AM UTC

    We are currently experiencing degraded platform performance connected to segmentation. In order to implement measures to solve it we have paused message dispatching. Our engineers are currently working on this. We will publish an update as soon as campaign execution resumes. Please accept our apologies for the inconvenience this causes and please do reach out if you have any further queries.

  2. monitoring May 01, 2024, 01:19 PM UTC

    The issue has been identified and a fix has been implemented. We are rapidly consuming data queues & will keep monitoring in the coming hours. Our apologies for the inconvenience this might have caused.

  3. investigating May 01, 2024, 04:41 PM UTC

    Our Datacenter provider informed us about degraded network performance. They are working to find a solution as fast as possible.Some instabilities are expected in the next hours. We have joined forces with our provider and keep monitoring the situation over the upcoming hours. Apologies for any inconvenience this may have caused you. Let us know if we can be of any assistance.

  4. identified May 01, 2024, 09:35 PM UTC

    The cause has been identified and our data centre provider is in the final stage of implementing the solution. Just a small reminder. All the dispatches are paused, the incoming data is being queued (no data has been lost) and will be processed as soon as the issue has been resolved. We will keep you updated about the progress. Our apologies for the inconvenience this is causing. Feel free to get in touch if you have any questions. Your CrossEngage Team

  5. monitoring May 02, 2024, 04:13 AM UTC

    After our data centre addressed network instabilities, our application recovered. Dispatching is now working as expected. We will keep monitoring the system in the upcoming hours. Once more, please accept our apologies for the inconvenience this causes and do reach out if you have any further queries. Your CrossEngage Team

  6. resolved May 02, 2024, 10:45 AM UTC

    After thoroughly monitoring our systems we are marking this incident as resolved. Once more, we apologize if this caused any inconvenience. Feel free to get in touch with us if you have any questions. Your CrossEngage Team