Iterable incident

C22 ingestion delays

Minor Resolved View vendor source →

Iterable experienced a minor incident on May 20, 2025 affecting Journey Processing and User Updates and 1 more component, lasting 4h 10m. The incident has been resolved; the full update timeline is below.

Started
May 20, 2025, 02:52 PM UTC
Resolved
May 20, 2025, 07:02 PM UTC
Duration
4h 10m
Detected by Pingoru
May 20, 2025, 02:52 PM UTC

Affected components

Journey ProcessingUser UpdatesList UploadsUser Deletions

Update timeline

  1. investigating May 20, 2025, 02:52 PM UTC

    Starting at around 9 pm PT 5/19, cluster 22 began experiencing delays delays on user updates, list uploads, user deletion, and event processing. Customers could experience delays of up to 6 hours currently. The engineering team is currently investigating and working to remediate these delays. Next update at 9:30 am PT or sooner.

  2. investigating May 20, 2025, 03:20 PM UTC

    We are continuing to investigate this issue.

  3. monitoring May 20, 2025, 04:34 PM UTC

    Our engineering team has rolled out a fix to remediate the delays seen on c22 for user updates, list uploads, user deletion, and event processing. The system is currently working through the backlog of data and we should see it drained in a couple of hours. Engineering will continue to monitor the cluster performance until the backlog is completely drained. Next update at 11:30 am PT or sooner.

  4. resolved May 20, 2025, 07:02 PM UTC

    The system has finished working on backlog of data and the data is caught up. All systems are operational and have no issues. Engineering will continue to monitor the cluster performance.