Iterable experienced a minor incident on November 12, 2024 affecting Email Sends and Email Sends and 1 more component, lasting 2h 5m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 12, 2024, 05:18 PM UTC
Starting at around 6:30 am PT, Iterable Engineers were notified that lag on our ingestion topics was growing on a subset of clusters. Customers on these impacted clusters could be experiencing delays with user updates, list uploads, user deletion, and event processing. Email sends and journey processing may see impact as well. Data is only being delayed. It is not being dropped. The engineering team is working to remediate these delays. Next update at 10:00 an PT or Sooner.
- monitoring Nov 12, 2024, 05:52 PM UTC
The engineering team has identified the cause and remediated the delays on these subset of clusters. Most impacted ingestion topics have recovered and customers should start to see data being ingested normally again. We are still working through some backlog, so the team will continue to monitor to ensure that these delays don't recur. Next Update at 11:00 am PT or sooner?
- resolved Nov 12, 2024, 07:24 PM UTC
Ingestion has returned to normal for all impacted clusters and this issue is now resolved.