Iterable experienced a minor incident on July 5, 2024 affecting Global API Success and Global API Ingestion, lasting 4h 50m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jul 05, 2024, 05:29 PM UTC
Beginning at approximately 9:12 AM PST, the engineering team is observing a performance degradation that is increasing the global API ingestion delay. We have identified the cause of the issue and are working to fix it. The next update will be at 12:00 PM PST or sooner.
- identified Jul 05, 2024, 05:30 PM UTC
The issue has been identified and the engineering team is working on a fix.
- identified Jul 05, 2024, 05:35 PM UTC
We are continuing to work on a fix for this issue.
- identified Jul 05, 2024, 05:36 PM UTC
We are continuing to work on a fix for this issue.
- identified Jul 05, 2024, 05:37 PM UTC
We are continuing to work on a fix for this issue.
- identified Jul 05, 2024, 05:41 PM UTC
We are continuing to work on a fix for this issue.
- identified Jul 05, 2024, 07:06 PM UTC
The engineering team continues working on a fix to address the performance degradation. The next update will be at 2:00 PM PST.
- identified Jul 05, 2024, 08:44 PM UTC
The engineering team restored service at 1:00 PM PST today. The team observed the ingestion lag is still improving, but it will take up to two hours to catch up. We will continue to monitor the ingestion lag. The next update will be at 3:30 PM PST.
- identified Jul 05, 2024, 08:44 PM UTC
The engineering team has restored service, and API ingestion processes incoming requests without lag. We are continuing to monitor API ingestions to ensure that it remains responsive. The next update will be at 6:00 PM PST.
- monitoring Jul 05, 2024, 09:39 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jul 06, 2024, 01:35 AM UTC
This incident has been resolved. Please contact support if you have further questions.