Drips experienced a major incident on October 12, 2020 affecting Drips System, lasting 6h 55m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 12, 2020, 04:20 PM UTC
The system is currently experiencing delays with outbound SMS messages and calls. We are currently investigating and will update with findings.
- investigating Oct 12, 2020, 05:22 PM UTC
Stand-alone Outbound Calls are now functional. Outbound Calls that follow SMS messages are still delayed, because Outbound SMS messages are still delayed. We are continuing to investigate, and will update with more findings and resolutions.
- investigating Oct 12, 2020, 07:40 PM UTC
As of 3:30pm ET, we are still investigating this issue. We will continue to update here with any new resolutions or findings.
- investigating Oct 12, 2020, 08:32 PM UTC
As of 4:30pm ET, this issue is still unresolved, and our teams are continuing to investigate this issue. We will post updates here every hour, and as we make progress on this issue.
- investigating Oct 12, 2020, 09:31 PM UTC
As of 5:30pm ET, our investigations continue. Please subscribe to updates, which we will continue to post hourly until this issue is resolved.
- identified Oct 12, 2020, 09:57 PM UTC
As of 6pm ET, our team has isolated the root causes, and are now working through catching up all outbound SMS and Calls. This should take the next 45-60 minutes, and we will post an update when it is completed.
- monitoring Oct 12, 2020, 11:02 PM UTC
As of 7pm ET, we have implemented the fix, processed our backlog of Drips, and are now back to our regularly schedule cadences. We are continuing to monitor the system to ensure the issue does not recur.
- resolved Oct 12, 2020, 11:15 PM UTC
As of 7pm ET, we have implemented the fix, processed our backlog of Drips, and are now back to our regularly schedule cadences. We are continuing to monitor the system to ensure the issue does not recur.