Verst Logistics experienced a major incident on November 4, 2024 affecting Cleo, lasting 3h 12m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 04, 2024, 05:42 PM UTC
Cleo Engineers are aware of a condition that is affecting inbound and outbound connectivity for communication protocols. Currently outbound transactions are being queued by CIC and inbound transactions are producing failures to trading partners. Customers utilizing API connections do not appear to be affected. Customers using Cleo hosted partner mailboxes (uploading files via a Cleo website) are receiving a “502 Bad Gateway” error in their browser. Cleo Engineers are working to resolve these issues.
- investigating Nov 04, 2024, 05:43 PM UTC
An update has been posted Based on the previous change outbound delays are down to fifteen minutes. For inbound, approximately 75% of inbound connections continue to be impacted. However, based on best practices, trading partners would have attempted retries and those retries have a 25% chance of successful execution, meaning after roughly four retry attempts, inbound traffic will get through. Cleo engineers are continuing to make adjustments to facilitate the successful processing of the remaining traffic. We will continue to provide periodic updates.
- monitoring Nov 04, 2024, 06:05 PM UTC
Service has been restored and inbound and outbound communications are working as expected. Verst IT and Cleo engineers will continue to closely monitor CIC to ensure there is no recurrence of the previous condition.
- resolved Nov 04, 2024, 06:43 PM UTC
This incident has been resolved.