xByte Cloud incident
Alert received about connection timeouts/slowness
xByte Cloud experienced a minor incident on December 30, 2023 affecting Zone C - US Central, lasting 1h 20m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Dec 30, 2023, 07:50 PM UTC
We are looking into an alert received with connection timeouts/slowness.
- identified Dec 30, 2023, 08:21 PM UTC
The issue has been identified and our network team has implemented a mitigation
- monitoring Dec 30, 2023, 08:40 PM UTC
At around 1:40 PM CT, our Network Team received alerts about network slowness. During this time, some customers might have experienced temporary slowness or timeouts with their services. Our team immediately identified an issue in one of our core services and implemented a failover to address it. We are currently investigating the root cause and will provide an update as soon as more information becomes available.
- resolved Dec 30, 2023, 09:56 PM UTC
After applying the mitigation our network team has continued monitoring core services and all services are fully operational. We will continue monitoring on our side for any ongoing issues.
- postmortem Jan 08, 2024, 03:38 PM UTC
The root cause of the incident was traced back to a failure in one of our ISP provider's network services. This triggered an automatic redundant failover on our end, mitigating the impact on our services. Subsequently, the ISP provider has since resolved their internal issue, ensuring a complete resolution on their side.