StarRez experienced a major incident on January 25, 2023 affecting Email Sending, lasting 3h 12m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 25, 2023, 07:58 AM UTC
Customers in all regions are experiencing a service disruption for Core Services. We are seeing sporatic network connectivity issues across a number of regions globally. - Engineers are actively investigating the issue, it is unknown at this time what the cause is. - Next update expected within 60 minutes, or as warranted by a change of events. Apologies for any inconvenience, StarRez Team
- investigating Jan 25, 2023, 08:53 AM UTC
We have confirmed that the problem is a networking issue with our upstream vendor. There is not yet an ETA on the resolution.
- investigating Jan 25, 2023, 11:10 AM UTC
We are continuing to investigate this issue.
- resolved Jan 25, 2023, 11:11 AM UTC
The network connectivity issues have been resolved.
- postmortem Jan 27, 2023, 03:08 AM UTC
**StarRez Root Cause Analysis** Global outage - 25th Jan 2023 **Summary** On the 25th Jan 2023, customers across all regions experienced intermittent downtime to a number of core services: The affected sites would be unreachable for a short time, then come back online shortly after, often several times a minute. **Root Cause** The root cause was determined to be a global network outage with our upstream provider **Resolution** Our upstream provider reverted a change made to their Wide Area Network \(WAN\), which restored stability for all affected StarRez sites.