ETHERFAX experienced a major incident on November 23, 2022 affecting Canada Outbound, lasting 2h 3m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 23, 2022, 04:12 PM UTC
Engineers are currently engaged with outbound carrier in our TR1 facility and actively troubleshooting outbound call failures.
- identified Nov 23, 2022, 04:43 PM UTC
Engineers have verified that a data center cross connect is down between etherFAX and our carrier. Data center and carrier engineers have been engaged and are troubleshooting the outage. At this time we have implemented a workaround for outbound calls and those calls should be completing.
- monitoring Nov 23, 2022, 05:04 PM UTC
Carrier has verified a circuit was in alarm on their side. They have corrected this and etherFAX engineers are continuing to monitor before returning traffic to the normal paths.
- monitoring Nov 23, 2022, 05:04 PM UTC
We are continuing to monitor for any further issues.
- resolved Nov 23, 2022, 06:16 PM UTC
At this time carrier and colocation provider have determined there was damage to cabling entering the data center due to construction. This is under repair currently but they have routed traffic around affected infrastructure.