TeraSwitch had 46 outages in the last 2 years totaling 2858h 22m of downtime — averaging 1.9 incidents per month.
There were 46 TeraSwitch outages since February 28, 2025 totaling 2858h 22m of downtime. Each is summarised below — incident details, duration, and resolution information.
Teraswitch is looking into issues related to the R102 compute rack within TYO2.
A possible firmware/hardware bug was identified, and Teraswitch NOC elected to do a sequential software upgrade on the redundant switch pair in R102. After the reloads were complete, operations seemed to normalized. We will investigate this network switch pair further to understand the event better. Please report any outstanding issues with TYO2 to [email protected] or to our team directly.
Teraswitch is investigating reports of Internet connectivity loss at SGP2. We will provide an update shortly.
Teraswitch has isolated this incident to a vendor power maintenance event with unexpected impact, affecting both of our network racks and causing an Internet connectivity loss at SGP2. Customer services are believed to be powered on without upstream connectivity. Facility staff are at work to restore Internet connectivity and bring the site back online. We will provide an update as soon as possible.
Our facility partner has restored power to Teraswitch's SGP2 network racks. Root cause: both redundant PDUs in each network rack were fed by power breakout boxes connected to the same overhead electrical bus bar, creating a single point of failure. Facility engineers relocated one breakout box to the correct bus, restoring power to the network equipment and eliminating the single point of failure. We expect no further impact at this time. Customer systems did not lose power at any point during this event.
Teraswitch is investigating reports of intermittent network connectivity issues affecting some services in our EWR2 (Newark, NJ) location.
We have identified the issue affecting network connectivity in our EWR2 (Newark, NJ) location and are implementing corrective changes.
We have located an unstable core router in EWR2 and have implemented mitigations to divert traffic away from it.
Teraswitch monitoring reports that all impacts are currently resolved. Please report any further issues to our team. At EWR2, a device sitting between our data center fabric and our internet edge routers experienced a hardware fault that triggered an ASIC-level reload. After the reload, the device came back online in a degraded state and was unable to forward traffic from the data center fabric to the internet edge routers (specifically, it could not process VXLAN-routed traffic). EWR2 operates six of these core routers. Five were unaffected; however, the impaired unit failed to signal that it was incapable of forwarding traffic. Because downstream routing protocols did not detect this bad state, traffic that landed on the affected device was blackholed. Our team manually removed the router from the pool of forwarding destinations for this traffic, restoring service. We will perform all necessary follow-up steps to fully resolve the underlying issue and prevent recurrence.
Teraswitch identified a brief impact to network connectivity at our DUB1 and DUB2 sites beginning at 23:43 UTC. Undersea cables from both London and Amsterdam were lost, resulting in the brief loss of backbone and Internet connectivity at DUB1 and DUB2. Traffic was quickly restored via Internet routes and backbone connectivity via London followed shortly thereafter. Repair of the Amsterdam cable is also underway. Teraswitch is monitoring to confirm stability at these sites and will provide an update in the event of any further issues.
The Concerto cable was fully restored - multiple redundant links are now back online.
Teraswitch has identified a loss of two undersea cables landing in our SEA1 location from Tokyo. This is likely a terrestrial issue inside Seattle. Traffic was automatically diverted to our LA to Tokyo path.
Tokyo to Seattle has been nominal operation for many hours now. We are working with our vendors to understand the issue, but at this time we consider the incident and impact resolved.
Teraswitch is investigating an issue affecting customer management of VAN1 services (power controls, virtual console, metadata retrieval, new service creation, reinstalls). VAN1 services themselves are unaffected by this issue and should be fully operational. We will provide an update as more information becomes available.
Customer management of VAN1 services should be restored at this time. Teraswitch staff are monitoring for any further impacts.
This incident has been resolved.
Teraswitch is investigating a core router failure in TYO2. Traffic is routing over the remaining redundant routers.
After an emergency software upgrade, the router is normalized. Teraswitch is monitoring the operation for further errors.
This incident has been resolved.
Teraswitch has identified a top-of-rack switch failure at our TYO1 site affecting a small subset of customers there. Services single homed to this switch are currently offline. We are working on an immediate replacement and will provide updates as more information becomes available.
The failed switch has been replaced and is being finalized for production. We will provide an update once affected services are fully restored.
Our replacement switch is in service and all affected services should be restored at this time. If you have any further issues, please contact Teraswitch Support at [email protected].
This incident has been resolved.
Teraswitch is investigating reports of network connectivity issues affecting some services at our EWR2 site. We have identified the likely cause and implemented a fix, and are monitoring to verify that the issue is resolved.
This incident has been resolved.
Teraswitch is investigating reports of select markets having issues connecting to PIT1.
We have located a network device with software issues, our team is about to reload the device. This issue affects mostly colocation customers at PIT1, but traffic from various sources may have passed through this device depending on the direction of network flow.
Reload is complete and the network operation appears to have returned to normal. We will follow up this notification with a RCA and repair plan.
This incident has been resolved.
Teraswitch is investigating a loss of the majority of VAN1 internet connectivity - possibly related to issues in Seattle.
Cogent has recovered as of 4:33am Eastern. Services should be normalized at this time.
TeraSwitch is aware and tracking a loss of two paths from Seattle to Tokyo. This has caused much of the internet traffic to divert via public Internet Transit.
This incident has been resolved.
We have reached out to the vendor asking for a update. Latency from Singapore to Europe will be elevated until the outage is resolved.
We have successfully brought online a second diverse sub-sea cable path and latency has returned to normal. Once the original severed cable comes back online, we will now have full path redundancy.
Teraswitch has identified thre core backbone links that are down within the central US. This is causing US-East traffic to wrap around the world to get to US-West. Chicago/Seattle Chicago/Salt Lake City Dallas /Los Angeles We are working with our fiber vendors for a ETR.
Chicago to Salt Lake City path is restored which has normalized operations with 1 of 3 links restored. We will close this status when 2 of 3 are restored which also restores redundancy.
This incident has been resolved.
Teraswitch is investigating reports that our console (console.tsw.io) is currently inaccessible. This issue has been identified as the result of an outage in an underlying cloud provider. We will provide an update as more information becomes available.
This issue is due to an ongoing Cloudflare outage: https://www.cloudflarestatus.com/incidents/m1xvmqf37z97. Cloudflare has implemented a fix and we are monitoring for recovery. Our API is unaffected by this issue and remains accessible.
Cloudflare has resolved the issue, and console.tsw.io is accessible once again as of 22:28 UTC. This incident has been resolved.
We are investing significant packet loss on the west coast US.
We are continuing to investigate this issue.
This incident has been resolved.
At 2PM UTC, Teraswitch reloaded software related to our route servers to fix "stuck" transport tunneling sessions. This may have caused interruptions to ULL/HFT and transport services over our network. We believe the first stuck session started approximately 6 hours before, with more dropping off over the next few hours. As of this moment, all stuck sessions appear to be resolved, and traffic is passing normally and via ULL links.
Due to multiple undersea cable cuts, SGP1 and SGP2 is no longer operating on the Teraswitch backbone. SGP1/2 networking being unreachable for a time period was seen as network routes shifted and Internet routes adjusted.
This incident has been resolved.
Teraswitch is investigating increased errors in our service portal (console.tsw.io). Teraswitch services themselves are unaffected. We will provide an update as more information becomes available.
The cause of the increased errors was identified and fixes put in place. This issue is resolved.
Terawitch Network Operations has observed a subsea cable fault between our Seattle and Tokyo POPs. Latency to Tokyo may be increased while the system is down. We have reached out to our cable provider for an update.
Fault has cleared and cable returned to service.
Teraswitch is investigating monitoring alerts/errors in our console (console.tsw.io) and API (api.tsw.io). This appears to be due to an underlying major cloud provider which is experiencing an outage. We will provide an update as more information becomes available.
This issue has been isolated to an ongoing Cloudflare network outage - see their incident page for more details: https://www.cloudflarestatus.com/incidents/8gmgl950y3h7
Cloudflare updated their incident report that their services are now operating normally - they will post a final update to the incident once their investigation has concluded. Teraswitch monitoring confirms our console / API services have stabilized with no further errors. This incident is now resolved.
Teraswitch is investigating moments of Internet connectivity instability.
Teraswitch was attacked by a DDOS attack that specifically targeted our infrastructure and blockchain services. While our network has the capacity and mitigation ability, it took longer than expected to identify good traffic from attack traffic without further causing worse issues. Some attack traffic sources came from very abnormal and specific locations, such as AT&T (US ISP) and a group of Brazilian ISPs. This traffic primarily landed in our Dallas data center, which overloaded select network links there on its way towards the attack targets. We have tightened our rule sets, shifted traffic around, and are working to ensure impacts are continually mitigated.
We are observing further Internet connectivity instability due to the attack and are continuing to apply mitigations. We will continue to monitor the situation until resolution and update on any further changes.
This incident is now considered resolved.
Our trans-Atlantic link from London to New York has dropped offline. Traffic is diverted via Ashburn to Frankfurt.
This detour has already cleared and was a momentary disruption. TeraSwitch will engage our transport vendor for a better understanding of this incident.