Intermittent outage of ViiBE web portals (*.viibewithus.com)
Timeline · 4 updates
- investigating Sep 02, 2026, 11:48 PM UTC
Starting at approximately 22:46 UTC (6:46 PM ET) on September 2, ViiBE portals are intermittently unreachable. Users may see connection failures or a "no healthy upstream" error. The cause is a production Kubernetes node that is repeatedly going unhealthy, which is disrupting the ingress gateway that serves these sites. We have identified the failing node and are working to move traffic off it. Next update in 30 minutes.
- investigating Sep 02, 2026, 11:49 PM UTC
We are continuing to investigate this issue.
- monitoring Sep 03, 2026, 12:15 AM UTC
As of 5:12 PM PT, ViiBE services have restarted and the portals are loading again. A production Kubernetes node in the cloud provider failed and repeatedly dropped offline, which took the ingress gateway and several backend services with it. We added capacity, removed the failing node, and moved all workloads to healthy nodes. We are monitoring closely while services finish recovering.
- resolved Sep 03, 2026, 12:57 AM UTC
A production Kubernetes node in the cloud provider failed and repeatedly dropped offline, which took the ingress gateway and several backend services with it. We added capacity, removed the failing node, and moved all workloads to healthy nodes.