Investigating failing calls and RoomService API failures in US Central/East and dashboard issues

Update timeline

1. investigating Oct 28, 2026, 11:47 PM UTC

   We are currently investigating an increased rate of failing outbound calls in US Central and US East. We will post another update with more information as soon as possible.

2. investigating Oct 28, 2026, 11:51 PM UTC

   The LiveKit cloud dashboard is currently inaccessible. We are investigating this and will provide another update within 15 minutes.

3. investigating Oct 28, 2026, 11:57 PM UTC

   We believe there is also impact to inbound calls in the affected regions. We are working on identifying the cause and will update again within 15 minutes.

4. investigating Oct 29, 2026, 12:00 AM UTC

   We are currently attempting a mitigation. We will follow up with more information in the next 15 minutes.

5. investigating Oct 29, 2026, 12:09 AM UTC

   We are still determining the full scope of impact, but we believe that the Room Service has also been impacted. We will update again within 15 minutes.

6. monitoring Oct 29, 2026, 12:16 AM UTC

   We believe the mitigation was successful and are seeing services return to normal. The dashboard is accessible and error rates have gone down across services. We are continuing to monitor to make sure that the situation is resolved. We will update again as soon as possible.

7. identified Oct 29, 2026, 12:36 AM UTC

   While the US regions still appear to be stable, we are tracking degraded services including SIP, RTC, and APIs in our Mexico region. We will update again as soon as possible.

8. monitoring Oct 29, 2026, 12:41 AM UTC

   We've applied a mitigation for our Mexico region and services have returned to baseline. We are continuing to monitor and will update again within 15 minutes.

9. resolved Oct 29, 2026, 01:19 AM UTC

   This incident has been resolved. Between 23:30 and 23:58 UTC, a network disruption at an upstream infrastructure provider affected our US East region, with impact also seen in US Central. During this window: Inbound SIP calls in US East and US Central failed at elevated rates, with affected calls rejected with 500 or 503 responses. Answered inbound calls were about 65% below normal across the window, and up to 90% below normal between 23:45 and 23:55 UTC. About 10% of outbound SIP call requests (CreateSIPParticipant) in US East failed, some returning 408 deadline_exceeded. Between 23:28 and 23:58 UTC, there were elevated rates of Room Service, Egress, and Ingress API request server errors in US East. Some participants in US East and US Central saw failed room joins or increased packet loss. The LiveKit Cloud dashboard was inaccessible for part of this window. Network connectivity recovered at approximately 23:58 UTC, and error rates and call volumes returned to normal by 00:20 UTC. A related network issue also affected our much smaller Mexico region from 23:29 UTC, disconnecting active sessions and degrading service until traffic was moved to nearby regions at approximately 01:01 UTC. Sincere apologies for the impact. We will follow up with a postmortem.