Is Tavus down?

We are investigating increased error rates when training a Face with an image.

Error rates are returning to baseline and we are continuing to monitor the pipeline

This incident has been resolved.

We are aware of an issue currently impacting tavus conversations, we are investigating.

It appears this issue is coming from one of our downstream providers, Crusoe. We're continuing to investigate full scope of conversations affected.

Starting at 17:40 UTC, conversations using our default LLM began experiencing long delays and unresponsive replicas. A GPU provider is returning errors and our automatic failover is recovering most turns, but with significant added latency. We are routing traffic to a backup provider.

This is affecting gemma-4; we recommend our customers switch to a different LLM for immediate needs.

The issue is isolated to our default LLM (gemma-4), served by a GPU provider that began returning errors at 17:40 UTC. As a temporary mitigation we are routing gemma-4 traffic to an alternate model. Customers on gemma-4 may notice a change in response style while this is in effect; we will revert once the provider recovers.

We've confirmed the issue originates with our GPU provider, who has declared a networking incident on their side. We're testing mitigations to route gemma-4 traffic elsewhere. Conversations using other LLMs are unaffected. If you need stability now, you can point your persona at a different LLM.

Our upstream GPU provider has applied a mitigation and gemma-4 error rates have returned to normal. We have alternate routing paths staged and ready to enable if the issue recurs. We are monitoring before marking this resolved.

This incident is now resolved.

Charlie builder is going through planned maintenance and will be up at 100% capacity within 10 minutes.

Charlie builder is back working as expected. This issue is resolved.

We are investigating an elevated error rate in image-to-face trainings.

A fix has been rolled out and we are seeing errors resolve. We are continuing to monitor the fix

This incident has been resolved.

When one of our GPU providers performed maintenance, we asked to migrate traffic to a different cluster beforehand to avoid disruptions. During the migration back, the provider encountered an issue that led to some conversations experiencing an elevated startup time.