Langfuse incident

Slow trace data retirival across API and UI in EU

Resolved after 1 hour 52 minutes, affecting EU (cloud.langfuse.com) (UI) and EU (cloud.langfuse.com) (Public API) and 1 more, on October 21, 2026.

resolved Oct 21, 2026, 09:39 AM UTC

should be finally resolved

Started
Oct 21, 2026, 07:46 AM UTC
Resolved
Oct 21, 2026, 09:39 AM UTC
Duration
1h 52m
Detected by Pingoru
Oct 21, 2026, 07:46 AM UTC

Affected components

EU (cloud.langfuse.com) (UI)EU (cloud.langfuse.com) (Public API)EU (cloud.langfuse.com) (LLM-as-a-judge)US (us.cloud.langfuse.com) (UI)US (us.cloud.langfuse.com) (Public API)US (us.cloud.langfuse.com) (LLM-as-a-judge)HIPAA (hipaa.cloud.langfuse.com) (UI)HIPAA (hipaa.cloud.langfuse.com) (Public API)HIPAA (hipaa.cloud.langfuse.com) (LLM-as-a-judge)JP (jp.cloud.langfuse.com (UI)

Update timeline

  1. investigating Oct 21, 2026, 07:46 AM UTC

    Slow traces data serving across API and UI

  2. investigating Oct 21, 2026, 07:46 AM UTC

    Slow traces data serving across API and UI

  3. identified Oct 21, 2026, 08:31 AM UTC

    Slow clickhouse read replicas affecting performance across the service. Draining and replacing them.

  4. identified Oct 21, 2026, 08:31 AM UTC

    Slow clickhouse read replicas affecting performance across the service. Draining and replacing them.

  5. resolved Oct 21, 2026, 08:45 AM UTC

    All replicas look healthy again

  6. resolved Oct 21, 2026, 08:45 AM UTC

    All replicas look healthy again

  7. monitoring Oct 21, 2026, 08:56 AM UTC

    Looks like some timeout are still occurring. Monitoring.

  8. monitoring Oct 21, 2026, 08:56 AM UTC

    Looks like some timeout are still occurring. Monitoring.

  9. resolved Oct 21, 2026, 09:39 AM UTC

    should be finally resolved

  10. resolved Oct 21, 2026, 09:39 AM UTC

    should be finally resolved

  11. resolved Oct 21, 2026, 09:39 AM UTC

    should be finally resolved

Your product runs on more than Langfuse

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →