Replicate incident

Delayed scaling due to node failure

Minor Resolved View vendor source →

Replicate experienced a minor incident on August 10, 2026 affecting L40S Hardware and H100 Hardware, lasting —. The incident has been resolved; the full update timeline is below.

Started
Aug 10, 2026, 09:45 PM UTC
Resolved
Aug 10, 2026, 09:45 PM UTC
Duration
Detected by Pingoru
Aug 10, 2026, 09:45 PM UTC

Affected components

L40S HardwareH100 Hardware

Update timeline

  1. monitoring Aug 10, 2026, 09:05 PM UTC

    Status: Monitoring Scaling decisions were delayed by nearly 1 hour after the controller responsible for emitting queue metrics failed to schedule on a soft-failed node. We have since cordoned and drained the node, and the controller is emitting queue metrics again. Affected components H100 Hardware (Degraded performance) L40S Hardware (Degraded performance)

  2. resolved Aug 10, 2026, 09:45 PM UTC

    Status: Resolved All affected models have been scaling correctly for more than 30 minutes at this point, and we see no residual prediction queues. Thank you for your patience! Affected components L40S Hardware (Operational) H100 Hardware (Operational)