Railway incident

Elevated error rates for deployments with volumes on Southeast Asia (Singapore) Metal region

Resolved after 2 hours 16 minutes, affecting Southeast Asia (Singapore) (Deployments (Southeast Asia)), on April 18, 2025.

monitoring Apr 18, 2025, 01:17 PM UTC

All workloads with a volume on the impacted host is back online. All other workloads have been successfully moved to a different host. We are actively monitoring the recovery. If your deployment failed due to “Container failed to start" error, please attempt a re-deploy.

Started
Apr 18, 2025, 11:42 AM UTC
Resolved
Apr 18, 2025, 01:58 PM UTC
Duration
2h 16m
Detected by Pingoru
Apr 18, 2025, 11:42 AM UTC

Affected components

Southeast Asia (Singapore) (Deployments (Southeast Asia))

Update timeline

  1. investigating Apr 18, 2025, 11:42 AM UTC

    We are currently investigating elevated error rates of deployments in Southeast Asia (Singapore) Metal. A small subset of services with volumes may be failing to deploy with the error message "Container failed to start"

  2. identified Apr 18, 2025, 12:01 PM UTC

    We have identified a single degraded host on Southeast Asia (Singapore) Metal. We are currently restoring active deployments without a volume on the host to a different host. Workloads with a volume located on the host may be unavailable during this period. Impacted users will see a deployment initiated by Railway as part of this restoration process.

  3. monitoring Apr 18, 2025, 01:17 PM UTC

    All workloads with a volume on the impacted host is back online. All other workloads have been successfully moved to a different host. We are actively monitoring the recovery. If your deployment failed due to “Container failed to start" error, please attempt a re-deploy.

  4. resolved Apr 18, 2025, 01:58 PM UTC

    This incident has been resolved.

Your product runs on more than Railway

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →