US - Studio Web - Outage

Update timeline

1. investigating Mar 13, 2026, 12:57 PM UTC

   We are currently investigating an issue impacting Studio Web in the US region. Further updates will be provided shortly.

2. monitoring Mar 13, 2026, 01:08 PM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Mar 13, 2026, 01:14 PM UTC

   This incident has been resolved.

4. postmortem Mar 18, 2026, 03:12 AM UTC

   ## Customer Impact On March 13, 2026, between 12:31 and 13:09 UTC, customers using Studio Web in the U.S. environment experienced a service outage lasting approximately 38 minutes. Users encountered HTTP 503 errors and were unable to access Studio Web during this period. The incident occurred at 11:31 am UTC, before the start of U.S. business hours, which limited the number of affected customers. Based on our analysis, a small number of customers experienced failed requests during the outage window. ## Root Cause A planned infrastructure change updated the network configuration of our enterprise U.S. production environment, changing the outbound IP address used by the service. The new IP address was not added to the access rules of a critical storage resource that the service depends on. When service instances restarted, they were unable to connect to this storage resource because the new IP address was blocked by the existing network access rules. The dependency between the environment's outbound IP and the storage resource's access rules was not tracked in the migration process, and there was no automated validation to verify that all dependent resources had been updated before the migration proceeded. ## Detection The issue was detected automatically by our monitoring systems within minutes of the service becoming unavailable. Our on-call engineering team was engaged immediately after being alerted. ## Response Our engineering team identified the root cause within 22 minutes of engagement by analyzing service logs and correlating them with the ongoing infrastructure migration. The issue was resolved at 13:06 UTC by updating the storage resource's network access rules to allow the new IP address. Full service restoration was confirmed at 13:09 UTC. Total time from first alert to confirmed resolution: 38 minutes. ## Follow-Up 1. Implement validation for such specific edge-cases before proceeding with further infrastructure changes. 2. Audit all storage resources and network-restricted dependencies across all regions to identify and remediate similar configuration gaps before resuming the migration. 3. Implement automated pre-migration validation that verifies all dependent resources have been updated with new network configurations. 4. Add dedicated monitoring for storage connectivity failures to enable faster detection of this specific failure mode.