Orchestry incident

Global Issue with Azure

Minor Resolved View vendor source →

Orchestry experienced a minor incident on October 29, 2025 affecting Web Application and API, lasting 8h 34m. The incident has been resolved; the full update timeline is below.

Started
Oct 29, 2025, 04:00 PM UTC
Resolved
Oct 30, 2025, 12:35 AM UTC
Duration
8h 34m
Detected by Pingoru
Oct 29, 2025, 04:00 PM UTC

Affected components

Web ApplicationAPI

Update timeline

  1. investigating Oct 29, 2025, 04:00 PM UTC

    We are aware of a global issue affecting Azure that may cause intermittent access issues or degraded performance for some users. The problem originates with Microsoft’s infrastructure and is impacting multiple regions. Our team is monitoring the situation closely and will provide updates as Microsoft releases more information.

  2. investigating Oct 29, 2025, 04:40 PM UTC

    We are continuing to monitor this issue for updates from Microsoft. Microsoft's latest update states that, "Starting at approximately 16:00 UTC, we began experiencing DNS issues resulting in availability degradation of some services. Customers may experience issues accessing the Azure Portal. We have taken action that is expected to address the portal access issues here shortly. We are actively investigating the underlying issue and additional mitigation actions." However, the incident appears to be affecting Azure Front Door in addition to Azure Portal. The outage is wide spread affecting regions globally. We will continue to update our status as our team learns more.

  3. identified Oct 29, 2025, 05:20 PM UTC

    The latest update from Microsoft confirms that it is DNS affecting Azure Front Door and that they are rolling back recent changes to resolve the issue, "We are taking two concurrent actions where we are blocking all changes to the AFD services and disabling a problematic route that we found to be related to this, and at the same time rolling back to our last known good state ... We do not have an ETA for when the rollback will be completed, but we will update this communication within 30 minutes or when we have an update." We will update our status when we have confirmed that Microsoft's rollback has resolved the issue. In the meantime we continue to monitor.

  4. monitoring Oct 29, 2025, 09:08 PM UTC

    Microsoft confirmed an Azure Front Door issue caused by a configuration change that led to latency and errors across multiple Azure services. They’ve rolled back the change and are rerouting traffic through healthy nodes, with recovery ongoing. Note that as recovery progresses, some requests may still land on unhealthy nodes, resulting in intermittent failures or reduced availability until more nodes are fully restored. Our services are stabilizing and performing normally again. We’ll continue to monitor until Microsoft confirms full recovery.

  5. resolved Oct 30, 2025, 12:35 AM UTC

    Microsoft has confirmed that the Azure Front Door issues have been resolved. Orchestry services are fully functional.

  6. postmortem Oct 30, 2025, 02:29 PM UTC

    ## Summary On October 29, 2025, Microsoft experienced a global issue affecting its Azure Front Door service, which handles traffic routing and content delivery for many cloud applications worldwide. This disruption caused intermittent slowdowns and brief unavailability across parts of the Orchestry platform. ## Impact Some users may have noticed temporary difficulty accessing Orchestry services or experienced slower page loads during the outage window. There was no impact to data integrity, security, or customer configurations, and all services continued to operate normally once Microsoft resolved the issue. ## Resolution Microsoft completed a rollback and restored full functionality to Azure Front Door by 00:40 UTC on October 30. Orchestry systems recovered at that time. ## Next Steps We continue to enhance our monitoring and resiliency measures to further minimize any future impact from upstream provider incidents. We appreciate your patience and understanding during this event.