Mayan incident

Airflow ORANGE Azure is not available due to "Internal Server Error"

Notice Resolved View vendor source →

Mayan experienced a notice incident on January 22, 2024 affecting Airflow (batch), lasting 1d. The incident has been resolved; the full update timeline is below.

Started
Jan 22, 2024, 04:29 PM UTC
Resolved
Jan 23, 2024, 05:18 PM UTC
Duration
1d
Detected by Pingoru
Jan 22, 2024, 04:29 PM UTC

Affected components

Airflow (batch)

Update timeline

  1. investigating Jan 22, 2024, 04:29 PM UTC

    We are currently investigating this issue.

  2. investigating Jan 22, 2024, 04:31 PM UTC

    Microsoft appears to have some internal issues on Azure Will continue to monitor and update as new information becomes available.

  3. identified Jan 22, 2024, 05:42 PM UTC

    The issue lies with an internal database that supports the Airflow Instance login session credentials. Until Microsoft is able to address this, we will likely see a continuing failure of this instance. We have asked them to address the issue but at their scale this takes days to resolve.

  4. identified Jan 23, 2024, 12:51 PM UTC

    Although the server is still down, we have been running workflows locally to avoid service disruptions.

  5. resolved Jan 23, 2024, 05:18 PM UTC

    All DAGs now available on Production-GREEN