Viedoc incident

Login issues on EU production instance

Major Resolved View vendor source →

Viedoc experienced a major incident on November 3, 2025, lasting —. The incident has been resolved; the full update timeline is below.

Started
Nov 03, 2025, 01:27 PM UTC
Resolved
Nov 02, 2025, 10:30 PM UTC
Duration
Detected by Pingoru
Nov 03, 2025, 01:27 PM UTC

Update timeline

  1. resolved Nov 03, 2025, 01:27 PM UTC

    From approximately 23.44 UTC to 23.54 UTC, users faced issues logging in to the European production instance. Investigations to find the root cause are ongoing. No other instances were affected.

  2. postmortem Nov 10, 2025, 10:45 AM UTC

    Description From approximately 23.44 UTC to 23.54 UTC on 2025-11-02, users were unable to log in to the European production instance. It is possible that users faced sporadic issues logging in also between approximately 23.00 UTC and 23.44 UTC. No other instances were affected. ‌ Cause The cause of the issue was an unexpected error in the identity provider \(idp\), leading to an accumulating memory consumption during the evening \(UTC\) of 2025-11-02. From approximately 23.00 UTC users would face sporadic failures when trying to log in. From approximately 23.44 UTC users could not log in. At 23.54 UTC on 2025-11-02 the idp was restarted, and users could log in as expected again. ‌ Corrective Action Alerts were triggered when the idp did not respond and the idp was restarted again at 2354 UTC on 2025-11-02. After the restart users could log in as expected again. ‌ Preventive Action Preliminary investigation identified a few improvements that could be made to make the idp more resilient. Implementation of these improvements have been initiated while the investigation of the issue continues.