Proxyclick incident

Intermittent failures of Login Attempts for Dashboard

Notice Resolved View vendor source →

Proxyclick experienced a notice incident on May 26, 2023, lasting —. The incident has been resolved; the full update timeline is below.

Started
May 26, 2023, 04:22 PM UTC
Resolved
May 25, 2023, 03:30 AM UTC
Duration
—
Detected by Pingoru
May 26, 2023, 04:22 PM UTC

Update timeline

  1. resolved May 26, 2023, 04:22 PM UTC

    We are currently investigating this issue.

  2. postmortem May 26, 2023, 04:23 PM UTC

    **Root Cause Analysis:** Between 05:35 to 07:10 login attempts for the admin dashboard were intermittently unsuccessful for certain users. We identified that one API node failed to correctly initiaize, this was logged by our monitoring platform and subsequently restarted. During this process 20% of login authentication attempts to the platform returned an error to end users, with additional attempts typically being successful. Further investigation showed a surge load access to the API node which led to a soft crash \(API running but not responding to all requests\).