Flxpoint incident

Flxpoint API : Intermittent 401 Response Codes

Minor Resolved View vendor source →

Flxpoint experienced a minor incident on July 6, 2023 affecting API, lasting 1h 11m. The incident has been resolved; the full update timeline is below.

Started
Jul 06, 2023, 06:59 AM UTC
Resolved
Jul 06, 2023, 08:11 AM UTC
Duration
1h 11m
Detected by Pingoru
Jul 06, 2023, 06:59 AM UTC

Affected components

API

Update timeline

  1. investigating Jul 06, 2023, 06:59 AM UTC

    Although we've restored service we are still experiencing intermittent 401s on Flxpoint API. This is being investigated

  2. identified Jul 06, 2023, 07:09 AM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Jul 06, 2023, 07:30 AM UTC

    This issue we identified has been resolved and we are observing API traffic returning to normal. The incident started from 2:39am and was gradually resolved for users until around 3:19am EST. During this time, you may have been logged out of the application or you may have received intermittent 401 response codes via our API. The outage we experienced earlier tonight was a result of a failure in one of our authentication cache nodes. The remediation for this involved creating a new cache node to replace it. Unfortunately this new node was not accessible by our application servers without additional intervention. Due to the nature of the problem, this resulted in intermittent failures which were not detected when closing out the prior incident. We will continue to monitor until can confirm a full recovery.

  4. resolved Jul 06, 2023, 08:11 AM UTC

    This incident has been resolved.