Next Identity incident

Latency Issue with Next Identity Journeys

Notice Resolved View vendor source →

Next Identity experienced a notice incident on September 28, 2022 affecting Journeys, lasting 6h 51m. The incident has been resolved; the full update timeline is below.

Started
Sep 28, 2022, 07:04 PM UTC
Resolved
Sep 29, 2022, 01:56 AM UTC
Duration
6h 51m
Detected by Pingoru
Sep 28, 2022, 07:04 PM UTC

Affected components

Journeys

Update timeline

  1. investigating Sep 28, 2022, 07:04 PM UTC

    Next Reason support team received a report that the Next Identity Journeys platform had experienced a brief p99 latency increase. The Engineering team is currently investigating the issue. Additional updates will be shared once available.

  2. identified Sep 28, 2022, 08:33 PM UTC

    We have identified an issue with Next Identity Journey that is causing a p99 latency increase which is related to Akamai. We have determined that the API Rate Limit was reached causing the latency. We are now working with Akamai to implement a fix on this issue.

  3. monitoring Sep 28, 2022, 09:03 PM UTC

    The fix has been implemented and we are monitoring the results. Akamai has temporarily increased the API Rate Limit to allow unusual traffic caused by activities on client side. We have coordinated with the client to check the recent activities causing the surge.

  4. monitoring Sep 29, 2022, 01:25 AM UTC

    We are continuously monitoring update from client for the cause of spike on API endpoint.

  5. resolved Sep 29, 2022, 01:56 AM UTC

    This incident has been resolved. Akamai confirmed traffic on endpoint is already back to normal.