Keysight Eggplant incident

Increased test failures

Notice Resolved View vendor source →

Keysight Eggplant experienced a notice incident on November 30, 2023 affecting Testing, lasting 9h 30m. The incident has been resolved; the full update timeline is below.

Started
Nov 30, 2023, 02:09 AM UTC
Resolved
Nov 30, 2023, 11:40 AM UTC
Duration
9h 30m
Detected by Pingoru
Nov 30, 2023, 02:09 AM UTC

Affected components

Testing

Update timeline

  1. investigating Nov 30, 2023, 02:09 AM UTC

    We are investigating an increase in test failures on Monitoring Insights.

  2. monitoring Nov 30, 2023, 03:37 AM UTC

    Platform has returned to stability but further investigations are underway to understand the root cause.

  3. monitoring Nov 30, 2023, 10:52 AM UTC

    We will be restaring test agents in a phased manner to address the cause of this issue. No interruption to service is expected.

  4. resolved Nov 30, 2023, 11:40 AM UTC

    The root cause of this was tracked down to a memory management issue in test agents due few customer tests got overloaded. We have restarted the services to stabilise the platform. The team is working on a fix to improve and prevent this happening in future.