Sekoia FRA2 incident

Event storage cluster experiencing lag

Minor Resolved View vendor source →

Sekoia FRA2 experienced a minor incident on July 23, 2025 affecting Event storage, lasting 18h 43m. The incident has been resolved; the full update timeline is below.

Started
Jul 23, 2025, 12:38 PM UTC
Resolved
Jul 24, 2025, 07:21 AM UTC
Duration
18h 43m
Detected by Pingoru
Jul 23, 2025, 12:38 PM UTC

Affected components

Event storage

Update timeline

  1. identified Jul 23, 2025, 12:38 PM UTC

    We are currently experiencing lag on our FRA2 event storage cluster due to an unexpected increase in traffic. Our engineers have taken immediate action by increasing the performance of the cluster to handle this traffic. We are in the process of contacting the source of this traffic to understand the cause of this sudden surge. Please note that performance might be slower than usual during this time. We appreciate your patience and understanding while we work towards resolving this issue.

  2. identified Jul 23, 2025, 01:15 PM UTC

    Steps have been taken to mitigate the high traffic events causing this issue. The platform is now catching up slowly on the backlog of events to process. We appreciate your continued patience and understanding.

  3. investigating Jul 23, 2025, 04:14 PM UTC

    The original incident causing lag on the FRA2 event storage cluster has been successfully resolved. The backlog has been fully consumed at 17:34 CEST However, we are currently experiencing slow disk operations on the storage cluster, linked to a maintenance operation on the region, that causes lag on the cluster again since 17:44 CEST. This is unexpected as these operations are not typically impactful. We are actively investigating this issue and working towards a resolution. We apologize for any inconvenience this may cause and appreciate your understanding.

  4. resolved Jul 24, 2025, 07:21 AM UTC

    The storage cluster has fully consumed the backlog at 20:50 CEST. This incident has been resolved.