LogRocket incident

Session Ingestion Delays

Major Resolved View vendor source →

LogRocket experienced a major incident on June 3, 2021 affecting Session Collection and Alerts, lasting 12h 13m. The incident has been resolved; the full update timeline is below.

Started
Jun 03, 2021, 03:52 PM UTC
Resolved
Jun 04, 2021, 04:06 AM UTC
Duration
12h 13m
Detected by Pingoru
Jun 03, 2021, 03:52 PM UTC

Affected components

Session CollectionAlerts

Update timeline

  1. investigating Jun 03, 2021, 03:52 PM UTC

    We are currently investigating an issue with session ingestion.

  2. identified Jun 03, 2021, 04:40 PM UTC

    The issue impacting ingestion has been identified. Session ingestion is currently delayed.

  3. monitoring Jun 03, 2021, 04:51 PM UTC

    A fix has been implemented. We are monitoring closely as we begin to process pending sessions.

  4. monitoring Jun 03, 2021, 06:13 PM UTC

    We are continuing to monitor as we process pending sessions. We expect session ingestion to be real-time within the hour, followed by secondary processing and alerting.

  5. identified Jun 03, 2021, 06:59 PM UTC

    A new issue has been identified and we're working with our vendor to resolve the issue so we can resume processing of delayed sessions.

  6. monitoring Jun 03, 2021, 11:09 PM UTC

    Operations have been restored with our primary search cluster. We have resumed processing of delayed sessions.

  7. monitoring Jun 04, 2021, 01:55 AM UTC

    Primary session processing is now real-time. Secondary processing of session data continues, and will complete over the next hours.

  8. resolved Jun 04, 2021, 04:06 AM UTC

    Systems have remained stable while continuing to process the backlog of events. We estimate secondary processing will be live in the next 6 hours.