Lexer incident

Respond and Listen returning errors

Major Resolved View vendor source →

Lexer experienced a major incident on May 18, 2021 affecting Conversations, lasting 1h 59m. The incident has been resolved; the full update timeline is below.

Started
May 18, 2021, 11:30 PM UTC
Resolved
May 19, 2021, 01:29 AM UTC
Duration
1h 59m
Detected by Pingoru
May 18, 2021, 11:30 PM UTC

Affected components

Conversations

Update timeline

  1. investigating May 18, 2021, 11:50 PM UTC

    We are currently investigating reports of errors when loading the Respond inbox and Listen results.

  2. identified May 18, 2021, 11:54 PM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring May 19, 2021, 12:01 AM UTC

    A fix has been implemented and we are monitoring the results.

  4. resolved May 19, 2021, 01:29 AM UTC

    This incident has been resolved. We have identified performance degradation in one of our database systems and have redeployed the system onto new hardware with additional resources available to it. Our monitoring now indicates that normal service has been restored. We will be continuing our investigation into the performance degradation to mitigate the impact of future incidents.