Telerivet incident

Intermittent errors with message queue

Notice Resolved View vendor source →

Telerivet experienced a notice incident on February 8, 2017, lasting 11h 40m. The incident has been resolved; the full update timeline is below.

Started
Feb 08, 2017, 09:17 AM UTC
Resolved
Feb 08, 2017, 08:57 PM UTC
Duration
11h 40m
Detected by Pingoru
Feb 08, 2017, 09:17 AM UTC

Update timeline

  1. investigating Feb 08, 2017, 09:17 AM UTC

    We are currently investigating intermittent errors with the message queue.

  2. monitoring Feb 08, 2017, 09:45 AM UTC

    The errors with the message queue appear to have been resolved, however we are still investigating the root cause.

  3. resolved Feb 08, 2017, 08:57 PM UTC

    It seems likely that the intermittent errors were caused by a new setting we were testing for certain message queues, including some queues that were processing a high number of messages when the errors occurred. We have disabled this setting for these queues and will continue monitoring to see if the errors recur.