Simplr incident

Database reads time increased causing platform issues

Minor Resolved View vendor source →

Simplr experienced a minor incident on November 27, 2020, lasting —. The incident has been resolved; the full update timeline is below.

Started
Nov 27, 2020, 07:59 PM UTC
Resolved
Nov 27, 2020, 07:00 PM UTC
Duration
—
Detected by Pingoru
Nov 27, 2020, 07:59 PM UTC

Update timeline

  1. resolved Nov 27, 2020, 07:59 PM UTC

    Database performance degradation causing platform load issues

  2. postmortem Nov 27, 2020, 07:59 PM UTC

    An incorrectly configured external service webhook caused extra load on the secondary database causing cascading read issues on all database instances. Disabling of the webhook and removing bad jobs fixed the issue. Platform is back to normal. No email or chat was lost. Chat reply was delayed.