Scout APM incident

Time Series Database Issue

Critical Resolved View vendor source →

Scout APM experienced a critical incident on April 17, 2018, lasting 3h 8m. The incident has been resolved; the full update timeline is below.

Started
Apr 17, 2018, 04:50 PM UTC
Resolved
Apr 17, 2018, 07:59 PM UTC
Duration
3h 8m
Detected by Pingoru
Apr 17, 2018, 04:50 PM UTC

Update timeline

  1. investigating Apr 17, 2018, 04:50 PM UTC

    The backend time-series database appears to be having issues. All incoming data is being buffered and will be ingested into the system, but the site is currently inaccessible.

  2. monitoring Apr 17, 2018, 05:20 PM UTC

    The time-series database is restarting and should be operational in a few minutes. After which buffered data from the downtime will be played into it.

  3. monitoring Apr 17, 2018, 05:59 PM UTC

    Our systems are replaying buffered data collected during the outage and ingesting these into our database.

  4. resolved Apr 17, 2018, 07:59 PM UTC

    Metric ingestion has caught back up.