Scalyr incident

Ingestion and query issues on app.us1.dataset.com

Notice Resolved View vendor source →

Scalyr experienced a notice incident on December 6, 2023, lasting 3h 21m. The incident has been resolved; the full update timeline is below.

Started
Dec 06, 2023, 03:35 AM UTC
Resolved
Dec 06, 2023, 06:56 AM UTC
Duration
3h 21m
Detected by Pingoru
Dec 06, 2023, 03:35 AM UTC

Update timeline

  1. identified Dec 06, 2023, 03:35 AM UTC

    Customers on app.us1.dataset.com are experiencing delayed ingestion and query issues. We've identified the root cause of the issue and are actively working to mitigate the impact at this time.

  2. identified Dec 06, 2023, 04:57 AM UTC

    Query performance has been restored as of 04:20 GMT. Ingest recovery is still in progress.

  3. monitoring Dec 06, 2023, 05:15 AM UTC

    We have implemented remediations to address these issues. A large ingest queue remains, so some ingestion delays may continue for the immediate future. We are continuing to monitor.

  4. resolved Dec 06, 2023, 06:56 AM UTC

    This incident has been resolved.