Auvik incident

Service Disruption - Performance issues with clients on the US4 cluster.

Minor Resolved View vendor source →

Auvik experienced a minor incident on June 3, 2024 affecting us4.my.auvik.com, lasting 2h 1m. The incident has been resolved; the full update timeline is below.

Started
Jun 03, 2024, 07:33 PM UTC
Resolved
Jun 03, 2024, 09:34 PM UTC
Duration
2h 1m
Detected by Pingoru
Jun 03, 2024, 07:33 PM UTC

Affected components

us4.my.auvik.com

Update timeline

  1. investigating Jun 03, 2024, 07:33 PM UTC

    We’re experiencing a performance disruption with services for clients on the US4 cluster. It is related to the incident earlier today. Tenants in the other clusters are not affected. We apologize for the continued issues. We will continue to provide updates as they become available.

  2. identified Jun 03, 2024, 07:56 PM UTC

    We've identified the source of the performance disruption with services for clients on the US4 cluster. It is related to the incident earlier today. Tenants in the other clusters are not affected. The will begin a maintenance window at 20:00 UTC (16:00 EDT) where updates to the UI and alerting will be delayed. We apologize for the continued issues. We will continue to provide updates as they become available.

  3. monitoring Jun 03, 2024, 08:24 PM UTC

    We've identified the source of the performance disruption with services for clients on the US4 cluster. It is related to the incident earlier today. Tenants in the other clusters are not affected. The maintenance window began at 20:00 UTC (16:00 EDT), during which updates to the UI and alerts will be delayed. This maintenance window is expected to last for around 90 minutes. We apologize for the continued issues. We will continue to provide updates as they become available..

  4. resolved Jun 03, 2024, 09:34 PM UTC

    We've applied the fix for the performance disruption with services for clients on the US4 cluster. It is related to the incident earlier today. Tenants in the other clusters are not affected. There may be some associated lag as the system comes up to full speed. Alerting will become active for US4 clients at 22:00 UTC (18:00 EDT). We apologize for the issues today. The source of the disruption has been resolved. Services have been fully restored. A Root Cause Analysis (RCA) will follow after a full review and posted to the original incident.

  5. postmortem Jun 20, 2024, 02:52 PM UTC

    Please see [Performance Issue - Map Rendering Delayed on US Cluster Customers](https://status.auvik.com/incidents/6t543bp2xdwr) for post mortem.