Netdata incident

Degraded chart syncing performance

Minor Resolved View vendor source →

Netdata experienced a minor incident on May 14, 2022 affecting Cloud Web UI and Agent - Cloud Connection (ACLK), lasting 4d 13h. The incident has been resolved; the full update timeline is below.

Started
May 14, 2022, 07:57 PM UTC
Resolved
May 19, 2022, 09:26 AM UTC
Duration
4d 13h
Detected by Pingoru
May 14, 2022, 07:57 PM UTC

Affected components

Cloud Web UIAgent - Cloud Connection (ACLK)

Update timeline

  1. investigating May 14, 2022, 07:57 PM UTC

    We're investigating an issue with slow chart syncing

  2. investigating May 14, 2022, 10:23 PM UTC

    Some users reported missing charts in the nodes tab and charts for older metrics that are not currently recording data. We found a synchronization issue that affects about 5% of nodes. No (meta)data is lost, and we are working on a solution.

  3. identified May 16, 2022, 08:45 AM UTC

    We are working on a fix now.

  4. monitoring May 18, 2022, 09:19 AM UTC

    We've applied the fix. We carefully monitor the situation now.

  5. resolved May 19, 2022, 09:26 AM UTC

    This incident has been resolved.