Netdata incident

Node status updates are delayed

Minor Resolved View vendor source →

Netdata experienced a minor incident on May 12, 2023 affecting Cloud Web UI, lasting 1h 12m. The incident has been resolved; the full update timeline is below.

Started
May 12, 2023, 07:30 AM UTC
Resolved
May 12, 2023, 08:42 AM UTC
Duration
1h 12m
Detected by Pingoru
May 12, 2023, 07:30 AM UTC

Affected components

Cloud Web UI

Update timeline

  1. investigating May 12, 2023, 07:30 AM UTC

    We had a problem with updating node status on Cloud UI. In practice it means that there was a delay between node changing status from for example online to offline and cloud realising this fact. This would also affect new agents or deleting an existing one. Due to connection update delay there is also a delay in chart metadata updates in our database with which we are dealing right now. This on the other hand from user perspective means, that charts for newly installed applications or charts in general for new nodes are not yet showing up on the Cloud UI.

  2. identified May 12, 2023, 07:30 AM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring May 12, 2023, 08:33 AM UTC

    A fix has been implemented and we are monitoring the results.

  4. resolved May 12, 2023, 08:42 AM UTC

    This incident has been resolved.