New Relic incident

Infrastructure Alerts- False Host Not Reporting

Minor Resolved View vendor source →

New Relic experienced a minor incident on March 18, 2025 affecting Infrastructure : US and Infrastructure Agent : US and 1 more component, lasting 2d 4h. The incident has been resolved; the full update timeline is below.

Started
Mar 18, 2025, 10:01 AM UTC
Resolved
Mar 20, 2025, 02:20 PM UTC
Duration
2d 4h
Detected by Pingoru
Mar 18, 2025, 10:01 AM UTC

Affected components

Infrastructure : USInfrastructure Agent : USInfrastructure : US

Update timeline

  1. investigating Mar 18, 2025, 10:01 AM UTC

    We are investigating a potential service interruption that may impact some customers. Please continue to monitor this page for updates.

  2. investigating Mar 18, 2025, 10:04 AM UTC

    We are continuing to investigate this issue.

  3. identified Mar 18, 2025, 10:36 AM UTC

    The issue has been identified and a fix is being implemented.

  4. identified Mar 18, 2025, 01:45 PM UTC

    We are continuing to work on a fix for this issue. This service interruption may impact some customers in in our US region

  5. identified Mar 18, 2025, 05:26 PM UTC

    We are investigating an issue affecting the reporting of infrastructure entity metadata from New Relic agents. Some customers in the US region may experience irregularities in charts, dashboards, and alert notifications for affected entities. New Relic teams have implemented a fix to restore affected reporting; however, additional steps are required to refresh existing entities and their visibility in the New Relic UI. Our teams are investigating several paths to complete that process and will continue to provide updates as they are available.

  6. identified Mar 18, 2025, 08:08 PM UTC

    Our teams are working to resolve an issue which has resulted in missing metadata for some infrastructure entities. Affected entities will be refreshed automatically through standard, automated mechanisms at various intervals; however, our teams are continuing to investigate opportunities to expedite the refresh for affected entities. We will provide our next update at 23:00 UTC or upon significant change.

  7. identified Mar 18, 2025, 11:09 PM UTC

    We are working to resolve the issue resulting in missing metadata for some infrastructure entities. Through standard operations, affected entities will be refreshed over the next 24 - 48 hours automatically. Additionally, our teams are continuing to investigate opportunities to expedite the refresh of metadata for affected entities, including restoring metadata from backups. We will provide our next update at 14:00 UTC on March 19th or upon significant change.

  8. identified Mar 19, 2025, 01:58 PM UTC

    We are working to resolve the issue resulting in missing metadata for some infrastructure entities. We are seeing recovery of some affected entities through standard agent refresh behaviors and our teams continue to prioritize expediting recovery of remaining affected entity metadata. We will provide our next update at 23:00 UTC or upon significant change.

  9. identified Mar 19, 2025, 11:01 PM UTC

    We are seeing continued restoration of affected entity metadata through both standard agent refresh behaviors, as well as backup and recovery actions taken by our teams. Some customers in the US region may experience irregularities in charts, dashboards, and alert notifications for affected entities until full recovery has completed. We will provide our next update at 14:00 UTC on March 20th or upon significant change.

  10. resolved Mar 20, 2025, 02:20 PM UTC

    The restoration of affected entity metadata has now been completed. Impacted services have returned to normal operations.