SolarWinds Observability incident

Degraded Data Ingestion [NA-01]

Notice Resolved View vendor source →

SolarWinds Observability experienced a notice incident on November 6, 2025, lasting —. The incident has been resolved; the full update timeline is below.

Started
Nov 06, 2025, 03:00 PM UTC
Resolved
Nov 06, 2025, 03:00 PM UTC
Duration
Detected by Pingoru
Nov 06, 2025, 03:00 PM UTC

Update timeline

  1. resolved Nov 07, 2025, 05:20 PM UTC

    Between 6:30 AM and 5:00 PM UTC on November 6, we experienced an intermittent data ingestion issue that primarily impacted Heroku add-on customers, who may have seen logplex errors and missing log data. The root cause was a database slowdown that caused timeouts in our authentication service. Our service incorrectly returned a 401 Unauthorized error instead of a 500 error, which likely prevented client systems like Heroku from retrying the request and resulted in data loss for approximately 1% of Heroku ingestion requests. We resolved the underlying database performance issue at 5:00 PM UTC, restoring normal data ingestion. We are also rolling out a preventative fix to ensure our service correctly returns a 500 error during any future server-side timeouts, signaling to clients that they should retry the data submission.