Highlight.io incident

Degraded Trace Ingest

Minor Resolved View vendor source →

Highlight.io experienced a minor incident on February 12, 2024, lasting 1d 11h. The incident has been resolved; the full update timeline is below.

Started
Feb 12, 2024, 09:28 PM UTC
Resolved
Feb 14, 2024, 08:52 AM UTC
Duration
1d 11h
Detected by Pingoru
Feb 12, 2024, 09:28 PM UTC

Update timeline

  1. investigating Feb 12, 2024, 09:28 PM UTC

    We are investigating a delay in ingesting traces since 12:30 PT.

  2. resolved Feb 14, 2024, 08:52 AM UTC

    Though this has intermittently occuring, we've identified the root cause of the issue (a misconfiguration with our opentelemetry collector which was exporting to our public graph over the internet rather than within the VPC) and have resolved this.