Highlight.io experienced a minor incident on January 16, 2024 affecting Public Graph, lasting 13h 25m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 16, 2024, 04:34 PM UTC
We've started to experience a delay in our session / errors ingest. We're working on a fix as we continue to intermittently process sessions. Most sessions are delayed until the issue is fixed. Updates to follow.
- identified Jan 16, 2024, 05:24 PM UTC
A fix to limit our kafka prefetching (which has been overloading a broker) has been deployed and is keeping the cluster stable. We're quickly catching up on the data backlog.
- monitoring Jan 16, 2024, 09:13 PM UTC
The fix has proven stable. We're catching up on previous data with an ETA to catch up of 22:00 PT tonight.
- resolved Jan 17, 2024, 06:00 AM UTC
We have caught up on the outstanding data backlog. Apologies for the inconvenience caused.