BigPanda experienced a notice incident on January 18, 2024 affecting Data Consumption, lasting 4h 38m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 18, 2024, 12:14 PM UTC
Some users may be experiencing data pipeline latencies. Our teams are engaged and investigating. What's impacted? • Data Pipeline - All data is being accepted into the BigPanda platform, but we are experiencing delays within pipeline services. Due to this, updates to users' Incident Feeds may be delayed. What’s not impacted? Other areas of the platform - including data ingest, analytics, incident sharing, user actions - should not be impacted. We are working to mitigate impacts to our customers as quickly as possible. Additional updates will be shared every 30 minutes or less as more details become available. We apologize for any inconvenience that this may be causing.
- investigating Jan 18, 2024, 12:23 PM UTC
We are continuing to investigate this issue.
- investigating Jan 18, 2024, 01:01 PM UTC
The team have performed a deployment rollback and a manual restart. We are currently seeing throttler crashes that we are currently investigating.
- investigating Jan 18, 2024, 01:51 PM UTC
We are making progress on investigation and we are continuing to work on this issue to mitigate impacts to our customers as quickly as possible.
- monitoring Jan 18, 2024, 02:59 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jan 18, 2024, 04:53 PM UTC
This incident has been resolved.