Bigeye experienced a minor incident on February 15, 2024 affecting API and UI, lasting 5d 6h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Feb 15, 2024, 08:05 PM UTC
We are investigating issues with invalid metric results for some metric configurations. We are temporarily pausing scheduled runs while we are investigating.
- investigating Feb 15, 2024, 08:36 PM UTC
Scheduled metric runs have been re-enabled. We are still investigating issues with invalid metric results for data time window grouped metrics.
- identified Feb 15, 2024, 09:38 PM UTC
The issue has been identified and a fix is being implemented.
- identified Feb 16, 2024, 02:55 AM UTC
We have fixed the problem and are correcting metrics that are missing portions of metric run history with an ETA of tomorrow (Feb 16).
- identified Feb 17, 2024, 01:43 AM UTC
The forward fix has been deployed and verified. All systems are operational. We are continuing to reprocess any historical metric runs which were affected by this issue
- identified Feb 20, 2024, 06:17 PM UTC
We are continuing to reprocess the remaining affected historical metric runs
- resolved Feb 21, 2024, 02:58 AM UTC
The incident has been resolved. All services are back online and healthy.