Lucky Orange experienced a major incident on August 16, 2022 affecting Data Collection, lasting 1h 43m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 16, 2022, 11:38 PM UTC
We are investigating an issue where data is not currently being collected from browser visitors.
- identified Aug 17, 2022, 12:34 AM UTC
We have identified that both our primary and backup data collection queues were taken offline. During this time, data was unable to be collected by Lucky Orange and is unrecoverable. The backup queue has been moved to secondary infrastructure to mitigate the failure and is now collecting new data. We are still working to mitigate the failure of our primary queue. We are also still investigating what could cause a double outage like this. Additionally, our alerts that are in place to notify us when a queue is unavailable never triggered. Updates will be provided as we know more.
- resolved Aug 17, 2022, 01:21 AM UTC
The primary incident has been resolved and data is being collected properly. We will continue to investigate the cause and share more information soon.