Lucky Orange experienced a major incident on January 11, 2022 affecting Data Collection, lasting 4h 19m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 11, 2022, 07:51 PM UTC
We are investigating an issue where customers storing data for 30 days are having an issue loading data and new data is delayed.
- identified Jan 11, 2022, 08:15 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Jan 11, 2022, 09:38 PM UTC
We have implemented a fix and the system is now working through the backlog of data. At this time, no data has been lost and all sessions and events will be visible in the dashboard soon.
- resolved Jan 12, 2022, 12:11 AM UTC
This incident has been resolved and no data was lost. We identified that a configuration change with one of our databases caused it to crash. We have updated the database to the correct configuration to prevent this issue from happening again. We are also looking into solutions to prevent the impact of similar situations in the future. Thank you for your patience as we worked through this!