Kindly experienced a minor incident on January 15, 2025 affecting Analytics dashboard, lasting 1d 1h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Jan 15, 2025, 05:20 PM UTC
During a routing maintenance upgrade of the analytics and unexpected event happened that requires us to reboot the database service. While this is ongoing analytics dashboards may be unavailable for some time. During this, all chatbot operations work as normal and all analytics events are recorded in a queue and no data is lost. Only the ability to view the data will be temporarily unavailable.
- monitoring Jan 16, 2025, 12:15 AM UTC
It has been determined that there is a bug in the analytics database that affects the behavior for how Kindly calculates sessions. We have an open ticket with the database provider where they have acknowledged this bug and are working on a fix, but they don't know how long it will take. In order to restore most of the functionality of the database we have disabled a part of the sessions calculation before resuming normal analytics operations. This will affect stats in dashboards that measure sessions. We will resume ingesting analytics events to still have accurate statistics about messages, fallbacks, labels and all stats that are not derived from sessions. Once a bugfix is available, we plan use the accurate data to recreate the sessions data for the affected period and have accurate sessions statistics, but in the meantime the charts that show sessions may not reflect accurate numbers.
- resolved Jan 16, 2025, 06:47 PM UTC
After a normal business day we have looked at the numbers and they seem to be normal. We will continue to monitor the sessions numbers to correct any discrepancies that might have occurred because of the database update issue.