Dixa experienced a minor incident on May 15, 2024 affecting Agent Interface, lasting 57m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 15, 2024, 05:09 AM UTC
We have received reports of instability in the platform. We are investigating the issue. Updates will follow
- identified May 15, 2024, 05:12 AM UTC
Agents may have trouble using the Dixa platform right now due to a planned maintenance which was regrettably not communicated. Please have your agents refresh Dixa's Agent Interface. We're monitoring the situation. If you continue to have issues, please reach out Dixa Support.
- investigating May 15, 2024, 05:21 AM UTC
We're still seeing slow loading times. We're investigating the issue.
- identified May 15, 2024, 05:35 AM UTC
We've identified the problem and are working on a solution. If you're still having trouble logging in, refreshing the loading page several times may work. We sincerely apologize for the inconvenience this is causing. Next update at 07:45 am CEST
- identified May 15, 2024, 05:45 AM UTC
We're still trying to find a solution that has the least amount of impact possible for already connected agents. We'll keep you updated, next update at 07:55 AM CEST. We again apologize for the inconvenience.
- monitoring May 15, 2024, 05:51 AM UTC
We have implemented a fix and are seeing improvement. If you're still not logged in, please refresh Dixa or try to log in again. If you're still unable to access Dixa after this, please reach out to Dixa Support via [email protected] or via the live chat in the Dixa agent interface via a signed in co-worker.
- resolved May 15, 2024, 06:07 AM UTC
This incident has been resolved.
- postmortem May 16, 2024, 11:33 AM UTC
On Wednesday, May 15, at 6 AM CEST, Dixa engineers started their planned maintenance by upgrading some backend services. It was expected that existing agent connections would be closed and agents would have to reload the agent interface. This was sadly not communicated on the status page due to a miscommunication between teams. Around 6.37 AM CEST we received the first report of a Dixa customer having trouble accessing Dixa. Due to the ongoing maintenance, and intermittent problems with connecting, it was assumed the issues were related to the maintenance which was still ongoing. After an increasing number of reports from customers, the incident process was triggered just after 7 AM CEST. We soon realized some agents had issues with accessing Dixa, while others did not, and seemingly after a few tries agents were able to get in. We did not see increased errors in our internal monitoring tools, and agent connections kept increasing as the day started in Europe, which is normal. Around 7.15 AM CEST, we also experienced intermittent issues with internal tooling returning an error we did not expect, and which later pointed us in the right direction as to where to look. 20 Minutes later, around 7.40 AM CEST, Dixa engineers and customer success teams found what was causing the issues and started implementing a fix and 7 minutes later, at 7.47 AM CEST the issue was resolved. We communicated the fix 7.51 AM CEST after internal verification everything was back to normal. We sincerely apologize for any inconvenience this may has caused and want to assure you we’ve taken action to prevent this issue from surfacing again in the future, and have added additional logging and error notifications to be able to detect issues earlier on. Furthermore, we’re re-evaluating the planned maintenance process to make sure maintenance is always communicated via this page.