TOPdesk experienced a critical incident on November 15, 2024 affecting AU1 SaaS hosting location and BR1 SaaS hosting location and 1 more component, lasting 56m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 15, 2024, 03:02 PM UTC
We are currently experiencing system wide disruptions with one of our services. As a result you might be unable to send out mail when you use our mail server. We are aware of the problem and are working on a solution. To help us troubleshoot this issue, please add the following information to your ticket when you are affected by this disruption: - The date and time of the last reproduction of this issue - The username of the user that reproduced this issue - A description of what the user was doing in TOPdesk - Browser activity and network traffic logs in a .HAR file, see KI 11015 on how to gather this information. Please contact TOPdesk Support after submitting an incident, so we can store additional logfiles to investigate this. Our apologies for the inconvenience. At the time of writing this we are not able to give you an estimate on when your environment will be available. We aim to update this status page every 30 minutes until the issue has been resolved. E-mail updates will be sent when the issue has been resolved. You can subscribe on the status page (https://status.topdesk.com) for additional updates. To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR24 11 4509.
- investigating Nov 15, 2024, 03:12 PM UTC
We are continuing to investigate this issue.
- resolved Nov 15, 2024, 03:59 PM UTC
This incident has been resolved.
- postmortem Nov 19, 2024, 09:00 AM UTC
On Friday 15th of November at around 14:45 CET we received notice of several customers not being able to send mail from TOPdesk. Due to the high number of incoming incidents, we treated this as a major incident and started research with high priority. Development was contacted and a fix was deployed on all servers at 16:16 CET. Further research showed that this particular error only occurs when you send a mail with an attachment. The root cause of this error was an inconsistency in a dependency that had been updated. Precautions will be implemented to avoid these discrepancies in the future.