TOPdesk incident

Some Environments are not reachable due to routing issues

Major Resolved View vendor source →

TOPdesk experienced a major incident on September 16, 2024 affecting AU1 SaaS hosting location and BR1 SaaS hosting location and 1 more component, lasting 4h 10m. The incident has been resolved; the full update timeline is below.

Started
Sep 16, 2024, 05:58 AM UTC
Resolved
Sep 16, 2024, 10:09 AM UTC
Duration
4h 10m
Detected by Pingoru
Sep 16, 2024, 05:58 AM UTC

Affected components

AU1 SaaS hosting locationBR1 SaaS hosting locationCA1 SaaS hosting locationEU1 SaaS hosting locationEU2 SaaS hosting locationEU3 SaaS hosting locationEU4 SaaS hosting locationNL4 SaaS hosting locationNO1 SaaS hosting locationUK1 SaaS hosting location

Update timeline

  1. investigating Sep 16, 2024, 05:58 AM UTC

    We are curently investigating this issue To inform TOPdesk you are affected by this issue, please visit https://my.topdesk.com/tas/public/ssp/ . Please refer to incident TDR24 09 4544

  2. identified Sep 16, 2024, 06:06 AM UTC

    After an initial check, it seems Cloudflare had routing issues this morning. They are implementing a fix. Our checks show the instances should be becoming available again. We will monitor the instances to check if all routing issues are gone.

  3. identified Sep 16, 2024, 06:34 AM UTC

    After some further checks, it seems not to be fixed yet. Cloudflare is reporting issues with their network and are implementing a fix since 5 AM UTC (7 AM CEST). We also see an increase in errors still on our end of our cloudflare dashboard. Issues seem to be intermittent, where a URL becomes unavailable for some time. For further information see also https://www.cloudflarestatus.com/

  4. identified Sep 16, 2024, 06:59 AM UTC

    A fix was implemented by Cloudflare. We are still seeing a similar amount of errors in our dashboards, which indicate the issues are still present. We are checking the issues, but besides the Cloudflare issues, we can't find further issues. We are monitoring the situation.

  5. identified Sep 16, 2024, 07:04 AM UTC

    We are continuing to work on a fix for this issue.

  6. identified Sep 16, 2024, 08:18 AM UTC

    Our team is actively investigating an elevated rate of HTTP 499 & 5XX errors affecting some services. We are working diligently to identify the root cause and implement mitigation measures. For real-time status updates, please refer to the Cloudflare status page through the following URL: https://www.cloudflarestatus.com/incidents/f116htkkp7hn We apologize for any inconvenience this may cause and appreciate your patience as we work to resolve the issue as swiftly as possible. Thank you for your understanding.

  7. resolved Sep 16, 2024, 10:09 AM UTC

    Following confirmation from the Cloudflare status page (https://www.cloudflarestatus.com/incidents/f116htkkp7hn) and verification through our internal monitoring systems, we can confirm that the issue has been fully resolved. As a result, this incident is now considered closed. Should you have any further questions or require additional support, please don't hesitate to contact us via https://my.topdesk.com/.