Ookla Outage History

Started Nov 18, 2025, 12:10 PM UTC· resolved Nov 19, 2025, 10:21 AM UTC

1. investigating Nov 18, 2025, 12:10 PM UTC

   We are currently experiencing service degradation across multiple Downdetector platforms, including the API, Explorer, and consumer-facing sites. The issue is linked to an ongoing Cloudflare network incident that is affecting traffic routing and edge performance.

2. monitoring Nov 18, 2025, 04:50 PM UTC

   We have implemented workarounds for all Cloudflare-related issues, and services have now recovered. Cloudflare is also reporting improvement, and our temporary mitigations remain in place as a precaution. We are actively monitoring the situation, and all Downdetector services appear to be operating normally. Further updates will be provided if required.

3. monitoring Nov 18, 2025, 05:05 PM UTC

   We have implemented workarounds for all Cloudflare-related issues, and services have now recovered. Cloudflare is also reporting improvement, and our temporary mitigations remain in place as a precaution. We are actively monitoring the situation, and all Downdetector services appear to be operating normally. Further updates will be provided if required.

4. resolved Nov 19, 2025, 10:21 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 01, 2025, 10:36 AM UTC· resolved Oct 01, 2025, 01:35 PM UTC

1. investigating Oct 01, 2025, 10:36 AM UTC

   We are currently investigating an issue with our database load balancers, causing an outage for customers.

2. identified Oct 01, 2025, 11:45 AM UTC

   The issue has been identified and a fix is being implemented.

3. monitoring Oct 01, 2025, 01:07 PM UTC

   A fix has been deployed, some caching issues remain for some login areas.

4. resolved Oct 01, 2025, 01:35 PM UTC

   This incident has been resolved.

5. postmortem Oct 01, 2025, 02:50 PM UTC

   ## Incident Summary **Date:** October 1, 2025 **Impact:** Full outage for all European customers **Status:** Resolved --- ### Incident Overview On **October 1st at 11:10 BST**, our internal service discovery system in the European region experienced a failure following a low-risk certificate update. This unexpectedly caused dependent systems—including databases and backend services—to become unavailable. As a result, **all customers using our European platform were impacted** and experienced downtime. --- ### Timeline \(BST\) | Time | Event | | --- | --- | | 10:58 | Internal service certificates were renewed and deployed \(low-risk change\) | | 11:00 | Backend services began receiving updated certificates | | 11:10 | Core coordination service failed to re-establish leadership; investigation began | | 11:15 | First customer reports of service errors received | | 11:36 | Incident publicly posted on status page | | 12:40 | Root cause identified and recovery process initiated | | 12:45 | Status page updated | | 13:40 | Coordination service quorum restored and systems recovered | | 13:50 | Application deployments restarted to refresh service connectivity | | 14:07 | Status page updated | | 14:35 | Incident marked as resolved | --- ### What Went Wrong An internal certificate update—considered low risk—led to a failure in one of our backend coordination systems. This caused a loss of service discovery across our European platform, impacting all services that rely on this mechanism to communicate, including core databases. The issue required manual intervention to restore control and quorum within the system. --- ### How We Resolved It * Launched a new set of coordination nodes using a known working configuration * Temporarily bypassed strict internal security controls to allow recovery * Manually brought the core cluster back online and ensured stability * Restarted dependent services to refresh their service discovery state * Closely monitored system health during and after recovery --- ### What We're Doing Next To prevent similar incidents in the future, we are: * **Improving certificate update validation and rollback procedures** * **Automating recovery workflows** for coordination service failures * Beginning to **evaluate alternative service discovery strategies** with reduced operational risk and complexity --- ### Final Note We sincerely apologise for the disruption caused. This incident affected all customers using our European platform, and we take this extremely seriously. We are committed to improving our systems to reduce the risk of similar outages in the future.

Read the full incident report →

Started Nov 08, 2024, 10:17 PM UTC· resolved Nov 11, 2024, 07:03 AM UTC

1. investigating Nov 08, 2024, 10:17 PM UTC

   We are currently investigating this issue.

2. investigating Nov 08, 2024, 10:18 PM UTC

   We are continuing to investigate this issue.

3. investigating Nov 08, 2024, 10:42 PM UTC

   speedtest.net is accessible. We are continuing to investigate this issue.

4. resolved Nov 11, 2024, 07:03 AM UTC

   This incident has been resolved.

Read the full incident report →