Resolved after 30 minutes, on May 15, 2025.
**Incident window \(IST\)** 15 May 2025 — 11:59 to 12:26 UTC \(≈ 27 minutes\) **Service impacted** [app.zeotap.com](http://app.zeotap.com) dashboards and APIs \(partial outage, global\) #### **What happened** A newly-deployed Cloudflare rule—meant to route a specific marketing-widget path to a new backend—was too broadly scoped. Legitimate UI traffic was unintentionally rewritten and forwarded to an inactive origin, resulting in 4xx/5xx errors. #### **Customer impact** * UI pages failed to load certain resources, leading to blank or error states. * Approx. 2 k requests returned errors during the 27-minute window #### **Root cause** A logical grouping mistake in the rule’s filter \(AND / OR precedence\) caused it to match all requests that contained particular referrer domains, regardless of host or path. #### **Immediate remediation** * 12:26 — Rule disabled and cache purged. * Error rates returned to normal within seconds. #### **Preventive actions** 1. **Safer change process** – Cloudflare rules will move to Infrastructure-as-Code with mandatory peer review. 2. **Staged roll-outs** – Future edge-rules will deploy first to a staging zone and a limited canary cohort. 3. **Faster detection** – Synthetic checks for critical UI paths will alert after three consecutive failures \(≈ 3 min\). We apologise for the disruption and appreciate your patience while we improve our safeguards.
Update timeline
- resolved May 16, 2025, 07:29 AM UTC
Between 12:03 UTC and 12:28 UTC we saw an elevated API error rate (peaking at ≈ 7 %). The underlying issue has been fixed and the platform is operating normally. We are monitoring closely.
- postmortem May 16, 2025, 07:32 AM UTC
**Incident window \(IST\)** 15 May 2025 — 11:59 to 12:26 UTC \(≈ 27 minutes\) **Service impacted** [app.zeotap.com](http://app.zeotap.com) dashboards and APIs \(partial outage, global\) #### **What happened** A newly-deployed Cloudflare rule—meant to route a specific marketing-widget path to a new backend—was too broadly scoped. Legitimate UI traffic was unintentionally rewritten and forwarded to an inactive origin, resulting in 4xx/5xx errors. #### **Customer impact** * UI pages failed to load certain resources, leading to blank or error states. * Approx. 2 k requests returned errors during the 27-minute window #### **Root cause** A logical grouping mistake in the rule’s filter \(AND / OR precedence\) caused it to match all requests that contained particular referrer domains, regardless of host or path. #### **Immediate remediation** * 12:26 — Rule disabled and cache purged. * Error rates returned to normal within seconds. #### **Preventive actions** 1. **Safer change process** – Cloudflare rules will move to Infrastructure-as-Code with mandatory peer review. 2. **Staged roll-outs** – Future edge-rules will deploy first to a staging zone and a limited canary cohort. 3. **Faster detection** – Synthetic checks for critical UI paths will alert after three consecutive failures \(≈ 3 min\). We apologise for the disruption and appreciate your patience while we improve our safeguards.
Your product runs on more than Zeotap
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services