Siena incident

Hooks Processing Service Downtime

Minor Resolved View vendor source →

Siena experienced a minor incident on January 14, 2025 affecting Shopify, lasting —. The incident has been resolved; the full update timeline is below.

Started
Jan 14, 2025, 04:00 AM UTC
Resolved
Jan 14, 2025, 04:00 AM UTC
Duration
Detected by Pingoru
Jan 14, 2025, 04:00 AM UTC

Affected components

Shopify

Update timeline

  1. resolved Jan 14, 2025, 08:38 AM UTC

    Incident has been resolved. - Our Backfill Service remained operational throughout the incident and successfully processed the missing data to ensure continuity. - All missing webhooks have been re-registered, and the system is now fully operational.

  2. postmortem Jan 14, 2025, 09:56 AM UTC

    **Incident Report**: Hooks Processing Service Downtime * **10 PM UTC, Jan 13 – 4 AM UTC, Jan 14**: * Our Hooks Processing Service was overwhelmed by a high volume of Shopify webhooks, specifically for the orders/update topic. This led to a temporary downtime of the service and de-registration of webhooks by Shopify. * **4 AM – 10 AM UTC, Jan 14**: * During this period, we were not receiving webhooks for the affected topics \(orders/update\). **Resolution:** * Our Backfill Service remained operational throughout the incident and successfully processed the missing data to ensure continuity. * All missing webhooks have been re-registered, and the system is now fully operational. **Current Status:** Everything is back to normal, and the service is stable.