Proxyclick incident

Degraded Performance event - Notifications and Watchlist

Minor Resolved View vendor source →

Proxyclick experienced a minor incident on June 22, 2022 affecting SMS and Mail and 1 more component, lasting 3h 23m. The incident has been resolved; the full update timeline is below.

Started
Jun 22, 2022, 01:00 PM UTC
Resolved
Jun 22, 2022, 04:23 PM UTC
Duration
3h 23m
Detected by Pingoru
Jun 22, 2022, 01:00 PM UTC

Affected components

SMSMailWebhooks

Update timeline

  1. investigating Jun 22, 2022, 02:24 PM UTC

    An impact has been observed to our real-time event processsing which is causing delays to outbound Notifications (email, SMS, and other channels) and Watchlist lookups. Access Control was briefly impacted at the start of this event, but this impact has already been fully mitigated.

  2. monitoring Jun 22, 2022, 02:27 PM UTC

    Additional resources have been brought online to tackle the backlog. We are seeing delays of up to 30 minutes on some message processing, and expect this impact will continue for some additional hours while we recover. No messages or data has been lost, and we will continue to monitor until we are fully back to real-time notifications.

  3. resolved Jun 22, 2022, 04:23 PM UTC

    We have completed processing of the backlogged messages and all systems have resumed real-time processing. We will update this Incident with more information and root cause once we have completed an internal review of the event.