Total Expert incident

Outbound Notifications Outage

Major Resolved View vendor source →

Total Expert experienced a major incident on April 4, 2024 affecting Event Processing, lasting 7h 34m. The incident has been resolved; the full update timeline is below.

Started
Apr 04, 2024, 04:00 PM UTC
Resolved
Apr 04, 2024, 11:35 PM UTC
Duration
7h 34m
Detected by Pingoru
Apr 04, 2024, 04:00 PM UTC

Affected components

Event Processing

Update timeline

  1. identified Apr 04, 2024, 04:00 PM UTC

    Outbound web hooks are currently disabled. We have identified the issue and this only affects customer web hook subscriptions in our event processing. We will provide an update within 3 hours.

  2. identified Apr 04, 2024, 06:00 PM UTC

    We are testing a fix for the issue. We will provide another update within 3 hours.

  3. monitoring Apr 04, 2024, 08:57 PM UTC

    A fix has been implemented and we're processing outbound web hooks again. There is a small backlog that we are working through and we will monitor this until we have caught up.

  4. resolved Apr 04, 2024, 11:35 PM UTC

    The backlog of events have been processed and we are considering this resolved.