FastSpring incident

Missing Webhooks and Duplicate Buyer Notifications

Major Resolved View vendor source →

FastSpring experienced a major incident on February 16, 2022 affecting FastSpring System, lasting 1d 3h. The incident has been resolved; the full update timeline is below.

Started
Feb 16, 2022, 07:30 PM UTC
Resolved
Feb 17, 2022, 10:49 PM UTC
Duration
1d 3h
Detected by Pingoru
Feb 16, 2022, 07:30 PM UTC

Affected components

FastSpring System

Update timeline

  1. investigating Feb 17, 2022, 02:47 PM UTC

    We are currently investigating the issue.

  2. identified Feb 17, 2022, 04:53 PM UTC

    We have identified the root cause of the issue. Solutions are currently being tested. We have also determined that multiple notifications were sent to buyers for some subset of orders. The root cause is the same.

  3. identified Feb 17, 2022, 08:40 PM UTC

    We are continuing to investigate, and we have been testing out multiple solutions. We appreciate your patience.

  4. resolved Feb 17, 2022, 10:49 PM UTC

    Root causes have been identified and solutions are applied. New webhooks will now fire as expected. We will be working towards identifying missing events. Notifications to customers will not continue to be duplicated going forward.