Sift incident

Elevated score latencies

Major Resolved View vendor source →

Sift experienced a major incident on March 23, 2019 affecting Score and Workflows, lasting 6h 26m. The incident has been resolved; the full update timeline is below.

Started
Mar 23, 2019, 09:08 AM UTC
Resolved
Mar 23, 2019, 03:34 PM UTC
Duration
6h 26m
Detected by Pingoru
Mar 23, 2019, 09:08 AM UTC

Affected components

ScoreWorkflows

Update timeline

  1. investigating Mar 23, 2019, 09:08 AM UTC

    We are currently experiencing elevated Score API latencies and investigating the underlying problem.

  2. investigating Mar 23, 2019, 09:19 AM UTC

    We have identified the underlying problem with the score api latencies and are working on a resolution.

  3. investigating Mar 23, 2019, 09:28 AM UTC

    We have implemented a fix to the elevated Score API solution and expect to resume normal service shortly.

  4. investigating Mar 23, 2019, 09:37 AM UTC

    We are continuing to investigate this issue.

  5. investigating Mar 23, 2019, 09:41 AM UTC

    The issue with elevated Score API latencies has been resolved. Workflows are recovering and we expect them to be fully operational shortly.

  6. investigating Mar 23, 2019, 09:55 AM UTC

    Workflows are still recovering from the earlier issues.

  7. investigating Mar 23, 2019, 10:11 AM UTC

    Workflows are still continuing to recover from the earlier issues.

  8. investigating Mar 23, 2019, 10:25 AM UTC

    Workflows are still continuing to recover from the earlier issues.

  9. investigating Mar 23, 2019, 10:41 AM UTC

    Workflows are still continuing to recover from the earlier issues.

  10. investigating Mar 23, 2019, 10:53 AM UTC

    Workflows are fully operational.

  11. resolved Mar 23, 2019, 03:34 PM UTC

    This incident has been resolved.