Permutive incident

Slow responses

Major Resolved View vendor source →

Permutive experienced a major incident on May 8, 2019 affecting Permutive API, lasting 13m. The incident has been resolved; the full update timeline is below.

Started
May 08, 2019, 11:59 AM UTC
Resolved
May 08, 2019, 12:13 PM UTC
Duration
13m
Detected by Pingoru
May 08, 2019, 11:59 AM UTC

Affected components

Permutive API

Update timeline

  1. investigating May 08, 2019, 11:59 AM UTC

    We've been alerted to an increase in latency being seen via our API, we're currently investigating the causes. We will update this as soon as we know more.

  2. identified May 08, 2019, 12:04 PM UTC

    We've identified the root cause of the issue, which is causing some requests to be rejected from our API. Further, we've implemented a fix, and we expect resolution to happen within 10 minutes once the fix propagates out into our API cluster.

  3. monitoring May 08, 2019, 12:08 PM UTC

    Our fix has propagated out to our API cluster and latencies should be returning to normal over the next few minutes.

  4. resolved May 08, 2019, 12:13 PM UTC

    The incident is now resolved and all services are returned to their normal state. We will share a full post-mortem within 24 hours.