Permutive incident

Dashboard and API service interuption

Major Resolved View vendor source →

Permutive experienced a major incident on December 19, 2018 affecting Permutive API and Permutive Dashboard, lasting 1h 17m. The incident has been resolved; the full update timeline is below.

Started
Dec 19, 2018, 02:17 PM UTC
Resolved
Dec 19, 2018, 03:34 PM UTC
Duration
1h 17m
Detected by Pingoru
Dec 19, 2018, 02:17 PM UTC

Affected components

Permutive APIPermutive Dashboard

Update timeline

  1. investigating Dec 19, 2018, 02:17 PM UTC

    Our monitoring has alerted us to failures occurring in our API (which our SDK sends events to) and our dashboard (where our customers manage their services). We are investigating the issues and will update as soon as we have clarity.

  2. identified Dec 19, 2018, 02:32 PM UTC

    The issue has been identified and is resulting from a networking outage with our cloud provider (Google Cloud), and more specifically the 'europe-west1-b' region some of our services run in. Google are investigating and a fix should follow imminently. We are among a large number of their customers effected. We are also investigating mitigations we can make, and will keep the you updated.

  3. monitoring Dec 19, 2018, 02:39 PM UTC

    The incidents with Google's Cloud services have been mitigated, functionality and performance of our API and dashboard have returned to normal. We will monitor the situation and update if any changes occur.

  4. resolved Dec 19, 2018, 03:34 PM UTC

    The incident is now resolved.