Kraken.io Image Optimizer incident

Network hardware failure

Critical Resolved View vendor source →

Kraken.io Image Optimizer experienced a critical incident on September 21, 2020 affecting Kraken.io API and Kraken.io Web Interface and 1 more component, lasting 2d 15h. The incident has been resolved; the full update timeline is below.

Started
Sep 21, 2020, 05:01 PM UTC
Resolved
Sep 24, 2020, 08:14 AM UTC
Duration
2d 15h
Detected by Pingoru
Sep 21, 2020, 05:01 PM UTC

Affected components

Kraken.io APIKraken.io Web InterfaceKraken.io Callback ServiceKraken.io StorageKraken.io Homepage

Update timeline

  1. identified Sep 21, 2020, 05:01 PM UTC

    Due to a failure in our networking hardware, kraken.io is unavailable. An engineer has been paged and is driving to the data center for repairs. ETA: 1h

  2. identified Sep 21, 2020, 05:02 PM UTC

    We are continuing to work on a fix for this issue.

  3. identified Sep 21, 2020, 05:29 PM UTC

    We have deployed a fix to the networking hardware, moving on to restoring other services.

  4. identified Sep 22, 2020, 09:06 AM UTC

    We shipped and replaced new hardware for a bunch of systems in our processing pipeline. API is back up, restoring web interface next.

  5. monitoring Sep 22, 2020, 11:45 AM UTC

    Operations have been restored, we'll keep on monitoring all systems.

  6. resolved Sep 24, 2020, 08:14 AM UTC

    Networking issues have been resolved, no further issues detected during monitoring.