Subbly incident

Main Services (Checkout & Admin) degraded performance

Major Resolved View vendor source →

Subbly experienced a major incident on September 12, 2023, lasting 12m. The incident has been resolved; the full update timeline is below.

Started
Sep 12, 2023, 10:30 AM UTC
Resolved
Sep 12, 2023, 10:42 AM UTC
Duration
12m
Detected by Pingoru
Sep 12, 2023, 10:30 AM UTC

Update timeline

  1. resolved Sep 12, 2023, 11:49 AM UTC

    We've experienced a downtime of the core features for ~12 minutes due to high database CPU usage. Cause of this was migrating old database events to the new structure (improvement that will improve logging on the admin). Problem was immediately isolated and we rectified the problem by splitting the script into smaller chunks to prevent throttling of the CPU. Problem has been fixed at 12.31PM UCT time.