Subbly incident

Some automations and jobs not running properly

Major Resolved View vendor source →

Subbly experienced a major incident on October 28, 2023 affecting Billing Engine and Automations & Jobs, lasting 40m. The incident has been resolved; the full update timeline is below.

Started
Oct 28, 2023, 08:20 PM UTC
Resolved
Oct 28, 2023, 09:00 PM UTC
Duration
40m
Detected by Pingoru
Oct 28, 2023, 08:20 PM UTC

Affected components

Billing EngineAutomations & Jobs

Update timeline

  1. identified Oct 28, 2023, 08:20 PM UTC

    We've identified problems with some of the jobs not being ran properly in the last 24 hours. As a result of that, some of the jobs haven't been or have been partially executed (most notably, customer's renewals). Problem has been diagnosed and we're actively working on fixing it.

  2. identified Oct 28, 2023, 08:20 PM UTC

    We are continuing to work on a fix for this issue.

  3. resolved Oct 28, 2023, 09:00 PM UTC

    Fix has been deployed, all jobs are back and running. Jobs and automations that have been clogged up in the queue because of the incident were processed successfully.