Butlr incident

Subsection of Butlr Hives received unintended sleep schedule settings update

Major Resolved View vendor source →

Butlr experienced a major incident on April 21, 2023, lasting 9h 44m. The incident has been resolved; the full update timeline is below.

Started
Apr 21, 2023, 04:36 AM UTC
Resolved
Apr 21, 2023, 02:20 PM UTC
Duration
9h 44m
Detected by Pingoru
Apr 21, 2023, 04:36 AM UTC

Update timeline

  1. identified Apr 21, 2023, 04:36 AM UTC

    We’re currently experiencing degraded performance issues with a subsection of Hives. Our team is currently working to restore normal performance levels. We apologize for any inconvenience. Users in the APAC and EU regions may be affected. Next update in 1 hour.

  2. monitoring Apr 21, 2023, 06:41 AM UTC

    We're currently monitoring the issue - right now all potentially affected Hives have successfully had a fix applied and read correctly. We will provide updates here at 9am EST.

  3. resolved Apr 21, 2023, 02:20 PM UTC

    Good news, we have resolved the issue with the subsection of Hives. All devices have been configured and operating correctly during the monitoring period. This was caused by an error in Hive update configuration and affected some of our users' devices, creating a data outage for some or all of the period between 2023-04-20 20:05:54 UTC and 2023-04-21 01:07:06 UTC. Thank you for your patience, and please let us know if you discover any additional issues.