Home Assistant incident

Continuing Forums Emergency Maintenance

Critical Resolved View vendor source →

Home Assistant experienced a critical incident on June 12, 2020 affecting Forums, lasting 7h 53m. The incident has been resolved; the full update timeline is below.

Started
Jun 12, 2020, 08:56 PM UTC
Resolved
Jun 13, 2020, 04:49 AM UTC
Duration
7h 53m
Detected by Pingoru
Jun 12, 2020, 08:56 PM UTC

Affected components

Forums

Update timeline

  1. identified Jun 12, 2020, 08:56 PM UTC

    During this window we are continuing to attempt recovery from yesterdays failed forum upgrades. The forums will be intermittently available and when they are available they will be read-only until the all clear is given. We don't have an ETA to provide at this time but will provide frequent updates.

  2. identified Jun 12, 2020, 10:55 PM UTC

    We have successfully gotten past the issues that bit us last night (we think) and are about to begin migration to a new machine. ETA 2 hours.

  3. identified Jun 13, 2020, 12:25 AM UTC

    We have been somewhat delayed due to opting to perform deferred maintenance as part of this migration. While we wait for uploads to transfer to a new S3 bucket we will disable read-only on the forums. Read only will be re-enabled shortly before the final migration. We estimate it will be 3 hours before we initiate the migration.

  4. identified Jun 13, 2020, 02:55 AM UTC

    Deferred maintenance has been completed. The forums are once again read only while we migrate. ETOR 2 hours.

  5. monitoring Jun 13, 2020, 03:56 AM UTC

    The migration has been successfully completed. We are verifying everything before declaring the incident over.

  6. resolved Jun 13, 2020, 04:49 AM UTC

    The forums have now returned to normal operations. We thank you for your patience during the last 24 hours. We strive to ensure good service on every service we provide and will be investigating the root cause of this extended outage to ensure that this won't happen again.