DeployGate incident

Some of applications in groups and/or enterprises became invisible

Major Resolved View vendor source →

DeployGate experienced a major incident on December 28, 2021 affecting Website and Android Application and 1 more component, lasting 3h 23m. The incident has been resolved; the full update timeline is below.

Started
Dec 28, 2021, 03:38 AM UTC
Resolved
Dec 28, 2021, 07:01 AM UTC
Duration
3h 23m
Detected by Pingoru
Dec 28, 2021, 03:38 AM UTC

Affected components

WebsiteAndroid ApplicationiOS Application

Update timeline

  1. investigating Dec 28, 2021, 03:38 AM UTC

    We're currently investigating the problem that some of users cannot see applications in groups and/or enterprises through teams.

  2. identified Dec 28, 2021, 03:44 AM UTC

    The problem has been identified and we are working on a fix.

  3. identified Dec 28, 2021, 05:45 AM UTC

    The problem has been identified and we are currently examining the deleted data for recovery. We expect to recover in a few hours.

  4. resolved Dec 28, 2021, 07:01 AM UTC

    This incident has been resolved and we have recovered all affected data. DeployGate is operating normally.

  5. postmortem Dec 28, 2021, 07:19 AM UTC

    # Issue * Since around 1:00 am on Dec 27, 2021 \(Japan Standard Time\), there was an issue where some DeployGate Organization customers were unable to access the app. * Due to an issue in DeployGate, some teams were experiencing an issue where associations with apps were being removed without permission. # Response * Dec 28 12:40 pm, we started the investigation. * Dec 28 1:20 pm, we have investigated the root cause and the issue was fixed. * Dec 28 3:40 pm, we have restored the unintentionally unlinked associations. \(Note: We have excluded from the restoration those associations that were manually deleted by the customer during this period, but there is a possibility that they were restored depending on the timing.\) ‌ We apologize for any inconvenience this may have caused during the busy holiday season. We will work to prevent this from happening again. Thank you for your continued support of DeployGate.