DealerSocket incident

Websites: Engine 6 Websites not updating

Minor Resolved View vendor source →

DealerSocket experienced a minor incident on July 9, 2020 affecting DealerFire - Engine6, lasting 13h 50m. The incident has been resolved; the full update timeline is below.

Started
Jul 09, 2020, 02:00 PM UTC
Resolved
Jul 10, 2020, 03:51 AM UTC
Duration
13h 50m
Detected by Pingoru
Jul 09, 2020, 02:00 PM UTC

Affected components

DealerFire - Engine6

Update timeline

  1. investigating Jul 09, 2020, 02:00 PM UTC

    We are investigating reports that changes to Engine6 Websites are not pushing to live sites. Teams will have an update shortly.

  2. identified Jul 09, 2020, 02:17 PM UTC

    We have identified the problem and put a fix in place. The system will take some time to catch up, but changes are pushing through to the production environment. We update this page with progress as we catch up.

  3. monitoring Jul 09, 2020, 02:50 PM UTC

    Our fix has worked. We are monitoring the progress as the system processes the backlog of changes to push through. It is expected that new changes made to websites will be delayed until the system processes the backlog.

  4. monitoring Jul 09, 2020, 03:51 PM UTC

    We are still seeing backlog changes pushing through the process. Teams are monitoring the environment for stability as we process the changes.

  5. monitoring Jul 09, 2020, 06:52 PM UTC

    We are still progressing through the backlog of changes.

  6. monitoring Jul 09, 2020, 07:37 PM UTC

    We are still moving through the backlog. Some changes will start to appear onto websites as the backlog is completed.

  7. monitoring Jul 09, 2020, 09:37 PM UTC

    We do appreciate your patience while the system works to continue processing the backlog.

  8. monitoring Jul 09, 2020, 11:59 PM UTC

    Are teams are still working to help the backlog process as quickly as possible.

  9. monitoring Jul 10, 2020, 01:59 AM UTC

    We are still monitoring the process as it progresses into the late evening hours. There may be some exceptions, however at this point, most changes from before 5 PM Central time should be processed.

  10. resolved Jul 10, 2020, 03:51 AM UTC

    This incident has been resolved.