Shift4 incident

Azure degradation of services: Shift4Shop

Minor Resolved View vendor source →

Shift4 experienced a minor incident on April 16, 2022 affecting Online Stores and API and 1 more component, lasting 4d 17h. The incident has been resolved; the full update timeline is below.

Started
Apr 16, 2022, 07:49 PM UTC
Resolved
Apr 21, 2022, 01:09 PM UTC
Duration
4d 17h
Detected by Pingoru
Apr 16, 2022, 07:49 PM UTC

Affected components

Online StoresAPISupport Systems

Update timeline

  1. identified Apr 16, 2022, 07:49 PM UTC

    Microsoft has reported intermittent Azure service issues due to a failing hard drive in the Azure hosting environment. This issue is impacting three clusters in the Azure East region where a number of Shift4Shop sites are hosted. Microsoft is actively working to replace the failing hard drive and live-migrate Shift4Shop VMs. An ETA for hardware replacement and migration has not been provided by Microsoft. The Shift4Shop team is in contact with Microsoft and will provide an update as soon as an ETA is provided.

  2. identified Apr 16, 2022, 07:50 PM UTC

    We are continuing to work on a fix for this issue.

  3. identified Apr 18, 2022, 05:16 PM UTC

    We are continuing to experience intermittent performance issues within our hosting environment and working with Microsoft to diagnose and resolve the issue

  4. monitoring Apr 18, 2022, 11:06 PM UTC

    The Shift4 team has worked with Microsoft in making changes to the hosting environment. We are now seeing performance improvements and will continue to monitor until we are certain the issue is fully resolved

  5. monitoring Apr 19, 2022, 10:51 PM UTC

    The latest optimization efforts allowed us to achieve stability in the hosting environment. As a result, overall performance should be at normal levels. Our team will continue to monitor the issue during the following hours closely

  6. resolved Apr 21, 2022, 01:09 PM UTC

    Shift4Shop hosting issues have been resolved