MageMojo incident

Emergency Maintenance on 1 node in USEast (53d61e5)

Major Resolved View vendor source →

MageMojo experienced a major incident on August 27, 2022 affecting Webscale STRATUS - Northern Virginia, lasting 1h 17m. The incident has been resolved; the full update timeline is below.

Started
Aug 27, 2022, 06:48 PM UTC
Resolved
Aug 27, 2022, 08:05 PM UTC
Duration
1h 17m
Detected by Pingoru
Aug 27, 2022, 06:48 PM UTC

Affected components

Webscale STRATUS - Northern Virginia

Update timeline

  1. investigating Aug 27, 2022, 06:48 PM UTC

    We are currently investigating this issue.

  2. identified Aug 27, 2022, 07:19 PM UTC

    The issue has been identified and a fix is being implemented.

  3. resolved Aug 27, 2022, 08:05 PM UTC

    This incident has been resolved.

  4. postmortem Aug 29, 2022, 04:57 PM UTC

    The Root cause of this incident for a small subset of stores was due to a bad/degraded AWS EC2 instance. Due to this degradation we have activated HA failover mechanism and balanced stores on new hardware nodes from our fleet. For real time HA recovery additional High Availability Strategies are available: [https://magemojo.com/kb/best-practices/high-availability-strategies/](https://magemojo.com/kb/best-practices/high-availability-strategies/) Please feel free to contact support if you have any questions.