MageMojo experienced a major incident on May 17, 2021 affecting Webscale STRATUS - Northern Virginia, lasting 3h 10m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 17, 2021, 12:06 PM UTC
We are currently investigating this issue.
- identified May 17, 2021, 12:17 PM UTC
The issue has been identified and a fix is being implemented.
- resolved May 17, 2021, 03:16 PM UTC
This incident has been resolved.
- postmortem May 19, 2021, 04:11 PM UTC
The cause of this intermittent response for a small subset of stores was due to a bad/degraded AWS EC2 instance. On May 17th, 2021 at 2:30a EDT AWS informed us that they have detected degradation of the underlying hardware hosting our Amazon EC2 instance. Due to this degradation instance was rebooted twice in hope of recovery and urgent migration to a fresh node was initiated. Please feel free to contact MageMojo Support if you have any questions.