Zid incident

Major outage on the Stores - Database issues related

Critical Resolved View vendor source →

Zid experienced a critical incident on September 17, 2022, lasting —. The incident has been resolved; the full update timeline is below.

Started
Sep 17, 2022, 11:00 PM UTC
Resolved
Sep 17, 2022, 11:00 PM UTC
Duration
—
Detected by Pingoru
Sep 17, 2022, 11:00 PM UTC

Update timeline

  1. resolved Sep 18, 2022, 12:00 PM UTC

    We experienced a major issue in our platform as a result of our main database failing unpredictably. Our failover mechanism was activated but unfortunately for reasons we are still studying also failed. We will provide further information. The first alarm was received at 2am and we got a few stores communicating to us at 3am. Our team started to work on the issue, however, due to the main database failing and the failover not working as expected it affected our ability to recover the system having an issue in the add to cart function. This had a partial issue (not affecting all stores) for 1 hour and additional 6 hours of outage to recover the system. We are further investigating on the issue to ensure this is not happening again due to the same problem and put all mechanisms in place to avoid this issues.