CrystalCommerce incident
MySQL Response Time Spikes resulting in downtime
CrystalCommerce experienced a critical incident on May 20, 2026 affecting Admin and POS and 1 more component, lasting 2h 7m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 20, 2026, 12:17 AM UTC
Our engineers are aware of a service interruption on all systems currently (admin, POS, and frontend websites) and are working on getting everything back up and running as soon as possible. This downtime is related to spikes in MySQL response times, which we are investigating the cause of. We will provide another update as soon as they have the errors corrected. Thank you.
- identified May 20, 2026, 12:22 AM UTC
Latest update from our team: "mysql is up again waiting for the pods to become healthy before sending traffic"
- identified May 20, 2026, 01:23 AM UTC
We've traced the problem to our datacenter, and are filing a ticket with them.
- identified May 20, 2026, 01:27 AM UTC
We are continuing to work on a fix for this issue.
- resolved May 20, 2026, 02:25 AM UTC
Earlier today, between approximately 4:15PM and 6:50PM Pacific, some members experienced slower response times or temporary issues accessing parts of their Crystal Commerce storefront and admin tools. We want to share a clear update — and the good news first: the issue is fully resolved and your data is safe. What happened: A hardware-level issue with one of the storage volumes provided by our cloud infrastructure provider, DigitalOcean, caused a database server to become unresponsive. The root cause was at the underlying infrastructure layer — not with your data, and not with our application itself. What we did: Our engineering team identified the source quickly and worked side-by-side with DigitalOcean’s infrastructure team to resolve it. DigitalOcean has confirmed the underlying issue is fully cleared on their side, and we have independently verified every layer of our platform is back to a healthy state. Where we are now: - All services are running normally - Your data is intact — nothing was lost - The database has been verified healthy - The platform is performing as expected - This was an infrastructure availability issue — there are no security or data-integrity concerns Moving forward: Even though the root cause originated outside of our application, we’re treating it as an opportunity to strengthen the platform. Over the coming weeks, we’ll be reviewing our failover and redundancy plans so that any similar provider-side issues in the future have even less impact on you. Thank you for your patience and trust today. If you notice anything unusual or have any questions, please reach out to your account team or our support team — we’re here to help. — The Crystal Commerce Team