Cadmium experienced a notice incident on May 8, 2026, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved May 08, 2026, 07:46 PM UTC
Incident window: 5/7/2026 9:02 PM ET – 5/8/26 7:40 AM ET Last night, EthosCE LMS sites experienced intermittent 500 errors on pages that use search functionality. The issue has been resolved and all sites are operating normally. What happened At 9:02 PM ET, one of the data nodes in the OpenSearch cluster supporting EthosCE became unavailable. The remaining nodes were near disk capacity, which prevented the cluster from automatically rebalancing and self-healing. At 4:51 ET, the original node came back online and sites were operational. While the system was online, the replicas were still in a bad state and could not handle a failover. At 7:41, space was added to the replicas and they were fully operational again. Impact Between 9:02 PM ET on 5/7 and 4:51ET on 5/8, intermittent 500 errors on EthosCE LMS pages that rely on the Drupal Search API. After 4:51 ET, all site were fully operational again. Course content, user accounts, and non-search functionality were not affected. No data was lost. Resolution Engineering restored the failed node and brought the cluster back to a healthy state. Search-dependent pages have been verified as functioning normally. We apologize for the disruption and appreciate your patience.