Nacelle experienced a major incident on November 13, 2023, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Nov 13, 2023, 09:57 PM UTC
Between 6:26 PM ET and 7:21 PM ET, Nacelle experienced a spike of new requests that required Nacelle’s DevOps team to scale additional Kubernetes pods. While these new pods were scaling up, the API experienced degraded performance (e.g., long latencies). The root cause of the issue was how Nacelle's custom schema fetching works across new additions to our GraphQL interface. To ensure stability over Black Friday / Cyber Monday, the Nacelle team has removed features still in development from the Storefront API schema service.