Qovery experienced a minor incident on August 28, 2024, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Aug 28, 2024, 02:00 PM UTC
Type: Incident Duration: 3 hours and 17 minutes Affected Components: Customer infratructure Aug 28, 14:00:00 GMT+0 - Identified - We want to inform you of a recent incident involving customers' Nginx configuration that affected some customers. Specifically, a conflicting setting was released that impacted customers with IP whitelists configured on their services, resulting in their requests being denied with an HTTP 403 error. Investigation is in progress Aug 28, 16:40:00 GMT+0 - Resolved - Our team has identified the root cause of this issue; however, further investigation is required, as the bug appears to have impacted configurations that should have remained unaffected. We are actively working to fully understand the scope and prevent any recurrence. We apologize for any inconvenience caused and appreciate your patience as we work to investigate this matter promptly. Aug 30, 13:18:57 GMT+0 - Resolved - Following a global cluster update triggered by our team, some configuration of the Nginx Ingress controller was wrongly modified on a subset of these clusters. Our team identified the root cause of the problem and deployed a fix on all clusters. The websites hosted on impacted clusters were unreachable during the outage. You can find more information on [this forum thread](https://discuss.qovery.com/t/post-mortem-http-requests-failing-with-403/3065) We apologize for any inconvenience caused.