pingoru pingoru Seald had 16 outages in the last 2 years, averaging 0.7 incidents per month.
There were 16 Seald outages since October 23, 2024. Each is summarised below. Incident details, duration, and resolution information.
On September 30, 2026, between 2:43 PM and 3:22 PM (Paris time), the staging environment was unavailable for 39 minutes. This incident affected the staging environment only; production environments were not impacted at any point.
⚠️ Staging only — production is not affected. We're updating the staging load balancer. No downtime is expected.
Some of our workers our down, so the users routed to them get 404 errors. We are investigating and re-routing to functional workers. EDIT : 9:05 : traffic re-routed to working nodes EDIT : 9:25 : all nodes are back up
On of the API workers was down, which resulted in failure for a portion of the requests. IP was redirected to a functional worker while we are investigating. EDIT: Fixed
One node has an expired certificate that has not been renewed automatically. It has been fixed at 11:40AM.
One of the workers had a certificate expiration. 12:06 : First reports of issues. 12:20 : The IP has been re-routed to another worker while we investigate the root cause of the renewal failure.
We experienced network-related errors that caused temporary service disruptions. The issue has been identified and resolved.
It seems many endpoints end up in 500 errors. We are investigating. The error seems to originate in a connection error between the application servers and the databases. It seems other customers of our hosting provider complain that hostnames for the databases cannot be resolved. We have opened a support ticket, OVHCloud status page does not reflect the incident yet. UPDATE: A similar incident has been reported on OVHCloud's status page: https://public-cloud.status-ovhcloud.com/incidents/9qmxv0gbky43 UPDATE: OVH announces that the incident should be fixed, but are still monitoring. UPDATE: Our internal monitoring shows that errors are reduced a lot, but not down to zero yet. UPDATE: Our internal monitoring does not show any errors anymore.
Between 10h25 and 10h30 Paris time, static assets were unreachable (dashboard). API was not impacted.
When trying a potential fix for the 502 errors, one of the workers got stuck in an unexpected state, which made it return 404 errors. This is now fixed.
We have observed a small but unusual number of requests ending in 502 and 504 errors on both Staging and Production environments. We are investigating to find the root cause. There may be a few unavailabilities on the Staging environment while we investigate.
It seems some of the requests made to api.staging-0.seald.io are getting timeouts. We are investigating. The incident does not affect the production environment. 23/07/2025 13:20: Fixed
Staging environement is currently unavailable. Investigating. EDIT : fixed at 17:03
Our caching service, which is used for certain operations, has crashed in a way that was not detected by our automated healthchecks. This has affected operations such as sending emails & SMS messages. We are working on re-establishing service. Update: Service seems re-established since 09:41. The issue was caused by a network problem at OVH which in turned caused a database update failure. We are still verifying the state of the database.
Staging environment was temporarily inaccessible. All is back to normal.
We are experiencing some instability related to the load balancer on the Staging environment. Production is unaffected.