October 2023: Universign service degradation

Update timeline

1. resolved Oct 06, 2023, 02:20 PM UTC

   We are currently experiencing difficulties on our platform. More information will be made available as it is conveyed to us.

2. postmortem Oct 11, 2023, 03:06 PM UTC

   We apologize for the inconvenience caused by the incident which occurred on 06/10/2023 between 2:43 p.m. and 5:43 p.m. Detection time * 06/10/2023 2:43 p.m. - 5:43 p.m.: latency alerts on WS service * 06/10/2023 4:43 p.m. - 5:40 p.m.: intermittent alerts on WS service unavailable Resolution * 06/10/2023 2:48 p.m.: backoffice shutdown * 06/10/2023 2:54 p.m. - 4:50 p.m.: exit of a server from the pool * 06/10/2023 4:18 p.m. - 4:32 p.m.: restart and switchover of load balancers * 06/10/2023 4:38 p.m.: increase in limits per server on load balancers * 06/10/2023 4:40 p.m. - 6:00 p.m.: monitoring and analysis Incident end time * 06/10/2023 5:43 p.m.: all U1 services are operational Cause of the incident We are investigating high contention at the database level which introduced an increase in query latency, going so far as to make the service unavailable. We will get back to you as soon as we have more information. Remediation : We are investigating high contention at the database level which introduced an increase in query latency, going so far as to make the service unavailable. We are investigating changing our database management service.