Sympa HR experienced a minor incident on March 21, 2025 affecting System availability, lasting 16m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Mar 21, 2025, 10:43 AM UTC
We are currently investigating an issue where some users may experience problems saving data in the system. Our team is actively working to identify the cause and implement a fix. We will provide an update as soon as we have more information. Thank you for your patience.
- resolved Mar 21, 2025, 11:00 AM UTC
We've identified the cause of the issue affecting data saving in the system. It was due to an operation which impacted the ability to save data. The problem has now been solved, and the system is functioning normally again. We appreciate your patience during this time.
- postmortem Mar 21, 2025, 02:13 PM UTC
**POST MORTEM: DISRUPTION 21st of March, Issues with Saving data** On March 21st, 2025, between 12:10 – 12:49 EET, an increase in system errors caused disruptions for some customers. During this time, users experienced issues saving data and delays in contracts being sent. The issue was identified at 12:30 EET, triggered by an alert following increased error rates. It was traced back to an operation performed in one of our databases, which unintentionally blocked changes to it. The blockage was resolved at 12:49 EET. **Timeline** * **12:10 EET** – The issue began, leading to increased errors and disruptions in saving data and sending contracts. * **12:30 EET** – The issue was detected by our monitoring. * **12:49 EET** – The underlying cause was addressed and error rates dropped **Preventative Measures** To reduce the risk of similar issues in the future, we are implementing the following improvements: * Introducing more specific alerts to quickly detect database operations that may cause system disruptions, allowing faster investigation and resolution. * Further limiting certain internal operations during working hours to reduce the risk of impact on active users. We sincerely apologize for the inconvenience and appreciate your understanding as we work to continuously improve our systems.