Paazl experienced a critical incident on September 8, 2023 affecting Paazl Checkout Widget and Paazl Live Webservice and 1 more component, lasting 3d 10h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Sep 08, 2023, 04:43 AM UTC
We are currently seeing some partial outages that might impact ship environment, checkout Widget and label service. We are currently investigating this with highest priority.
- investigating Sep 08, 2023, 05:16 AM UTC
We are continuing to investigate this issue.
- monitoring Sep 08, 2023, 06:30 AM UTC
A fix has been implemented and all services are performing fully again. We will continue to monitor performance.
- monitoring Sep 08, 2023, 08:49 AM UTC
We are continuing to monitor for any further issues.
- resolved Sep 11, 2023, 03:07 PM UTC
This incident has been resolved.
- postmortem Sep 11, 2023, 03:07 PM UTC
On Friday the 8th of September, we noticed our webservice response for creating labels and orders slowed down. Our customers were experiencing issues with creating labels, committing orders, or loading of the widget services to our platform. After an internal investigation and technical support from our hosting provider, we discovered the root cause. Based on this investigation we can provide the following timeline. · 4:30 our monitoring provided the first internal messages · 5:30 the amount of monitoring messages increased · 6:30 our 24/7 support team was notified and started their investigation · 7:30 we found the root cause and our team started working on a fix · 8:00 a fix was deployed · 8:15 monitoring and testing showed all systems were clear and back to normal functioning. In order to avoid a similar occurrence we will increase the resources of our servers. We would like to apologize to our customers for the experienced inconvenience. If you have any further questions, feel free to contact us by email or phone at [[email protected]](mailto:[email protected]) or \+31 \(0\)20 - 773 - 63 03.