Ambra experienced a minor incident on September 5, 2023 affecting Web Services and Image Processing and 1 more component, lasting 1h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Sep 05, 2023, 02:56 PM UTC
We have received reports of issues on the Ambra platform. Engineering teams are currently investigating. Additional information will be provided as soon as it is available.
- investigating Sep 05, 2023, 03:11 PM UTC
Our Engineering teams are actively investigating and working to identify the root cause. We understand the urgency and we appreciate your patience as we work to address the issue.
- identified Sep 05, 2023, 03:31 PM UTC
Our engineering teams have identified an issue regarding database services which have become overloaded. This is affecting areas of the platform such as loading of images, and upload slowness. The teams continue to work towards a resolution for this issue, and further updates will be provided.
- identified Sep 05, 2023, 03:50 PM UTC
Our engineering teams continue to work towards a resolution for our database service issues at this time, and we will provide any new updates as they are received.
- identified Sep 05, 2023, 04:06 PM UTC
At this time, our database load is decreasing, and services are recovering. The investigation into the load and performance is continuing.
- monitoring Sep 05, 2023, 04:23 PM UTC
Our team continues investigating and working towards full recovery of affected services. We will continue to monitor the situation to ensure there are no further issues and send additional updates on any new developments.
- resolved Sep 05, 2023, 04:50 PM UTC
The incident has been fully resolved and service is back to normal levels. Our team will be conducting a root cause analysis and sharing as soon as possible. We will continue to monitor the situation to ensure there are no further issues.
- postmortem Sep 12, 2023, 06:40 PM UTC
Ambra experienced slow performance in the storage database cluster due to a large number of database queries executing concurrently, some of which were hung. A database server restart was required to terminate these queries and restore optimal performance. We also identified a frequently executed suboptimal database query and deployed an improved query in order to reduce the overall load on the database.