Proemion experienced a major incident on October 13, 2017, lasting 12m. The incident has been resolved; the full update timeline is below.
Update timeline
- identified Oct 13, 2017, 05:29 AM UTC
During an early morning release of a new version, we run into a problem with the Dataportal. We are fixing it right now.
- resolved Oct 13, 2017, 05:41 AM UTC
All services are up and running again. Sorry for the inconvenience. The Dataportal Team is looking into the case right now from preventing it in the future.
- postmortem Aug 02, 2018, 04:40 PM UTC
Today we introduced an eager statistic collection about the structure of our datasets. Using these statistics we can do more targeted and hence faster queries. Unfortunately statistics collection put significant load on our systems. As the Dataportal usage kicked in the backend did not keep up reprocessing and delivering services to the Dataportal at the same time. We did not catch this problem ahead of time due to different usage pattern on the non-production environments that the backend change went though first. We are evaluating options to provide a more comparable usage pattern on the non-production environments. We are sorry for the interruption of the service delivery.