Proemion incident

Full Outage on Proemion Data Platform

Notice Resolved View vendor source →

Proemion experienced a notice incident on March 15, 2017, lasting —. The incident has been resolved; the full update timeline is below.

Started
Mar 15, 2017, 04:03 PM UTC
Resolved
Mar 15, 2017, 04:03 PM UTC
Duration
Detected by Pingoru
Mar 15, 2017, 04:03 PM UTC

Update timeline

  1. resolved Mar 15, 2017, 04:03 PM UTC

    Outage on March 14th - 15th Facts Yesterday we had a major file system cluster failure. This failure caused the entire PSP to stall at 23:08 GMT+1. We mitigated the failure at 06:51 GMT+1, starting to parse the backlog that had accumulated in the meantime. Everything was running normally again around 7:30 GMT+1. Our focus during the restore was on the analysis of the problem and 100% recovery of the data. No data has been lost and no customer action has been required. The automatic update of the status.proemion.com page was affected by the outage. Mitigation During the analysis of the problem we found problematic incompatibilities of the file system cluster with our setup. To avoid further problems we have moved the Proemion Data Platform onto a different file system environment. The failure on the statuspage update will be addressed during the next days. In Conclusion To deliver the high available Data Platform that our customers deserve, we have simplified the file system stack. Our team is committed to follow up on problems like this one and we take them absolutely seriously. Besides the push to get new features, APIs and web frontends to our customers we are constantly working on infrastructure of our platform.