Mono Solutions experienced a minor incident on November 23, 2021 affecting RAI, lasting 15h 4m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 23, 2021, 08:54 PM UTC
We are experiencing issues with creating new sites through RAI. This includes copying/creating templates as well. If you provision sites through API you should be able to continue doing so. The team is and will continue investigating the issue.
- resolved Nov 24, 2021, 11:58 AM UTC
This is now resolved. Creation of sites through RAI is now working. We will udpate the incident page with Postmortem later.
- postmortem Nov 24, 2021, 03:17 PM UTC
### **Summary** After a restart of the sattelite service in the Frankfurt AWS region, EU partners were unable to create new sites from RAI. Engineers tracked down the root cause to a configuration problem of PHP. ### **Impact** Not able to create new sites from RAI, European partners. HAL not able to successfully call the create site method from the backend. ### **Root Causes** Non persistent hot fixes where made to the server. Upon refreshing the fleet instances, the hotfix was lost. As a result the sattelite treated warnings as errors when HAL was calling the "createSite" function. ### **Resolution** The issue was resolved temporarily by disabling error reporting.