Mural experienced a critical incident on November 30, 2022 affecting Realtime collaboration, lasting 1h 7m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 30, 2022, 10:05 AM UTC
MURAL is experiencing connection issues, leaving users unable to collaborate, seeing a 'Trying to Reconnect' message. We're investigating and will restore regular service ASAP. Please check our status page for the most up-to-date info 👉 status.mural.co/
- identified Nov 30, 2022, 10:13 AM UTC
The issue has been identified and a fix has been implemented.
- identified Nov 30, 2022, 10:13 AM UTC
We are continuing to work on a fix for this issue.
- monitoring Nov 30, 2022, 10:17 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved Nov 30, 2022, 11:12 AM UTC
This incident has been resolved.
- postmortem Dec 05, 2022, 12:43 PM UTC
**What happened?** At 06:40 UTC we started to receive notifications from Mural users that were unable to access their content in our European datacenter. This was caused by a configuration change that resulted in connections to our EU database being rejected. Once the cause of the issue was identified, an emergency fix was created, tested and deployed. Normal service was restored at 07:17 UTC. The impact of the emergency fix was monitored for an hour, after which the incident was declared as ‘resolved’. **Summary** Murals in our EU datacenter were unavailable from 06:40 to 07:17 UTC, for a total of 37 minutes of service interruption. **What we've done to avoid this happening again** We are improving our deployment processes and monitoring to include anomalies in the volume of realtime connections.