Cadmium experienced a minor incident on August 13, 2024 affecting Onsite, lasting 3h 2m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 13, 2024, 03:31 PM UTC
Our team is aware there is a service disruption with Eventscribe Onsite and is working swiftly to identify the cause. Thank you for your patience.
- identified Aug 13, 2024, 03:55 PM UTC
The root cause has been identified, and our team is working as quickly as possible to fix the issue. Thank you for your patience! We will provide an update as soon as we have more information.
- monitoring Aug 13, 2024, 03:57 PM UTC
A fix has been put in place, and service has been restored. Our team is continuing to monitor the system. If you are still experiencing issues, please notify your project manager.
- resolved Aug 13, 2024, 06:33 PM UTC
The issue has been resolved. We will share root cause analysis as soon as possible.
- postmortem Aug 22, 2024, 02:30 PM UTC
The outage was caused by an error in the deployment of an update to one of the login systems that caused failures in accessing Eventscribe Onsite. Processes have been altered to catch such deployment issues as they occur, as well as adding preventative measures, increased monitoring and alerting.