Uscreen experienced a major incident on October 14, 2024 affecting API V1 and Admin Portal and 1 more component, lasting 1h 48m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 14, 2024, 04:57 PM UTC
We’re investigating a slow loading issue affecting both the Admin portal and the storefront.
- investigating Oct 14, 2024, 04:59 PM UTC
We have identified the problem
- investigating Oct 14, 2024, 05:27 PM UTC
Our team is continuing to investigate the issue and work on implementing a fix. Thank you for your continued patience.
- identified Oct 14, 2024, 05:33 PM UTC
We identified the issue and applied a fix to improve performance as a quick measure. Our team will continue to work on the root cause.
- monitoring Oct 14, 2024, 05:51 PM UTC
Our team has implemented a fix for this issue and all platform functionality should be restored. Our team will continue to monitor. Thank you for your patience.
- resolved Oct 14, 2024, 06:31 PM UTC
This issue has been resolved.
- postmortem Oct 15, 2024, 02:27 PM UTC
**Incident Summary** On October 14, 2024, Uscreen encountered degraded performance across its API \(V1 and V2\), Admin Portal, and Storefront due to an unexpected surge in server load. This resulted in delayed response times and temporary service degradation. **Impact** The performance issue caused slow load times, affecting users' ability to access and use the platform's key features efficiently. **Root Cause** A specific endpoint could not handle the unexpected load increase, leading to performance degradation. **Resolution** Our engineering team acted swiftly to deploy a temporary patch that throttled the affected endpoint, preventing a full outage and ensuring that services remained partially operational. A permanent code update was later released, optimizing the performance of the endpoint and ensuring it can handle higher traffic loads in the future. **Next Steps** We are confident that these improvements, combined with continuous monitoring and proactive load management, will ensure the platform’s stability and resilience, even under higher usage demands. The recent updates safeguard against similar incidents and enhance the overall user experience on Uscreen.