Sirv experienced a minor incident on November 14, 2024 affecting Current status by service (Sirv API), lasting 18h 50m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 14, 2024, 09:52 AM UTC
An elevated number of 504 timeouts are being returned for API file upload requests.
- resolved Nov 15, 2024, 04:42 AM UTC
The issue has been fully resolved. The issue started when heavy image upscaling API requests overloaded the upload API, causing some API requests to return 504 errors. This was solved but a knock-on effect caused approximately 5% of API requests to fail with a 502 response. To prevent this issue from happening again, improvements have been made to reduce the load of image upscaling and better distribute load. Changes are also being made to logging and alerting, for faster identification of such scenarios. The greatest impact of this issue was on the Sirv WordPress plugin. The plugin was sending an API request on all admin pages, which blocked the page from loading until the response had been received - typically less than 300ms but up to 20 seconds if API requests timed out. Those API requests were not required, so a new WordPress plugin version v7.3.3 was released at 1805 UTC (14 November) to remove such requests and prevent this issue from recurring: https://wordpress.org/plugins/sirv/ If you use the Sirv API, we recommend that your scripts are written to retry requests that receive a 502 response. Recommendations for handling 502 and other error responses are provided here: https://sirv.com/help/articles/sirv-rest-api/#error-handling-guide