Sirv experienced a minor incident on March 16, 2023 affecting Sirv CDN (CDN request from London), lasting 4h 5m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 16, 2023, 12:17 PM UTC
Some requests to the London POP of Sirv CDN, are returning errors due to high load. This POP serves UK and Ireland. We are working on a resolution.
- resolved Mar 16, 2023, 04:22 PM UTC
Some requests received by one of the London CDN servers failed due to a combination of local DNS resolution and pulsating network timeouts. The issue began at 1215 UTC and was resolved at 1332 UTC. During this period, 25% of requests from UK and Ireland returned a 404 or 500 error. The issue would have been solved sooner but an unrelated SSL certificate validation failure a few hours earlier blocked alerts from being received by our monitoring system. Changes have already been deployed to prevent this issue from recurring. The SSL certificate issue has been resolved; a second level of independent monitoring has been configured; an additional health automation is being trialled that has the potential to resolve such issues automatically. Classified as severity level 3: Minor impact.