Sirv experienced a minor incident on February 22, 2024 affecting Current status by service (Sirv primary datacenter), lasting 1h 14m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Feb 22, 2024, 10:52 AM UTC
Some files are not being returned for a small number of accounts. This issue is being addressed urgently.
- resolved Feb 22, 2024, 12:06 PM UTC
The issue has been resolved. At 1052 UTC, about 11% of requests to a small number of accounts (12 accounts, out of 50,000+) were returning 404 responses (file not found) when they should have returned 200 responses (OK). When the Sirv team identified the issue, they quickly worked to solve it and by 1206 UTC all such requests were returning the normal 200 response. The issue occurred due to a misconfiguration by Sirv engineers while they were configuring a new storage cluster shard. Some requests were going to the new shard before it was ready to receive them. Classified as severity level 3: Minor impact.