Yandex Cloud experienced a minor incident on May 21, 2020 affecting Identity and Access Management and ru-central1-a and 1 more component, lasting 38m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 21, 2020, 04:38 PM UTC
We noticed some problems with IAM queries. Some users may receive timeout errors. The developer team is notified and currently investigating this issue.
- identified May 21, 2020, 05:22 PM UTC
The issue was solved at 20:08 (GMT +3). We're investigating the causes of the problem.
- resolved May 25, 2020, 04:24 PM UTC
During the resource container (cloud or directory) deletion as a result of user blocking, it's necessary to clear the resources of various services, including IAM. We solve this task with a particular internal service – Resource Reaper. The IAM resource removal query is designed for less load than the one which the service was trying to use for removal. It worked correctly, but on May 21, it ceased to fit into a timeout. It led to an avalanche-like increase in the number of simultaneously executing queries of this kind. As a result, other queries to the same data began to run much slower. The Cloud team is already working on a systematic solution to this problem.