Solana experienced a major incident on January 21, 2022 affecting Mainnet Beta - Cluster, lasting 2d 16h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 21, 2022, 03:04 AM UTC
The mainnet-beta cluster is experiencing some performance degradation, we currently investigating the issue.
- investigating Jan 21, 2022, 12:52 PM UTC
We are continuing to investigate this issue.
- investigating Jan 22, 2022, 04:05 PM UTC
The mainnet-beta cluster is experiencing performance degradation, we currently investigating the issue.
- identified Jan 22, 2022, 05:55 PM UTC
Solana mainnet beta is experiencing high levels of network congestion. This is related to issues previously identified that engineers have been working to improve and resolve, starting with the 1.8 release. 1.8.12 was specifically aimed at improving issues of program cache exhaustion, which are largely responsible for recent performance degradations. Developers have made lots of progress on this, but the work is not complete. The last 24 hours have shown these systems need to be improved to meet the demands of users, and support the more complex transactions now common on the network. The current issue experienced by validators is due to excessive duplicate transactions. Engineers have just released 1.8.14, which will attempt to mitigate the worst effects of this issue. These forthcoming releases are aimed at improving the state of the network, with more improvements expected to roll out in the next 8-12 weeks. Many of these features are currently live on Testnet, where they are being rigorously tested. Release 1.8.14 is now available at https://github.com/solana-labs/solana/releases/tag/v1.8.14
- resolved Jan 23, 2022, 04:19 PM UTC
With the adoption of 1.8.14 the incident has been resolved.