Solana experienced a critical incident on February 25, 2023 affecting Mainnet Beta - Cluster, lasting 20h 8m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Feb 25, 2023, 06:37 AM UTC
Engineers are currently investigating slow root production on mainnet beta. 2023-02-25 05:46:16 UTC the Solana network entered degraded mode (vote-only). This means that economic activity on the network stopped while the network attempts to recover. Core contributors of the protocol are working to identify the root cause of this issue.
- monitoring Feb 25, 2023, 12:32 PM UTC
A coordinated restart is underway to address an issue during the upgrade from 1.13 to 1.14 that caused block finalization to slow significantly. Validators: please follow instructions below https://docs.google.com/document/d/e/2PACX-1vQzf7XpdoXB99lCWu6na9PhLy4SOx-xcjP1LlGiVR0PB6vpyGpol1c8_PufAa8TZZp_R0NlgaqlC7a1/pub
- monitoring Feb 25, 2023, 10:12 PM UTC
A coordinated restart is underway to address an issue during the upgrade from 1.13 to 1.14 that caused block finalization to slow significantly. Validators: please follow updated instructions below https://docs.google.com/document/d/e/2PACX-1vQVMKVqh1o-qF4Z8ATM-Qxa_FhT96qoCmqGHFroq4e1EbrHkfg-H8Q8Z_FRnX3mYOxRgBbK6yzDols_/pub
- monitoring Feb 26, 2023, 01:28 AM UTC
The Solana community successfully restarted the cluster - engineers will continue to monitor network performance
- resolved Feb 26, 2023, 02:09 AM UTC
This incident has been resolved.