Flow Swiss Outage History

Started Feb 24, 2026, 01:15 AM UTC· resolved Feb 24, 2026, 05:13 AM UTC

1. investigating Feb 24, 2026, 01:15 AM UTC

   Block sealing on Flow mainnet has stopped. We are investigating the issue.

2. identified Feb 24, 2026, 04:07 AM UTC

   Flow is still in maintenance as the team works to restore full operations. The issue was caused by an incomplete node update in preparation for tomorrow’s height coordinated upgrade, which triggered a consensus safety mechanism that halted block sealing. Flow engineers are actively working on a fix, and the next update will be shared at 10 PM PT.

3. monitoring Feb 24, 2026, 04:59 AM UTC

   The root cause of the issue has been resolved, and block production has resumed, though it is currently progressing slowly. Consensus node operators are actively upgrading their nodes, and once a majority of the consensus nodes in the network have completed the update, block sealing is expected to return to normal performance.

4. resolved Feb 24, 2026, 05:13 AM UTC

   The incident has been resolved and the network has resumed.

5. postmortem Feb 25, 2026, 01:43 AM UTC

   ### **Summary** On February 23, 2026, Flow Mainnet experienced an outage that temporarily halted block production. The network entered a safe state after detecting inconsistent results between nodes during preparations for a scheduled height coordinated upgrade. As designed, the protocol stopped sealing new blocks to protect network integrity. No user funds were impacted or at risk. The issue has since been resolved and normal operations have resumed. ### **What Happened** In preparation for a scheduled height coordinated upgrade, a new software version was rolled out to one execution node ahead of the coordinated update. That node generated results which did not match those produced by the other execution nodes. At the same time, one verification node was running an older software version. Running an older verification node version by itself would not have caused an outage. However, in this specific situation, the combination of an execution node and a verification node operating on mismatched versions contributed to inconsistent approvals being produced. The consensus safety mechanism detected that the required agreement conditions for sealing a block were not met and halted block production to prevent inconsistent state from being finalized. While this safety mechanism worked as intended, it resulted in temporary network downtime until the mismatch was identified and corrected. ### **Resolution** The version mismatches were identified and corrected, and nodes were aligned to the expected software versions. Once consistency was restored, block production resumed. Performance returned to normal as consensus node operators completed their updates. ### **Action Items** To reduce the likelihood of similar incidents in the future, the following actions are being implemented: 1. Height Coordinated Upgrade protections for verification nodesVerification nodes will soon adopt the upgrade enforcement behavior similar to execution nodes. If a verification node is not running the expected version after a coordinated upgrade, it will automatically crash and restart rather than continue participating. 2. CI and deployment safeguards for Cadence versionsA strengthening of the deployment process to ensure that node software upgrade cannot deploy a lower version of the node software. Automated checks will prevent accidental deployment of incorrect or older versions, reducing the risk of human error. ### **Closing** Network reliability and safety remain our top priorities. In this case, the protocol’s safety mechanisms correctly prevented inconsistent state from being finalized. The improvements outlined above will further strengthen Flow Mainnet’s resilience and upgrade processes going forward.

Read the full incident report →

Started Feb 12, 2026, 03:21 AM UTC· resolved Feb 12, 2026, 05:33 AM UTC

1. investigating Feb 12, 2026, 03:21 AM UTC

   Block production on testnet has halted. We are currently investigating the issue. No impact on mainnet.

2. investigating Feb 12, 2026, 04:17 AM UTC

   Flow Testnet will be undergoing a network upgrade (spork) to recover from the current issue.

3. resolved Feb 12, 2026, 05:33 AM UTC

   The issue has been resolved with a full network upgrade (spork) on testnet

4. postmortem Feb 12, 2026, 09:29 PM UTC

   Yesterday 2/11, Flow Testnet experienced an outage during routine maintenance. ### **What Happened** The Flow team was performing maintenance to ensure testnet consensus nodes had enough disk space. During this work, a disk space reclamation process was run. There was a flaw in this process when applied to consensus nodes that caused the DKG file to be deleted. Once a majority of the nodes were updated, they were no longer able to finalize blocks. As a result, finalization halted and the network stopped progressing. _Please note: Mainnet was not impacted._ ### **Recovery** Possible recovery paths were evaluated. The safest and least disruptive way to restore the network was to perform a full network upgrade on testnet. The upgrade restored consensus and returned the network to a healthy state. ### **EVM Follow Up** After the upgrade, EVM nodes encountered a separate issue and were not able to smoothly transition to the new network state. Additional remediation was required to bring EVM services back into sync. ### **Next Steps** * Updating and testing the disk space reclamation process, adding more safety checks to ensure critical consensus state is preserved, along with clearer documentation. * Improving the EVM gateway to make it more robust during network upgrades and transitions. * Reviewing alerting and operational safeguards to surface issues more quickly.

Read the full incident report →

Started Jan 28, 2026, 07:36 PM UTC· resolved Jan 28, 2026, 07:59 PM UTC

1. investigating Jan 28, 2026, 07:36 PM UTC

   Transaction ingestion is currently degraded. Testnet is also in the Epoch Fallback Mode.

2. resolved Jan 28, 2026, 07:59 PM UTC

   The incident has been resolved and transactions ingestion has recovered. Testnet will also automatically recover from the Epoch Fallback Mode (thanks to Epoch Fallback Recover mechanism introduced last year - https://github.com/onflow/flow-go/releases/tag/v0.39.0)

Read the full incident report →

Started Jan 07, 2026, 04:38 PM UTC· resolved Jan 07, 2026, 05:48 PM UTC

1. investigating Jan 07, 2026, 04:38 PM UTC

   The public EVM endpoint is currently experiencing degraded performance. We’re working with our partners to restore normal service. This is both on Testnet and Mainnet.

2. resolved Jan 07, 2026, 05:48 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Dec 27, 2025, 01:15 PM UTC· resolved Dec 28, 2025, 03:28 AM UTC

1. investigating Dec 27, 2025, 01:15 PM UTC

   The chain is currently experiencing an issue. We are investigating and will provide updates as they become available.

2. identified Dec 27, 2025, 04:39 PM UTC

   The Flow Foundation is currently investigating a potential security incident affecting the Flow network. Validators have worked together to halt transaction ingestion (the network won't be processing new transactions for the duration of the halt.) Flow Foundation is actively collaborating with network partners and relevant authorities to address the issue as quickly as possible. Please stay tuned for timely ongoing updates.

3. resolved Dec 28, 2025, 03:28 AM UTC

   The root cause of the issue has been resolved and the network has been upgraded. See maintenance incident: https://manage.statuspage.io/pages/ytw5bdg6zr13/incidents. However, the collection ingestion is still paused.

Read the full incident report →

Started Dec 05, 2025, 10:15 PM UTC· resolved Dec 06, 2025, 06:43 AM UTC

1. investigating Dec 05, 2025, 10:15 PM UTC

   We are currently aware of an issue where the EVM Endpoint is not responding or is responding significantly slower than expected. This may affect applications relying on EVM RPC calls and related functionality. Current Action: We are actively investigating the root cause and working to restore normal performance as quickly as possible. Next Update: We will provide an update as soon as more information becomes available. Thank you for your patience.

2. monitoring Dec 06, 2025, 01:23 AM UTC

   We’re seeing temporary degraded performance on the EVM gateway due to elevated traffic during a scheduled mainnet load test earlier in the day. Flow mainnet is operating normally, but the indexing lag on the gateways is causing reduced responsiveness for Flow EVM apps. We are working on scaling the gateway capacity and bringing indexing back to real time now. We’ll share another update once everything is fully back to normal

3. monitoring Dec 06, 2025, 02:23 AM UTC

   All EVM gateway nodes have now fully caught up and are serving traffic normally. However, we are observing a high volume of clients attempting to backfill data, which is causing elevated load on the nodes. As a result, response times may remain higher than usual while the system processes this backlog. Our team is actively monitoring node performance and working to mitigate saturation. We will provide another update as conditions improve.

4. resolved Dec 06, 2025, 06:43 AM UTC

   Gateway performance has fully recovered and Flow EVM endpoint is now operating normally. Flow mainnet remained healthy throughout and we will continue monitoring to ensure everything stays stable. Thanks for your patience.

5. postmortem Dec 10, 2025, 11:08 PM UTC

   # **Postmortem: EVM Endpoint Incident - December 5, 2025** ## **Introduction** On December 5, 2025, the Flow EVM JSON-RPC endpoint experienced five hours of unresponsiveness, followed by an additional five hours of degraded performance characterized by very high response times. During this period, the chain itself remained fully operational and continued producing blocks normally but EVM based apps and wallets had trouble interacting with the chain. This postmortem summarizes what happened and the steps we are taking to prevent similar issues in the future. ## **Summary** A load test earlier that morning revealed an inherent limitation in the current EVM Gateway architecture: gateways that are simultaneously ingesting blocks with a high number of transactions and serving RPC traffic can fall behind the chain. Once behind, they become unable to serve RPC requests efficiently. This caused the public EVM endpoint to be non-responsive for several hours, followed by degraded performance until additional gateway nodes were added. Horizontal scaling ultimately restored service to normal levels. ## **Action Items to Prevent Future Incidents** ### **1. Gateway Performance Investigation** * Deep-dive into the root contention between block ingestion and RPC serving. * Reproduce the issue in controlled environments to isolate performance bottlenecks. * Implement architectural or performance improvements to ensure gateways remain responsive under heavy load. ### **2. Stronger Detection & Alerting** * Add detailed alerting for RPC latency and slow endpoint behavior for the public endpoints.

Read the full incident report →

Started Jul 10, 2025, 07:02 PM UTC· resolved Jul 10, 2025, 07:22 PM UTC

1. resolved Jul 10, 2025, 07:10 PM UTC

   The issue has been resolved and the network has fully recovered. Please note that this incident impacted Testnet only — Mainnet remained unaffected and continues to operate normally.

2. postmortem Jul 11, 2025, 11:23 PM UTC

   ## **Postmortem: Testnet Sealing and Finalization Interruption – July 10, 2025** **Summary** On July 10, 2025, from 11:02 AM to 11:22 AM PST, Testnet experienced a temporary halt in block sealing and finalization. The issue was identified and resolved within 20 minutes. Mainnet remained fully operational and was not affected at any point. **Root Cause** An internal testing framework inadvertently interfered with Testnet node communications, resulting in a disruption to the consensus process. This occurred due to overlapping configurations between the testing environment and Testnet. **Resolution and Preventative Measures** To ensure this does not happen again: * Firewall rules have been updated to sandbox the internal testing framework, fully isolating it from Testnet environments. * The testing framework has also been modified to avoid any attempts to connect to or interact with the Testnet network. Thank you, Flow Team

Read the full incident report →

Started Jun 12, 2025, 07:36 PM UTC· resolved Jun 12, 2025, 11:31 PM UTC

1. monitoring Jun 12, 2025, 07:36 PM UTC

   The Flow mainnet and testnet are functioning normally. However, we are currently monitoring a Google Cloud outage that may impact some API endpoints and third-party integrations. While core network operations are unaffected, services relying on Google-hosted components may experience intermittent issues or degraded performance. We will continue to monitor the situation closely and provide updates as more information becomes available. Thank you for your patience.

2. resolved Jun 12, 2025, 11:31 PM UTC

   The Google Cloud outage has now been resolved. We can confirm that there was no impact on Flow mainnet or testnet during this period. All systems remain fully operational. Thank you for your understanding.

Read the full incident report →

Started May 23, 2025, 01:17 PM UTC· resolved May 23, 2025, 06:42 PM UTC

1. investigating May 23, 2025, 01:17 PM UTC

   We are currently investigating this issue.

2. investigating May 23, 2025, 03:12 PM UTC

   We have identified the fix, we are preparing upgrade on Testnet

3. resolved May 23, 2025, 06:42 PM UTC

   This incident has been resolved.

4. postmortem Jun 09, 2025, 03:05 AM UTC

   ## 📅 Incident Summary * **Date**: May 23, 2025 * **Duration**: 6:00 AM – 10:00 AM Pacific Time * **Impact**: Transaction execution on Flow Testnet halted. * **Status**: Resolved ## 🧨 Root Cause A specific transaction submitted to Flow Testnet triggered an unhandled edge case in the Cadence resulting in a complete halt in transaction processing. ## ✅ Resolution & Fixes Several targeted Cadence code fixes were implemented promptly to mitigate the issue: * 🛠️ **Parser fixes** to handle the edge case. * 🛠️ **Error reporting improvements** for better diagnostics. * 🛡️ **Defensive parsing logic** added via [Cadence PR #3974](https://github.com/onflow/cadence/pull/3974) These fixes were deployed through an Height Coordinated Upgrade \(HCU\) on testnet and then subsequently an HCU mainnet on the same day. ## 🧪 Prevention & Follow-Up Actions To avoid similar incidents in the future, the following preventive measures are being implemented: ### **1. Enhanced Fuzz Testing** * Setup regularly run of the `cadence-fuzzer`, a tool designed to generate and test random Cadence programs as part of CI/CD \(Cadence issue [#3985](https://github.com/onflow/cadence/issues/3985)\) ### **2. Consider Pre-execution Simulation \(long-term\)** * Introduce simulation of transactions in a controlled environment \(e.g., Access Node\) before execution on core nodes to catch anomalies early. However, this is a long term strategy and needs to be investigated further.

Read the full incident report →

Started Apr 30, 2025, 11:11 PM UTC· resolved May 01, 2025, 01:05 AM UTC

1. investigating Apr 30, 2025, 11:11 PM UTC

   Metamask is currently experiencing issues on Flow EVM, causing their wallet transactions to fail. The Flow team is actively working to resolve the issue. In the meantime, the Flow EVM network and all other wallets, such as Flow Wallet, continue to operate normally.

2. resolved May 01, 2025, 01:05 AM UTC

   This incident has now been resolved, and Metamask transactions on Flow EVM are now working normally.

Read the full incident report →

Started Apr 08, 2025, 03:13 PM UTC· resolved Apr 08, 2025, 03:43 PM UTC

1. investigating Apr 08, 2025, 03:13 PM UTC

   We are currently investigating this issue.

2. identified Apr 08, 2025, 03:25 PM UTC

   A fix has been rolled out and we are monitoring recovery

3. resolved Apr 08, 2025, 03:43 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Mar 13, 2025, 04:55 AM UTC· resolved Mar 14, 2025, 05:01 AM UTC

1. investigating Mar 13, 2025, 04:55 AM UTC

   We are currently observing a lower-than-usual block finalization rate (blocks/second) on Flow mainnet. As a result, users may experience increased time to finalization and time to seal for their transactions. We are investigating the issue, and we will provide updates as we gather more information.

2. monitoring Mar 13, 2025, 05:40 PM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Mar 14, 2025, 05:01 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Mar 06, 2025, 09:23 PM UTC· resolved Mar 08, 2025, 12:56 AM UTC

1. investigating Mar 06, 2025, 09:23 PM UTC

   Hello, Testnet is currently in the Epoch Fallback mode and epoch transitions are paused. Staking operations will not take effect till the network resumes regular epoch transition. Everything else on testnet is working as expected. This is part of the testing of the new Epoch Fallback Mode recovery process. Thank you.

2. resolved Mar 08, 2025, 12:56 AM UTC

   The test of the Epoch Fallback Mode recovery process was successful. Flow testnet has now automatically come out of the Epoch Fallback Mode without incurring any downtime. Staking operation on testnet have resumed.

Read the full incident report →

Started Mar 05, 2025, 05:37 PM UTC· resolved Mar 05, 2025, 06:02 PM UTC

1. investigating Mar 05, 2025, 05:37 PM UTC

   We are currently investigating this issue.

2. investigating Mar 05, 2025, 05:40 PM UTC

   We have rolled out a fix and are monitoring recovery.

3. resolved Mar 05, 2025, 06:02 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 31, 2024, 04:10 AM UTC· resolved Oct 31, 2024, 06:03 AM UTC

1. investigating Oct 31, 2024, 04:10 AM UTC

   Flow mainnet is currently experiencing a networking issue. We are working on resolving it.

2. investigating Oct 31, 2024, 05:00 AM UTC

   We are continuing to investigate this issue.

3. monitoring Oct 31, 2024, 05:34 AM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Oct 31, 2024, 06:03 AM UTC

   This incident has been resolved.

Read the full incident report →