Optimism had 13 outages in the last 2 years — averaging 0.5 incidents per month.
There were 13 Optimism outages since July 23, 2024. Each is summarised below — incident details, duration, and resolution information.
Type: Incident Duration: 1 hour and 2 minutes Affected Components: Transaction Sequencing Mar 4, 23:54:08 GMT+0 - Investigating - Due to an investigation of network bottlenecks the Flashblocks streaming websocket endpoints are temporarily unavailable. Mar 5, 00:55:49 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 10 minutes Affected Components: Batch Submission, Public API, Withdrawals, , Deposits, Transaction Sequencing, Node Sync, Mainnet → Mar 4, 23:04:00 GMT+0 - Investigating - OP Mainnet has been observed to have various issues within its internal components causing delayed block building and sporadic chain stalls. We're investigating the incident. Mar 4, 23:10:00 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Mar 4, 23:14:00 GMT+0 - Resolved - This incident has been resolved, signified by the restored stable chain progression. Mar 4, 23:07:00 GMT+0 - Identified - The incident's root cause has been identified to be the problems with the networking infrastructure backing the communication between various critical block building components. We are continuing to work on a fix for this incident.
Type: Incident Duration: 4 hours and 2 minutes Affected Components: Transaction Sequencing Nov 19, 19:13:03 GMT+0 - Identified - The problem has been diagnosed and a fix is being worked on by the flashbots team. Nov 19, 21:30:24 GMT+0 - Monitoring - A fix has been applied and we are monitoring for stability. Nov 19, 21:36:43 GMT+0 - Resolved - This incident has been resolved, flashblocks are now operational again. Nov 19, 17:34:30 GMT+0 - Investigating - Following the activation of the Jovian hardfork, we detected some problems with op-rbuilder and are investigating the root cause. Nov 19, 17:45:20 GMT+0 - Investigating - To clarify, regular (non-flashblocks) usage is unaffected. Unsafe and safe head progression and the rest of the system are fully operational.
Type: Incident Duration: 22 minutes Affected Components: Public API Aug 19, 18:05:31 GMT+0 - Resolved - This incident has been resolved. Aug 19, 17:43:56 GMT+0 - Monitoring - We experienced an outage of our mainnet public endpoint during routine cluster upgrade. We implemented a fix and are currently monitoring the result.
Type: Incident Affected Components: Withdrawals May 14, 14:52:14 GMT+0 - Resolved - This is a test and not a real security incident. This is dedicated for testing our Incident Response Process in case of an emergency May 14, 15:08:28 GMT+0 - Resolved - Resolved DRILL TEST.
Type: Incident Duration: 16 minutes Affected Components: Withdrawals Apr 14, 15:47:34 GMT+0 - Investigating - This is test for testing our process for incident response this is a DRILL. Apr 14, 16:03:54 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 1 hour and 28 minutes Affected Components: Deposits, Withdrawals, Batch Submission, Node Sync, Public API, Transaction Sequencing Dec 5, 19:32:00 GMT+0 - Identified - OP Mainnet experienced a safe head stall from \~19:30- 21:00 UTC. The incident is already resolved and the safe head has caught up. Dec 5, 21:00:00 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 5 hours and 35 minutes Affected Components: Public API, Deposits, Node Sync, Withdrawals, Batch Submission, Transaction Sequencing Sep 6, 00:59:12 GMT+0 - Resolved - False alarm, no network degradation occurred Sep 6, 00:53:38 GMT+0 - Investigating - We are currently investigating this incident.
Type: Incident Duration: 19 minutes Affected Components: Node Sync, Transaction Sequencing Sep 3, 18:08:06 GMT+0 - Identified - We have identified the cause of the stall, and are deploying a mitigation. Nodes may need to wait for the next safe head increase before they resume syncing. Sep 3, 17:58:58 GMT+0 - Investigating - We are currently investigating this incident. Sep 3, 18:11:28 GMT+0 - Monitoring - We have deployed the mitigations and the network is syncing again. At this time the network is fully operational. Sep 3, 18:18:22 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 23 minutes Affected Components: Public API, Node Sync, Transaction Sequencing Aug 13, 17:23:25 GMT+0 - Resolved - This incident has been resolved, safe and unsafe chain are advancing normally. Thanks for your patience with us! Aug 13, 17:18:24 GMT+0 - Monitoring - The sequencer has caught up sequencing and the remaining network should pick up after the next safe head increase. This should resolve within the next few minutes. Aug 13, 17:00:00 GMT+0 - Investigating - We are investigating an OP Sepolia chain halt. All users are affected.
Type: Incident Duration: 16 minutes Affected Components: Withdrawals Aug 9, 16:03:16 GMT+0 - Identified - We have identified the problem and are working on a fix. Aug 9, 16:15:38 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Aug 9, 16:02:56 GMT+0 - Investigating - We are seeing a delay in our mainnet proposer instance submitting proposals on chain. Aug 9, 16:18:28 GMT+0 - Resolved - This incident has been resolved, proposals have resumed.