Portal Labs had 26 outages in the last 2 years totaling 11h 25m of downtime — averaging 1.1 incidents per month.
There were 26 Portal Labs outages since December 14, 2024 totaling 11h 25m of downtime. Each is summarised below — incident details, duration, and resolution information.
We are currently investigating this incident.
We implemented a fix and are currently monitoring the result.
This incident has been resolved.
We are currently investigating this incident.
This incident has been resolved.
Type: Incident Duration: 5 hours and 26 minutes Affected Components: Alert Webhook Relayer Sep 2, 09:45:00 GMT+0 - Investigating - We are currently investigating this incident. Sep 2, 13:24:09 GMT+0 - Identified - We are continuing to work on a fix for this incident. Sep 2, 15:11:10 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 20 minutes Affected Components: Client API Aug 31, 14:28:00 GMT+0 - Investigating - We are currently investigating this incident. Aug 31, 14:49:00 GMT+0 - Resolved - This incident has been resolved.
We are currently investigating this incident.
This incident has been resolved.
We are currently investigating this incident.
This incident has been resolved.
The Client API endpoint \`/api/v3/clients/me/chains/{chainId}/assets\` is returning \`Unsupported Chain\` for valid chains. We are working on a fix for this incident.
We implemented a fix and are currently monitoring the result.
This incident has been resolved.
We are currently investigating elevated latency and intermittent errors affecting balance/asset reads and transaction building across several EVM networks (notably Arbitrum and Polygon) and Solana.
This incident has been resolved.
Type: Incident Duration: 1 hour and 7 minutes Affected Components: MPC Operations, Enclave MPC API, Client API Jun 18, 13:47:00 GMT+0 - Investigating - We are currently investigating elevated latency and intermittent errors affecting balance/asset reads and transaction building across several EVM networks (notably Arbitrum and Polygon) and Solana. Jun 18, 17:09:00 GMT+0 - Resolved - This incident has been resolved.
New wallets created since `Wed Jun 10, 20 PM UTC` are not receiving `SOLANA_TX_V1` alert webhook events
We implemented a fix and are currently monitoring the result.
This incident has been resolved.
We are currently investigating this incident.
This incident has been resolved.
We are currently investigating this incident.
Our upstream vendor implemented a fix impacting the GET endpoint on EVM chains and are currently monitoring the result.
This incident has been resolved.
Type: Incident Duration: 10 minutes Affected Components: Enclave MPC API, MPC Operations Apr 16, 18:03:00 GMT+0 - Identified - We are working on a fix for this incident. This should get resolved shortly. Apr 16, 18:13:00 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 26 minutes Affected Components: Portal Admin Dashboard Mar 3, 21:19:12 GMT+0 - Investigating - We are currently investigating this incident. Mar 3, 21:42:11 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Mar 3, 21:45:25 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 2 hours and 44 minutes Affected Components: Enclave MPC API, MPC Operations Feb 2, 20:26:27 GMT+0 - Investigating - We are currently investigating this incident. Feb 2, 22:59:02 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Feb 2, 23:10:00 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 1 hour and 40 minutes Affected Components: MPC Operations Jan 27, 21:50:00 GMT+0 - Investigating - Jan 27, 22:30:00 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Jan 27, 23:30:29 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 26 minutes Affected Components: Custodian API, Enclave MPC API, MPC Operations, Client API, Alert Webhook Relayer, Portal Admin Dashboard Jan 14, 18:38:15 GMT+0 - Monitoring - We noticed an issue with our database connection. We have already implemented a fix and are currently monitoring the result. Service should resume in a few minutes. Jan 14, 19:04:04 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 3 hours and 26 minutes Affected Components: MPC Operations Dec 2, 15:36:35 GMT+0 - Investigating - We are currently investigating this incident. Dec 2, 16:06:10 GMT+0 - Identified - We are continuing to work on a fix for this incident. Dec 2, 18:24:26 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Dec 2, 19:02:18 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 2 hours and 56 minutes Affected Components: Enclave MPC API Dec 1, 20:56:15 GMT+0 - Investigating - We are currently investigating this incident. Dec 1, 20:56:43 GMT+0 - Identified - Downstream outage with our RPC Provider, working to fix. Dec 1, 23:52:00 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 5 hours and 4 minutes Affected Components: Alert Webhook Relayer Nov 12, 11:10:00 GMT+0 - Investigating - We are currently investigating this incident. Nov 12, 16:03:42 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Nov 12, 16:13:39 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 2 hours and 37 minutes Affected Components: Alert Webhook Relayer Oct 8, 21:43:25 GMT+0 - Resolved - This incident has been resolved. Oct 8, 19:06:00 GMT+0 - Identified - We are continuing to work on a fix for this incident. Oct 8, 21:26:59 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result.
Type: Incident Duration: 1 hour and 17 minutes Affected Components: Enclave MPC API, MPC Operations Sep 19, 17:39:34 GMT+0 - Identified - We identified the issue and are working on a fix for this incident. Sep 19, 17:48:53 GMT+0 - Resolved - This incident has been resolved. Sep 19, 16:32:21 GMT+0 - Investigating - We are currently investigating this incident.
Type: Incident Duration: 17 minutes Affected Components: MPC Operations, Enclave MPC API Aug 27, 18:35:00 GMT+0 - Investigating - We are currently investigating this incident. Aug 27, 18:52:03 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 1 hour and 5 minutes Affected Components: Client API Aug 27, 18:25:00 GMT+0 - Investigating - We are currently investigating this incident. Aug 27, 19:30:00 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 1 day, 1 hour and 14 minutes Affected Components: MPC Operations, Custodian API, Enclave MPC API, Client API, Alert Webhook Relayer, Portal Admin Dashboard Jun 12, 18:50:00 GMT+0 - Monitoring - Health checks appear to be stabilizing and performance appears to be improving. Jun 12, 18:00:00 GMT+0 - Identified - We're experiencing degraded service due to a global disruption to cloud providing services. Our testing shows that all of our backend services are operating as expected, but networking issue are causing intermittent failures. We are monitoring. Jun 12, 19:13:36 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 11 hours and 51 minutes Affected Components: MPC Operations, Enclave MPC API May 2, 10:27:00 GMT+0 - Investigating - We are currently investigating this incident. May 2, 22:17:58 GMT+0 - Resolved - This incident has been resolved.