Portal Labs had 25 outages in the last 2 years totaling 34m of downtime — averaging 1 incident per month.
There were 25 Portal Labs outages since November 9, 2024 totaling 34m of downtime. Each is summarised below — incident details, duration, and resolution information.
We are currently investigating this incident.
Our upstream vendor implemented a fix impacting the GET endpoint on EVM chains and are currently monitoring the result.
This incident has been resolved.
Type: Incident Duration: 10 minutes Affected Components: Enclave MPC API, MPC Operations Apr 16, 18:03:00 GMT+0 - Identified - We are working on a fix for this incident. This should get resolved shortly. Apr 16, 18:13:00 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 26 minutes Affected Components: Portal Admin Dashboard Mar 3, 21:19:12 GMT+0 - Investigating - We are currently investigating this incident. Mar 3, 21:42:11 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Mar 3, 21:45:25 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 2 hours and 44 minutes Affected Components: Enclave MPC API, MPC Operations Feb 2, 20:26:27 GMT+0 - Investigating - We are currently investigating this incident. Feb 2, 22:59:02 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Feb 2, 23:10:00 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 1 hour and 40 minutes Affected Components: MPC Operations Jan 27, 21:50:00 GMT+0 - Investigating - Jan 27, 22:30:00 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Jan 27, 23:30:29 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 26 minutes Affected Components: Custodian API, Enclave MPC API, MPC Operations, Client API, Alert Webhook Relayer, Portal Admin Dashboard Jan 14, 18:38:15 GMT+0 - Monitoring - We noticed an issue with our database connection. We have already implemented a fix and are currently monitoring the result. Service should resume in a few minutes. Jan 14, 19:04:04 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 3 hours and 26 minutes Affected Components: MPC Operations Dec 2, 15:36:35 GMT+0 - Investigating - We are currently investigating this incident. Dec 2, 16:06:10 GMT+0 - Identified - We are continuing to work on a fix for this incident. Dec 2, 18:24:26 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Dec 2, 19:02:18 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 2 hours and 56 minutes Affected Components: Enclave MPC API Dec 1, 20:56:15 GMT+0 - Investigating - We are currently investigating this incident. Dec 1, 20:56:43 GMT+0 - Identified - Downstream outage with our RPC Provider, working to fix. Dec 1, 23:52:00 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 5 hours and 4 minutes Affected Components: Alert Webhook Relayer Nov 12, 11:10:00 GMT+0 - Investigating - We are currently investigating this incident. Nov 12, 16:03:42 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Nov 12, 16:13:39 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 2 hours and 37 minutes Affected Components: Alert Webhook Relayer Oct 8, 21:43:25 GMT+0 - Resolved - This incident has been resolved. Oct 8, 19:06:00 GMT+0 - Identified - We are continuing to work on a fix for this incident. Oct 8, 21:26:59 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result.
Type: Incident Duration: 1 hour and 17 minutes Affected Components: Enclave MPC API, MPC Operations Sep 19, 17:39:34 GMT+0 - Identified - We identified the issue and are working on a fix for this incident. Sep 19, 17:48:53 GMT+0 - Resolved - This incident has been resolved. Sep 19, 16:32:21 GMT+0 - Investigating - We are currently investigating this incident.
Type: Incident Duration: 17 minutes Affected Components: MPC Operations, Enclave MPC API Aug 27, 18:35:00 GMT+0 - Investigating - We are currently investigating this incident. Aug 27, 18:52:03 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 1 hour and 5 minutes Affected Components: Client API Aug 27, 18:25:00 GMT+0 - Investigating - We are currently investigating this incident. Aug 27, 19:30:00 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 1 day, 1 hour and 14 minutes Affected Components: MPC Operations, Custodian API, Enclave MPC API, Client API, Alert Webhook Relayer, Portal Admin Dashboard Jun 12, 18:50:00 GMT+0 - Monitoring - Health checks appear to be stabilizing and performance appears to be improving. Jun 12, 18:00:00 GMT+0 - Identified - We're experiencing degraded service due to a global disruption to cloud providing services. Our testing shows that all of our backend services are operating as expected, but networking issue are causing intermittent failures. We are monitoring. Jun 12, 19:13:36 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 11 hours and 51 minutes Affected Components: MPC Operations, Enclave MPC API May 2, 10:27:00 GMT+0 - Investigating - We are currently investigating this incident. May 2, 22:17:58 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 3 hours and 49 minutes Affected Components: Enclave MPC API Apr 24, 17:58:39 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Apr 24, 21:41:22 GMT+0 - Resolved - This incident has been resolved. Apr 24, 17:52:41 GMT+0 - Investigating - Enclave MPC API request times are taking abnormally long. We are currently investigating this incident. Apr 24, 19:34:03 GMT+0 - Monitoring - We're continuing to scale our systems and are monitoring the result. We expect to see request times decrease shortly.
Type: Incident Duration: 1 hour and 30 minutes Affected Components: Enclave MPC API Apr 15, 17:22:00 GMT+0 - Investigating - We are currently investigating this incident. Apr 15, 17:55:00 GMT+0 - Resolved - This incident has been resolved. Apr 15, 17:54:49 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result.
Type: Incident Duration: 4 minutes Affected Components: Client API Mar 13, 15:24:08 GMT+0 - Resolved - This incident has been resolved. Mar 13, 15:20:30 GMT+0 - Investigating - We are currently investigating this incident.
Type: Incident Duration: 1 hour and 37 minutes Feb 13, 23:10:00 GMT+0 - Identified - `GET /api/v3/clients/me/chains/:chain/assets`[ endpoint](https://docs.portalhq.io/reference/client-api/v3-endpoints#get-the-clients-assets-by-chain) is currently experiencing downtime for EVM and Solana chains. We have identified the issue and are working on a fix. Feb 14, 00:47:13 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 16 minutes Affected Components: MPC Operations Jan 30, 16:20:00 GMT+0 - Investigating - We are currently investigating this incident. Jan 30, 16:36:00 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 1 day, 1 hour and 52 minutes Affected Components: MPC Operations Jan 29, 23:14:00 GMT+0 - Investigating - We are currently investigating this incident. Jan 30, 13:06:00 GMT+0 - Resolved - This incident has been resolved.
Type: Incident Duration: 8 minutes Affected Components: MPC Operations Jan 29, 17:33:13 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Jan 29, 17:35:33 GMT+0 - Resolved - The issue has now been resolved. Jan 29, 17:27:30 GMT+0 - Investigating - We are currently investigating this incident. Jan 29, 17:32:15 GMT+0 - Identified - A fix is being deployed.
Type: Maintenance Duration: 30 minutes Affected Components: Portal Admin Dashboard, MPC Operations Dec 14, 08:00:00 GMT+0 - Identified - We are planning for a scheduled maintenance for **30 minutes** of downtime Friday night at midnight. Dec 14, 08:30:00 GMT+0 - Completed - Maintenance has completed successfully Dec 14, 08:00:01 GMT+0 - Identified - Maintenance is now in progress
Type: Maintenance Duration: 3 minutes Affected Components: MPC Operations Dec 13, 08:00:00 GMT+0 - Identified - We are planning for a scheduled maintenance for **3 minutes** of downtime Thursday night at midnight. Dec 13, 08:03:00 GMT+0 - Completed - Maintenance has completed successfully Dec 13, 08:00:01 GMT+0 - Identified - Maintenance is now in progress
Type: Incident Duration: 1 hour Affected Components: MPC Operations Nov 10, 00:00:00 GMT+0 - Monitoring - Identified root issue and monitoring fix Nov 9, 23:00:00 GMT+0 - Investigating - Investigating degraded network performance of mpc operations Nov 10, 00:00:00 GMT+0 - Resolved - This incident has been resolved.