Plainly experienced a major incident on November 6, 2025 affecting Integrations, lasting 5d 23h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 06, 2025, 09:24 AM UTC
We’re currently investigating an issue with integrations not working as expected. Our team is in contact with the integration service provider to understand more and will share updates as soon as we have additional information.
- identified Nov 06, 2025, 01:02 PM UTC
The integration service provider has identified the issue causing integrations to fail. We’re waiting for their next update and will continue to monitor the situation closely.
- identified Nov 07, 2025, 07:23 AM UTC
The integration service provider has identified the root cause of the issue, but the fix is taking longer than initially expected to be applied. We’re staying in close contact with them and will continue to provide updates as progress is made.
- identified Nov 07, 2025, 07:53 AM UTC
Our integration service provider shared the following update: “We are experiencing an infrastructure-related outage that has extended beyond our initial recovery estimate. Our full DevOps team is actively working on manual recovery procedures, with management oversight to ensure this issue receives absolute priority.” We’ll continue to share updates as we receive them. As a gesture of goodwill, we’re happy to refund any rendering usage during the incident. Please contact our support team if you were affected.
- identified Nov 07, 2025, 10:54 AM UTC
Latest update from the integration service provider: "We’ve fixed the root cause affecting our Ceph cluster, and the cluster is now being restored. The issue turned out to be a bug in ceph cluster, where the Python sub-interpreter used by the mgr modules wasn’t loading plugins correctly. In the meantime, our team has fixed the manager side and is working on restoring the broken OSDs. We expect this to take the next 2-3 hours."
- identified Nov 07, 2025, 03:35 PM UTC
Another update from the integration service provider: "We’re still working on the broken OSDs — it’s taking longer than expected to get them all back online."
- identified Nov 09, 2025, 08:39 AM UTC
The integration service provider’s infrastructure outage is still ongoing, and unfortunately, we haven’t received a recent update from their team. We’ve reached out multiple times and are awaiting further information on recovery progress. We understand how disruptive this is and sincerely apologize for the extended downtime. We’ll continue to share updates as soon as we receive new information.
- identified Nov 10, 2025, 09:46 AM UTC
The integration service provider has prepared a new cluster and started moving affected customers to the new cluster. We’re hopeful the work will be completed by the end of the day and will share updates as soon as integration services are back online.
- identified Nov 11, 2025, 08:44 AM UTC
We just received information from our integration service provider that new AWS infrastructure is ready, and tenant restoration will begin today in phases. We’ve been assured that the service will be fully restored today, and we’ll share updates as progress continues.
- monitoring Nov 11, 2025, 06:29 PM UTC
The issue has been resolved, and we’re now moving into the monitoring phase. We urge all customers to check their integrations and ensure connected accounts are working properly, as integrations may fail if any accounts are disconnected. A detailed post-incident report will be shared once the incident is fully closed.
- resolved Nov 12, 2025, 08:54 AM UTC
The integration service has been fully restored, and after an extended monitoring period, we’re closing this incident. We sincerely apologize for the disruption this caused and thank all affected customers for their patience and understanding throughout the outage. Going forward, we’ll implement several improvements to strengthen reliability and transparency: - Improve user notifications in case of integration service failures. Currently, these alerts did not trigger because the system responsible for notifications was also impacted. - Add a notification when an integration cannot be triggered after a successful rendering. - Provide some integrations as built-in features (e.g., signed URL uploads, form-based render triggering, and others) to reduce dependency on third-party services. - Adopt a more proactive update policy for third-party software to stay on the latest stable versions.