Form.io incident

API Network Connectivity Issues

Resolved on July 27, 2023.

postmortem Jul 27, 2023, 01:24 AM UTC

This issue has been resolved. The problem was caused by a database update process that was creating a bottle neck in certain email lookups. This update made some of our indexes drop causing some of the queries within our system to be slow in performant. After a certain amount of time, these queries piled up and caused our system to go down. We have mitigated this problem by forcing our database updates to execute and then set the update versions to the correct version. We are also going to take steps to ensure that none of our deployed customers are not going to be affected by this version update.

Started
Jul 27, 2023, 12:30 AM UTC
Resolved
Jul 27, 2023, 12:30 AM UTC
Duration
–
Detected by Pingoru
Jul 27, 2023, 12:30 AM UTC

Update timeline

  1. resolved Jul 27, 2023, 12:55 AM UTC

    We are currently experiencing issues with our hosted platform. We are in the process of investigation.

  2. postmortem Jul 27, 2023, 01:24 AM UTC

    This issue has been resolved. The problem was caused by a database update process that was creating a bottle neck in certain email lookups. This update made some of our indexes drop causing some of the queries within our system to be slow in performant. After a certain amount of time, these queries piled up and caused our system to go down. We have mitigated this problem by forcing our database updates to execute and then set the update versions to the correct version. We are also going to take steps to ensure that none of our deployed customers are not going to be affected by this version update.

Your product runs on more than Form.io

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →