Form.io incident

Intermittent API Failures

Major Resolved View vendor source →

Form.io experienced a major incident on April 9, 2021, lasting —. The incident has been resolved; the full update timeline is below.

Started
Apr 09, 2021, 08:00 AM UTC
Resolved
Apr 09, 2021, 08:00 AM UTC
Duration
Detected by Pingoru
Apr 09, 2021, 08:00 AM UTC

Update timeline

  1. resolved Apr 09, 2021, 11:20 AM UTC

    We have started to see intermittent API failures.

  2. postmortem Apr 09, 2021, 02:20 PM UTC

    The failure was caused by a single instance within the Load Balancer intermittently crashing causing the ELB to restart the servers. The failure of the single instance was diagnosed as a full internal hard drive causing certain processes to hang. The resolution was to remove the failing instance and replace it with a fresh instance. We have also sequentially replaced all other instances in our production load balancer with fresh instances to further solidify the deployment.