Form3 Staging experienced a minor incident on December 9, 2021, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Dec 09, 2021, 01:36 PM UTC
We have identified an issue impacting the FPS Direct simulator between 09:50 - 12:59. During this time admission and submissions were unavailable. A period of monitoring confirmed no ongoing impact.
- postmortem Dec 13, 2021, 05:00 PM UTC
On 09 December between 09:50 - 12:59 UTC, there was a degradation to FPS test payment processing on the FPS Simulator. Outbound payments were stuck in released\_to\_gateway and inbound payments were not received. Any test payments impacted during this period will need to be replayed. Investigations identified that Form3 testing was being carried out in the Staging environment, in relation to a planned non-customer impacting change that was scheduled for deployment to in Production. The testing, combined with the planned maintenance that was underway resulted in the Staging simulator not being in a high-availability configuration. A change has been made to amend the CI simulator deployment to be in a high-availability configuration to prevent a future reoccurrence.