Form3 Staging incident

INC #27965 - Degradation to Bacs Simulator

Minor Resolved View vendor source →

Form3 Staging experienced a minor incident on December 6, 2021 affecting Bacs - Direct Participant, lasting 1h 29m. The incident has been resolved; the full update timeline is below.

Started
Dec 06, 2021, 12:30 PM UTC
Resolved
Dec 06, 2021, 01:59 PM UTC
Duration
1h 29m
Detected by Pingoru
Dec 06, 2021, 12:30 PM UTC

Affected components

Bacs - Direct Participant

Update timeline

  1. identified Dec 06, 2021, 12:30 PM UTC

    We have identified an issue impacting the Bacs simulator. Our engineers are currently investigating and further details will be posted as the investigation progresses

  2. resolved Dec 06, 2021, 01:59 PM UTC

    The Bacs simulator has recovered and normal service has resumed as of 13:57 UTC. We will continue to monitor the service to ensure there is no ongoing impact. Further details will be included in the Post Mortem.

  3. postmortem Dec 08, 2021, 09:46 AM UTC

    Form3 were alerted to latency in the Bacs simulator from 08:05-13:57 UTC, preventing payments reaching a final state. Investigations identified that a service needed to be reconfigured in the Staging environment. Once this was completed, all queued payments were processed. A period of monitoring confirmed no ongoing impact from 13:57 UTC and that all impacted payments reached the correct final state. Form3 continue to proactively monitor the Staging environment to identify service enhancements and to prevent impact to customer testing.