Banked experienced a major incident on October 30, 2023 affecting API and Checkout, lasting 4h 35m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 30, 2023, 11:28 AM UTC
We are currently experiencing a performance degradation to the payments and checkout service. Investigations are ongoing and an update will be provided as soon as more information becomes available.
- investigating Oct 30, 2023, 11:45 AM UTC
We are continuing to investigate this issue.
- identified Oct 30, 2023, 11:47 AM UTC
The issue has been identified and a fix is being implemented.
- identified Oct 30, 2023, 11:58 AM UTC
We are continuing to work on a fix for this issue.
- monitoring Oct 30, 2023, 12:01 PM UTC
A fix has been implemented and we are monitoring the results.
- monitoring Oct 30, 2023, 12:04 PM UTC
We are continuing to monitor for any further issues.
- monitoring Oct 30, 2023, 12:39 PM UTC
We are still continuing to monitor the results following the earlier fix.
- monitoring Oct 30, 2023, 01:14 PM UTC
We are still continuing to monitor the results following the earlier fix.
- monitoring Oct 30, 2023, 02:07 PM UTC
We are still continuing to monitor the results following the earlier fix.
- monitoring Oct 30, 2023, 02:43 PM UTC
We are still continuing to monitor the results following the earlier fix.
- resolved Oct 30, 2023, 04:04 PM UTC
This incident has been resolved. Further details will be provided in the postmortem in due course.
- postmortem Nov 01, 2023, 07:09 PM UTC
On October 30th, 2023 between 11:18 -11:43 UTC, Banked experienced a partial outage in the payments API and checkout product in the UK/EU region. During the outage, about half of the attempts to create payments and checkout failed. This issue occurred due to a failure in a routine upgrade of a component in our infrastructure. Our on-call team responded immediately to the failure and stopped the upgrade to investigate with our cloud service provider. We are currently working with our cloud service provider to prevent similar issues from occurring in the future.