Breadcrumb experienced a major incident on October 2, 2022 affecting U-Series POS App and HQ and U-Series Payments, lasting 19h 16m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 02, 2022, 10:54 PM UTC
Our engineering team continues to investigate connectivity issues and your Upserve Point of Sale may be in offline mode. Offline Mode allows you to continue operating and collecting credit card payments. More on offline mode here: https://help.upserve.com/s/article/Offline-Mode-Overview and https://help.upserve.com/s/article/Internet-Connectivity-Issues-Android-Tableside
- identified Oct 02, 2022, 11:40 PM UTC
Our engineering team has detected possible payment delays due to our 3rd party processing partner. Your Upserve POS may be in offline mode. Offline Mode allows you to continue operating and collecting credit card payments. More on offline mode here: http://bit.ly/bcomode.
- identified Oct 03, 2022, 12:45 AM UTC
Our engineering team is continuing to investigate higher than normal decline rates on credit card transactions due to a 3rd Party Payment Processing issue. Your Upserve Point of Sale may be in Offline Mode for this reason. Offline Mode allows you to continue operating and collecting credit card payments. More on offline mode here: http://bit.ly/bcomode.
- identified Oct 03, 2022, 12:46 AM UTC
We are continuing to work on a fix for this issue.
- identified Oct 03, 2022, 12:47 AM UTC
Our engineering team is continuing to investigate higher than normal decline rates on credit card transactions due to a 3rd Party Payment Processing issue. Your Upserve Point of Sale may be in Offline Mode for this reason. Offline Mode allows you to continue operating and collecting credit card payments. More on offline mode here: http://bit.ly/bcomode.
- identified Oct 03, 2022, 11:53 AM UTC
Our engineering team is continuing to investigate higher than normal decline rates on credit card transactions due to a 3rd Party Payment Processing issue. Your Upserve Point of Sale may still be in Offline Mode for this reason. We are continuing to work with our 3rd Party Payment Partner to resolve this outage. Offline Mode allows you to continue operating and collecting credit card payments. More on offline mode here: https://help.upserve.com/s/article/Internet-Connectivity-Issues-Android-Tableside
- identified Oct 03, 2022, 02:43 PM UTC
Our engineering team is continuing to investigate elevated error rates on card transactions due to an outage with Fiserv/First Data, our 3rd Party Payment Processor. Your Upserve Point of Sale may still be in Offline Mode for this reason. We are continuing to work with Fiserv/First Data to resolve this outage. We will update this page as soon as an update becomes available. Offline Mode allows you to continue operating and collecting credit card payments. More on offline mode here: https://help.upserve.com/s/article/Internet-Connectivity-Issues-Android-Tableside
- identified Oct 03, 2022, 03:55 PM UTC
Our engineering team is continuing to investigate elevated error rates on card transactions due to an outage with Fiserv/First Data, our 3rd Party Payment Processor. Your Upserve Point of Sale may still be in Offline Mode for this reason. We are continuing to work with Fiserv/First Data to resolve this outage. We will update this page as soon as an update becomes available. Offline Mode allows you to continue operating and collecting credit card payments. More on offline mode here: https://help.upserve.com/s/article/Internet-Connectivity-Issues-Android-Tableside
- monitoring Oct 03, 2022, 05:28 PM UTC
Since our last update, our 3rd Party Payment Processor Partner Fiserv/First Data has taken action to resolve their service incident affecting declined card transactions for Upserve customers. Upserve has observed a lower rate of declined transactions and customers previously operating in offline mode are now returning to normal operation. Upserve will continue to monitor this incident further prior to declaring the incident as resolved.
- resolved Oct 03, 2022, 06:10 PM UTC
Since our last update, our 3rd Party Payment Processing Partner Fiserv/First Data has confirmed that they have completed the work to resolve the outage affecting card authorizations. Our monitoring has shown consistent processing activity for all Upserve customers, therefore we are resolving this issue. Allow up to (1) day for any open checks with payments to sync. Please note batches from 10/2 will be delayed up to (1) day for processing. Today's transactions will batch with 10/3 as expected. If you have any questions, please contact our support team via chat in-app or at help.upserve.com.