SAP Concur incident

Expense, Travel, Invoice, Request: P1

Resolved after 4 hours 1 minute, affecting Expense and Invoice and 1 more, on May 14, 2025.

resolved May 26, 2025, 04:00 PM UTC

Impact as Reported: In the EU2 Data Center, logins and use of SAP Concur services is below expected level through the SAP Concur website. Affected users may intermittently be unable to login or experience error “Sorry, something went wrong” after landing the homepage. Alternatively, the users might get error “Sorry, something went wrong with this task” or “There has been an error processing your request” when trying to create an expense report or request, or miss a print button in Expense module Root cause: The root cause of the incident has been determined to be an issue with the infrastructure controlling internal connectivity across micro-services responsible for the authentication workflow. Due to a set of nodes unexpectedly found in a non-responsive state, a percentage of traffic was not being correctly routed resulting in intermittent errors or time outs. Once the issue was identified the nodes were refreshed to restore service.What caused the impacted nodes to be in an unresponsive state is still being investigated, additional logging and instrumentation will be configured to enable deeper data collection in the event the same issue would re-occurs in the future. Corrective Actions: Various initiatives have been identified to enhance monitoring and alerting in order to improve detection for this type of scenarios and decrease reaction and recovery times.

Started
May 14, 2025, 07:00 AM UTC
Resolved
May 14, 2025, 11:01 AM UTC
Duration
4h 1m
Detected by Pingoru
May 14, 2025, 07:00 AM UTC

Affected components

ExpenseInvoiceRequestTravel

Update timeline

  1. resolved May 14, 2025, 08:28 AM UTC

    We are working to determine the user impact and will provide information as it becomes available.

  2. resolved May 14, 2025, 09:03 AM UTC

    In the EU2 Data Center, logins and use of SAP Concur services through SAP Concur website is below expected level. Affected users may intermittently be unable to login or experience error “Sorry, something went wrong” after being logged in. Alternatively, the users might get error “Sorry, something went wrong with this task” when trying to create an expense report. The Incident Response Team (IRT) has been engaged and are investigating. Our next scheduled update will be in one hour or sooner, should a significant change occur.

  3. resolved May 14, 2025, 10:09 AM UTC

    In the EU2 Data Center, logins and use of SAP Concur services is below expected level through the SAP Concur website. Affected users may intermittently be unable to login or experience error “Sorry, something went wrong” after being logged in. Alternatively, the users might get error “Sorry, something went wrong with this task” or “There has been an error processing your request” when trying to create an expense report or request, or miss a print button in Expense module. Since our last update the IRT has restarted the application service tier, which failed to restore the service. The IRT is in the process of restarting the user login functionality server tier. Our next scheduled update will be in one hour or sooner, should a significant change occur.

  4. resolved May 14, 2025, 11:05 AM UTC

    In the EU2 Data Center, logins and use of SAP Concur services is below expected level through the SAP Concur website. Affected users may intermittently be unable to login or experience error “Sorry, something went wrong” after landing the homepage. Alternatively, the users might get error “Sorry, something went wrong with this task” or “There has been an error processing your request” when trying to create an expense report or request, or miss a print button in Expense module. Workaround: The affected users may try to refresh the homepage several times after they receive the error in order to access the services. Since our last update the IRT continues to restart the user login functionality server tier while checking service performance.

  5. resolved May 14, 2025, 11:18 AM UTC

    In the EU2 Data Center, logins and use of SAP Concur services was below expected level through the SAP Concur website. Affected users may have intermittently been unable to login or would experience error “Sorry, something went wrong” after landing the homepage. Alternatively, the users might got error “Sorry, something went wrong with this task” or “There has been an error processing your request” when trying to create an expense report or request, or miss a print button in Expense module. While service performance is now operating at normal levels, we will continue monitoring performance for the next 15 minutes to verify stability. Once verified as stable, the incident will be resolved.

  6. resolved May 14, 2025, 11:31 AM UTC

    In the EU2 Data Center, logins and use of SAP Concur services were below expected level through the SAP Concur website. Affected users may have intermittently been unable to login or would experience error “Sorry, something went wrong” after landing the homepage. Alternatively, the users might got error “Sorry, something went wrong with this task” or “There has been an error processing your request” when trying to create an expense report or request, or miss a print button in Expense module. The Incident Response Team restarted the user login functionality server tier which restored the service for all the affected users. We have verified that service performance is stable and will now resolve this incident. An investigation into the root cause of this incident will now be conducted, and a root cause analysis report provided when that investigation is complete.

  7. resolved May 26, 2025, 04:00 PM UTC

    Impact as Reported: In the EU2 Data Center, logins and use of SAP Concur services is below expected level through the SAP Concur website. Affected users may intermittently be unable to login or experience error “Sorry, something went wrong” after landing the homepage. Alternatively, the users might get error “Sorry, something went wrong with this task” or “There has been an error processing your request” when trying to create an expense report or request, or miss a print button in Expense module Root cause: The root cause of the incident has been determined to be an issue with the infrastructure controlling internal connectivity across micro-services responsible for the authentication workflow. Due to a set of nodes unexpectedly found in a non-responsive state, a percentage of traffic was not being correctly routed resulting in intermittent errors or time outs. Once the issue was identified the nodes were refreshed to restore service.What caused the impacted nodes to be in an unresponsive state is still being investigated, additional logging and instrumentation will be configured to enable deeper data collection in the event the same issue would re-occurs in the future. Corrective Actions: Various initiatives have been identified to enhance monitoring and alerting in order to improve detection for this type of scenarios and decrease reaction and recovery times.

Your product runs on more than SAP Concur

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →