SAP Concur incident

Japan - Japan Public Transportation (JPT): P1

Resolved after 5 hours 13 minutes, affecting Japan - Japan Public Transportation (JPT), on May 30, 2025.

resolved Feb 04, 2026, 07:51 AM UTC

Impact as Reported: In the US2 Data Center, creating JPT Route Search entries through the SAP Concur website and SAP Concur mobile app were below expected level. Affected users may have encountered the error message, “Error. Contact system administrator.,” when attempting to create a JPT Route Search entry or when viewing "Route Details" on an existing JPT expense. Root cause: Due to a sub-optimal stored procedure, three separate API actions for different users reports caused a continuous deadlock on a database table, this caused a cascade of events that led to resource exhaustion on the database server, resulting in performance degradation for all the companies hosted on that server. Corrective Actions: The offending stored procedure has been analyzed and changes have been implemented to prevent deadlocks.

Started
May 30, 2025, 12:05 AM UTC
Resolved
May 30, 2025, 05:18 AM UTC
Duration
5h 13m
Detected by Pingoru
May 30, 2025, 12:05 AM UTC

Affected components

Japan - Japan Public Transportation (JPT)

Update timeline

  1. resolved May 30, 2025, 03:28 AM UTC

    We are working to determine the user impact and will provide information as it becomes available.

  2. resolved May 30, 2025, 04:01 AM UTC

    In the US2 Data Center, creating JPT Route Search entries through the SAP Concur website and SAP Concur mobile app is below expected level. Affected users may encounter an error when attempting to create a JPT Route Search entry or when viewing "Route Details" on an existing JPT expense. The Incident Response Team (IRT) is currently investigating. Our next scheduled update will be in one hour or sooner should a significant change occur.

  3. resolved May 30, 2025, 04:53 AM UTC

    In the US2 Data Center, creating JPT Route Search entries through the SAP Concur website and SAP Concur mobile app is below expected level. Affected users may encounter the error message, “Error. Contact system administrator.,” when attempting to create a JPT Route Search entry or when viewing "Route Details" on an existing JPT expense. The Incident Response Team (IRT) has engaged additional resources to assist with the investigation and determine the root cause. Our next scheduled update will be in one hour or sooner should a significant change occur.

  4. resolved May 30, 2025, 05:50 AM UTC

    In the US2 Data Center, creating JPT Route Search entries through the SAP Concur website and SAP Concur mobile app is below expected level. Affected users may encounter the error message, “Error. Contact system administrator.,” when attempting to create a JPT Route Search entry or when viewing "Route Details" on an existing JPT expense. The Incident Response Team (IRT) has restored the application service cluster and terminated certain processes on the server to improve performance; however, these actions did not yield expected results. The team is currently analyzing system logs and configurations to identify the root cause. Our next scheduled update will be in one hour or sooner should a significant change occur.

  5. resolved May 30, 2025, 06:40 AM UTC

    In the US2 Data Center, creating JPT Route Search entries through the SAP Concur website and SAP Concur mobile app were below expected level. Affected users may have encountered the error message, “Error. Contact system administrator.,” when attempting to create a JPT Route Search entry or when viewing "Route Details" on an existing JPT expense. Services have returned to normal operating levels. We will continue monitoring performance for the next 15-30 minutes to verify stability. Once verified as stable the incident will be resolved.

  6. resolved May 30, 2025, 06:52 AM UTC

    In the US2 Data Center, creating JPT Route Search entries through the SAP Concur website and SAP Concur mobile app were below expected level. Affected users may have encountered the error message, “Error. Contact system administrator.,” when attempting to create a JPT Route Search entry or when viewing "Route Details" on an existing JPT expense. The Incident Response Team (IRT) took action to stop blocking on affected database server to restore service. We have verified that service performance is stable and will now resolve this incident. An investigation into the root cause of this incident will now be conducted, and a root cause analysis report provided when that investigation is complete.

  7. resolved Jun 13, 2025, 05:31 PM UTC

    In the US2 Data Center, creating JPT Route Search entries through the SAP Concur website and SAP Concur mobile app were below expected level. Affected users may have encountered the error message, “Error. Contact system administrator.,” when attempting to create a JPT Route Search entry or when viewing "Route Details" on an existing JPT expense. Due to a sub-optimal stored procedure, three separate API actions for different users reports caused a continuous deadlock on a database table, this caused a cascade of events that led to resource exhaustion on the database server, resulting in performance degradation for all the companies hosted on that server. Corrective Actions: Corrective and preventive actions are still being discussed.

  8. resolved Feb 04, 2026, 07:51 AM UTC

    Impact as Reported: In the US2 Data Center, creating JPT Route Search entries through the SAP Concur website and SAP Concur mobile app were below expected level. Affected users may have encountered the error message, “Error. Contact system administrator.,” when attempting to create a JPT Route Search entry or when viewing "Route Details" on an existing JPT expense. Root cause: Due to a sub-optimal stored procedure, three separate API actions for different users reports caused a continuous deadlock on a database table, this caused a cascade of events that led to resource exhaustion on the database server, resulting in performance degradation for all the companies hosted on that server. Corrective Actions: The offending stored procedure has been analyzed and changes have been implemented to prevent deadlocks.

Your product runs on more than SAP Concur

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →