SAP Concur incident

Travel: P1

Resolved after 20 minutes, affecting Travel, on September 2, 2025.

resolved Sep 15, 2025, 07:01 AM UTC

Impact as Reported: In the US2 Data Center, travel searches in New Concur Travel were 40 % below expected level. Affected users may have experienced an error, “No location found” when attempting to complete travel search criteria. A workaround was not identified Root cause: The deployment of a new version of a travel micro service triggered an autoscaling operation on the cluster hosting it. Another service hosted on the same cluster entered a degraded state when new serving instances were being provisioned due to the auto scaling, as new instances were put into service before having completed their boot strap due to a defect on a life check end point and given the load levels at the time of the operation started receiving traffic before being ready. The resulting errors when calling the service, combined with an aggressive retry logic on the client side, caused the calls volume to spike, further exacerbating the service degradation. The incident response team manually stopped the service and manually scaled it to enable service restoration. Corrective Actions: The following corrective and preventative actions were identified: Stopped all requests and manually increased new instances to remediate the issue Update the health checks to fix the defect and ensure seving instances only start receiving traffic once fully ready Implement a circuit breaker to prevent retry logic to overwhelm the service in case of degradation.

Started
Sep 02, 2025, 05:05 PM UTC
Resolved
Sep 02, 2025, 05:25 PM UTC
Duration
20m
Detected by Pingoru
Sep 02, 2025, 05:05 PM UTC

Affected components

Travel

Update timeline

  1. resolved Sep 02, 2025, 06:26 PM UTC

    In the US2 Data Center, travel searches in New Concur Travel were 40 % below expected level. Affected users may have experienced an error, “No location found” when attempting to complete travel search criteria. A workaround was not identified. The Incident Response Team identified a capacity issue and immediately took action to restart a business service tier to restore service. An investigation into the root cause of this incident will now be conducted, and a root cause analysis report provided when that investigation is complete.

  2. resolved Sep 15, 2025, 07:01 AM UTC

    Impact as Reported: In the US2 Data Center, travel searches in New Concur Travel were 40 % below expected level. Affected users may have experienced an error, “No location found” when attempting to complete travel search criteria. A workaround was not identified Root cause: The deployment of a new version of a travel micro service triggered an autoscaling operation on the cluster hosting it. Another service hosted on the same cluster entered a degraded state when new serving instances were being provisioned due to the auto scaling, as new instances were put into service before having completed their boot strap due to a defect on a life check end point and given the load levels at the time of the operation started receiving traffic before being ready. The resulting errors when calling the service, combined with an aggressive retry logic on the client side, caused the calls volume to spike, further exacerbating the service degradation. The incident response team manually stopped the service and manually scaled it to enable service restoration. Corrective Actions: The following corrective and preventative actions were identified: Stopped all requests and manually increased new instances to remediate the issue Update the health checks to fix the defect and ensure seving instances only start receiving traffic once fully ready Implement a circuit breaker to prevent retry logic to overwhelm the service in case of degradation.

Your product runs on more than SAP Concur

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →