Expense, Travel, Invoice, Request, Imaging, Analysis/Intelligence: P1

Update timeline

1. resolved Apr 15, 2025, 09:23 PM UTC

   In the US2 Data Center, logins through the SAP Concur website and SAP Concur mobile app were below expected level. Affected users may have experienced intermittent errors when attempting to log in. As the Incident Response Team (IRT) was gathering, the issue self-recovered. We have verified that service performance is stable and will now resolve this incident. An investigation into the root cause of this incident will now be conducted, and a root cause analysis report provided when that investigation is complete.

2. resolved May 06, 2025, 08:03 AM UTC

   Impact as Reported: In the US2 Data Center, logins through the SAP Concur website and SAP Concur mobile app were below expected level. Affected users may have experienced intermittent errors when attempting to log in. Root cause: As part of a standard maintenance operation users were being moved from a container cluster providing gateway services to a new cluster. The service team performing the operation failed to realize that the new cluster was not scaling up capacity quickly enough to support the incoming load, resulting in latency and timeouts. The incident response team moved all users back to the original cluster, restoring service to normal levels. The migration operation was then completed after manually scaling the capacity of the new cluster to match the old one. Corrective Actions: Add a step to the deployment process to manually scale the secondary cluster to the same as production instead of relying on auto-scaling.