Gainsight incident
CS-US1 - Investigating - Elevated error rates
Gainsight experienced a minor incident on July 18, 2023 affecting Gainsight CS US1 Application, lasting 51m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jul 18, 2023, 06:36 PM UTC
We are investigating elevated error rates which may result in slowness or availability issues for some customers. More details to follow.
- investigating Jul 18, 2023, 06:41 PM UTC
We are continuing to investigate this issue.
- monitoring Jul 18, 2023, 07:00 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jul 18, 2023, 07:28 PM UTC
Some customers may have experienced slowness or errors while trying to load CS-US1 services between 18:25 and 18:38 UTC. Engineers responded quickly to restore impacted services. We will post root cause details as they become available.
- postmortem Aug 11, 2023, 04:22 AM UTC
**Incident:** Some customers may have experienced degraded performance while trying to load CS-US1 services between 18:25 and 18:38 UTC on the 18th of July. **Root Cause:** API Gateway instability due to network contention at the data layer was found to be the root cause. **Recovery Action:** Engineers performed a rolling restart of API services once this issue was detected. **Preventive Measures:** System configuration adjustments have been made to prevent these issues moving forward.