Gainsight experienced a minor incident on August 26, 2024 affecting Gainsight CS US1 Application, lasting 20h 24m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 26, 2024, 05:06 PM UTC
We are investigating reports of latency with loading pages in CS-US1. We will post further updates as we have more details.
- identified Aug 26, 2024, 05:19 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Aug 26, 2024, 06:28 PM UTC
A fix has been implemented and we are monitoring the results.
- monitoring Aug 26, 2024, 07:33 PM UTC
We are keeping the incident open at this time to monitor for further issues.
- monitoring Aug 27, 2024, 04:36 AM UTC
We are continuing to monitor for any further issues.
- resolved Aug 27, 2024, 01:31 PM UTC
The issue is resolved. Systems are back to their normal performance level. We will share the RCA as soon as it is available.
- postmortem Sep 04, 2024, 04:54 AM UTC
**Incident:** 26th August, 2024: Some customers may have experienced elevated levels of latency while using our CS-US1 application. **Root Cause:** * There was an increased frequency of database calls tied to a recent hotfix. The increased traffic was intermittent but led to slower API response times and thus impacted customer experience from browser. * Customers in CS-US1 may have experienced the latency while browsing. Impact was relatively less for our CS-US2 and CS-EU regions. * Sanity testing in lower environments did not initially produce this issue which was only observed with production traffic in US1. **Recovery Action:** Once the source was identified, a subsequent fix was applied which restored performance to expected levels. **Preventive Measures:** The mentioned fix has resolved this issue. We will continue to review and learn from issues like this in effort to provide the best possible customer experience. Please contact [[email protected]](mailto:[email protected]) if there are any questions.