Gainsight incident

CS-US1 - Degraded performance in Timeline

Minor Resolved View vendor source →

Gainsight experienced a minor incident on July 28, 2023 affecting Gainsight CS US1 Application, lasting 36m. The incident has been resolved; the full update timeline is below.

Started
Jul 28, 2023, 03:49 PM UTC
Resolved
Jul 28, 2023, 04:26 PM UTC
Duration
36m
Detected by Pingoru
Jul 28, 2023, 03:49 PM UTC

Affected components

Gainsight CS US1 Application

Update timeline

  1. investigating Jul 28, 2023, 03:49 PM UTC

    We are investigating performance issues related to Timeline and will update as we have more information.

  2. monitoring Jul 28, 2023, 03:53 PM UTC

    A fix has been implemented and we are monitoring the results.

  3. resolved Jul 28, 2023, 04:26 PM UTC

    Some customers may have experienced slowness or errors while trying to load Timeline in CS-US1. Engineers responded quickly to restore services. We will post root cause details as they become available.

  4. postmortem Aug 11, 2023, 04:23 AM UTC

    **Incident:** Some customers may have experienced degraded performance in CS-US1 Timeline services between 15:44 and 15:48 UTC on the 28th of July. **Root Cause:** A memory issue on multiple web servers led to Timeline becoming unresponsive intermittently, while other servers in the pool continued to serve traffic. **Recovery Action:** Monitoring systems alerted engineers of a memory issue on Timeline web servers. Engineers quickly made adjustments and redeployed Timeline web servers as a temporary fix. ‌ **Preventive Measures:** A permanent hotfix will be deployed in our next release window.