Gainsight PX incident

PX Engagements and Knowledge Center (KC) Bots

Resolved after 1 hour 36 minutes, affecting In-app Guides and In-app Surveys, on July 17, 2023.

postmortem Jul 26, 2023, 05:24 PM UTC

**Summary of Incident:** Some customers were unable to use Engagements and KC Bots between 9:02 and 10:38 PDT on July 17, then again between 7:18 and 14:38 PDT on July 18, 2023. ‌ **Root Cause:** A high availability caching cluster re-election had an issue while handling real time traffic due to a bug, which caused an increase in latency. ‌ **Action Items:** Our Engineering Team identified and resolved the issue caused from the initial bug. Please reach out to [[email protected]](mailto:[email protected]) if you have questions.

Started
Jul 17, 2023, 04:02 PM UTC
Resolved
Jul 17, 2023, 05:38 PM UTC
Duration
1h 36m
Detected by Pingoru
Jul 17, 2023, 04:02 PM UTC

Affected components

In-app GuidesIn-app Surveys

Update timeline

  1. identified Jul 17, 2023, 04:02 PM UTC

    We are working to resolve an identified issue causing slowness in PX engagement and Knowledge Center (KC) Bots. Please email [email protected] with any questions.

  2. resolved Jul 17, 2023, 05:38 PM UTC

    An infrastructure issue was identified which caused lag in PX Engagements and Knowledge Center (KC) Bots for a subset of customers. This is resolved and affected customers should be seeing improvements now. Please email [email protected] with any questions.

  3. postmortem Jul 26, 2023, 05:24 PM UTC

    **Summary of Incident:** Some customers were unable to use Engagements and KC Bots between 9:02 and 10:38 PDT on July 17, then again between 7:18 and 14:38 PDT on July 18, 2023. ‌ **Root Cause:** A high availability caching cluster re-election had an issue while handling real time traffic due to a bug, which caused an increase in latency. ‌ **Action Items:** Our Engineering Team identified and resolved the issue caused from the initial bug. Please reach out to [[email protected]](mailto:[email protected]) if you have questions.

Your product runs on more than Gainsight PX

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →