SearchStax incident

False Heartbeat alerts from SearchStax Solr Service

Minor Resolved View vendor source →

SearchStax experienced a minor incident on May 30, 2020 affecting Admin App, lasting 6h 53m. The incident has been resolved; the full update timeline is below.

Started
May 30, 2020, 03:07 PM UTC
Resolved
May 30, 2020, 10:01 PM UTC
Duration
6h 53m
Detected by Pingoru
May 30, 2020, 03:07 PM UTC

Affected components

Admin App

Update timeline

  1. monitoring May 30, 2020, 03:07 PM UTC

    Due to an expired Sectigo Root certificate, named the AddTrust External CA Root, SearchStax Pulse (Monitoring) agents were not able to communicate to our Pulse APIs. The fix for the issue was put in place at 7:57 am PT and the team is monitoring to ensure that this issue has been fixed.

  2. identified May 30, 2020, 03:22 PM UTC

    We are updating the incident and changing the status to "Identified" as the fix that was deployed did not work for all deployments. Certain deployments are still facing problems where the Dashboard is showing "Agent Down" on the Servers list. At this time, this issue seems to be affecting older class deployment (starting with SS or SB). The team is working to find a fix to a the problem

  3. identified May 30, 2020, 05:02 PM UTC

    We are continuing to work on this issue and identifying deployments for which the "Agent" shows down. Once we have identified these, the team is applying a fix to resolve the issue for those deployments.

  4. resolved May 30, 2020, 10:01 PM UTC

    The team has successfully implemented a fix to most of the known deployments. Clients with deployments that still have the problem have bee notified and we will be working with them to resolve the issue. If you continue to see any issues, please open up a support ticket.