SearchStax incident

False Pulse Heartbeat Alerts

Notice Resolved View vendor source →

SearchStax experienced a notice incident on July 18, 2020 affecting Admin App, lasting 4m. The incident has been resolved; the full update timeline is below.

Started
Jul 18, 2020, 04:14 PM UTC
Resolved
Jul 18, 2020, 04:19 PM UTC
Duration
4m
Detected by Pingoru
Jul 18, 2020, 04:14 PM UTC

Affected components

Admin App

Update timeline

  1. investigating Jul 18, 2020, 04:14 PM UTC

    Between 07:50 and approximately 08:45 UTC on 18 Jul 2020, several Azure SearchStax deployments received False Heartbeat alerts

  2. resolved Jul 18, 2020, 04:19 PM UTC

    Between 07:50 UTC and approximately 08:45 UTC on 18 Jul 2020, there was a DNS outage on Azure - https://status.azure.com/en-us/status/history/. Due to this outage, Pulse agents on Azure deployments were not able to connect to the Pulse servers on searchstax.com, triggering false alerts. Our second monitoring system that connects to the deployment through the Gateway and checks Solr/Zookeeper status was still working, and all deployment have reported being healthy.