SearchStax incident

False Heartbeat Alerts Triggered

Resolved after 4 hours 12 minutes, affecting Pulse Monitoring, on May 11, 2023.

resolved May 11, 2023, 06:53 PM UTC

Due to the recent fixes that were done for the April 24th incident with Pulse, our services had an issue updating some of the servers today that caused False Heartbeat alerts to go out. Our team has resolved the issue and put measures in place to prevent this in the future.

Started
May 11, 2023, 02:40 PM UTC
Resolved
May 11, 2023, 06:53 PM UTC
Duration
4h 12m
Detected by Pingoru
May 11, 2023, 02:40 PM UTC

Affected components

Pulse Monitoring

Update timeline

  1. investigating May 11, 2023, 02:40 PM UTC

    We have discovered false Heartbeat alerts for several deployments starting May 11, 2023,2:20pm UTC. Our team is investigating the issue. Our team has separate monitoring that is set up for all deployments for which they get alerted on if a node or deployment is unavailable. At this time, we checked all alerts triggered via Pulse and have confirmed that the deployments for which these alerts were triggered are all in a healthy state.

  2. resolved May 11, 2023, 06:53 PM UTC

    Due to the recent fixes that were done for the April 24th incident with Pulse, our services had an issue updating some of the servers today that caused False Heartbeat alerts to go out. Our team has resolved the issue and put measures in place to prevent this in the future.

Your product runs on more than SearchStax

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →