Heron Data incident

Major outage: Load balancer 502s

Notice Resolved View vendor source →

Heron Data experienced a notice incident on July 8, 2022, lasting —. The incident has been resolved; the full update timeline is below.

Started
Jul 08, 2022, 10:03 AM UTC
Resolved
Jul 08, 2022, 12:00 AM UTC
Duration
Detected by Pingoru
Jul 08, 2022, 10:03 AM UTC

Update timeline

  1. resolved Jul 08, 2022, 10:03 AM UTC

    From 00:10-00:19 UTC (9 minutes), we had a major outage in our API when we experienced a high amount of 502 errors that rendered our backend API unusable. This was caused by issues in our load balancer, which was caused by usually high amounts of traffic and unavailable pods to service the traffic. We are in the process of adding better scalability to our backend to handle such spikes in traffic moving forward, including better horizontal scaling metrics.