Fasterize incident

Logs cluster failure

Notice Resolved View vendor source →

Fasterize experienced a notice incident on July 27, 2021, lasting —. The incident has been resolved; the full update timeline is below.

Started
Jul 27, 2021, 12:14 PM UTC
Resolved
Jul 27, 2021, 09:00 AM UTC
Duration
Detected by Pingoru
Jul 27, 2021, 12:14 PM UTC

Update timeline

  1. resolved Jul 27, 2021, 12:14 PM UTC

    Due to logs cluster issues, we have lost some logs from friday 23 to tuesday 27 th of july 2021. Our logs cluster had some performance issues since sunday 16:30 UTC. It seems that some logs indices experimented sharding issue that have corrupted 23 to 24 th of july 2021 indexes. Indexes from sunday 25 to tuesday 27 th 9h00 UTC of july 2021 may be incomplete. We deleted corrupted indexes. This allowed us to delete unhealthy nodes and re-balance the cluster. Logs are now fully operational. We are sorry for logs lost during this incident.