Technolutions incident

Increased latency for databases on LUNA cluster

Notice Resolved View vendor source →

Technolutions experienced a notice incident on September 27, 2022 affecting Slate, lasting 10h 8m. The incident has been resolved; the full update timeline is below.

Started
Sep 27, 2022, 04:51 PM UTC
Resolved
Sep 28, 2022, 03:00 AM UTC
Duration
10h 8m
Detected by Pingoru
Sep 27, 2022, 04:51 PM UTC

Affected components

Slate

Update timeline

  1. investigating Sep 27, 2022, 04:51 PM UTC

    We are investigating increased latency for databases on the LUNA cluster, which appears to be the result of increased disk latency in AWS for this particular node.

  2. monitoring Sep 27, 2022, 08:10 PM UTC

    Earlier today, elevated latency was observed for databases on the LUNA cluster. This appears to have been the result of a temporary hardware/network issue within the AWS infrastructure that resulted in increased latency for transaction log writes to the storage infrastructure. Performance levels have been within normal ranges for the past hour, and we will continue to monitor the storage volumes. Should increased latency be observed again, we will transition the cluster to new hardware with AWS.

  3. resolved Sep 28, 2022, 03:40 AM UTC

    No further latency has been observed since the original event. Should increased latency be observed again, we will reopen this incident and take the additional measures described to move to new hardware.