Quarterhorse Technology incident

QTICloud Outage

Major Resolved View vendor source →

Quarterhorse Technology experienced a major incident on July 20, 2023 affecting Private Switch 1 and Private Switch 2, lasting 8d 8h. The incident has been resolved; the full update timeline is below.

Started
Jul 20, 2023, 03:31 PM UTC
Resolved
Jul 29, 2023, 12:04 AM UTC
Duration
8d 8h
Detected by Pingoru
Jul 20, 2023, 03:31 PM UTC

Affected components

Private Switch 1Private Switch 2

Update timeline

  1. investigating Jul 20, 2023, 03:31 PM UTC

    We are currently investigating an issue with access to QTICloud services

  2. investigating Jul 20, 2023, 03:51 PM UTC

    we are continuing to investigate the issue. however all services have resumed normal operation

  3. identified Jul 20, 2023, 04:06 PM UTC

    We have identified this incident as a firewall event with Primary and Secondary peer status.

  4. monitoring Jul 20, 2023, 04:07 PM UTC

    We are continuing to monitor the situation and the affected components. A support case was opened with the vendor.

  5. monitoring Jul 24, 2023, 06:57 PM UTC

    Maintenance is scheduled for 7/28 @ 2pm with vendor support to further investigate.

  6. monitoring Jul 28, 2023, 09:32 PM UTC

    Working with vendor support initiated and unexpected reboot of hardware. We are monitoring the restoration of services.

  7. monitoring Jul 28, 2023, 09:58 PM UTC

    reboots are complete and service appears to be restored

  8. resolved Jul 29, 2023, 12:04 AM UTC

    issue has been resolved and faulty hardware has been removed from active service