NZIX incident

[EMERGENCY OUTAGE] CHC-IX | CCL | pe1.chc2 Switch Reboot

Major Resolved View vendor source →

NZIX experienced a major incident on April 11, 2023 affecting CHC-IX, lasting 8m. The incident has been resolved; the full update timeline is below.

Started
Apr 11, 2023, 03:10 PM UTC
Resolved
Apr 11, 2023, 03:18 PM UTC
Duration
8m
Detected by Pingoru
Apr 11, 2023, 03:10 PM UTC

Affected components

CHC-IX

Update timeline

  1. identified Apr 11, 2023, 03:10 PM UTC

    Our switch at Computer Concepts Limited (CCL) in CHC-IX has suffered the same management plane lockup as experienced recently by our switch at Christchurch Central Exchange (CCE). An emergency reboot is underway to resolve this now. This bug is also impacting statistics on https://metrics.ix.nz/ for any service on this switch. Unfortunately this issue was expected but was yet to be booked in for the fix, apologies for the short notice! Expected Downtime: ~5-10 minutes Summary of impact during this window: * ALL Members at CHC-IX should expect connectivity to rs2.chc.ix.nz to go down. * Members connected to pe1.chc2 should expect their port to go offline. BCP214 compliant filters will be applied toward their RS1 sessions to reduce the impact.

  2. resolved Apr 11, 2023, 03:18 PM UTC

    This incident has been resolved.