FireNet incident

SIP - Outbound calls failing

Major Resolved View vendor source →

FireNet experienced a major incident on July 29, 2019 affecting Voice trunking and Hosted voice, lasting 2h 7m. The incident has been resolved; the full update timeline is below.

Started
Jul 29, 2019, 03:26 AM UTC
Resolved
Jul 29, 2019, 05:34 AM UTC
Duration
2h 7m
Detected by Pingoru
Jul 29, 2019, 03:26 AM UTC

Affected components

Voice trunkingHosted voice

Update timeline

  1. identified Jul 29, 2019, 03:26 AM UTC

    An outbound calling outage is currently affecting some customers, the upstream carrier is aware of the issue and is now investigating. Updates as soon as we get a response.

  2. monitoring Jul 29, 2019, 04:26 AM UTC

    A fix was applied roughly 1:33pm AEST however we are just waiting for confirmation this issue is resolved from the upstream.

  3. resolved Jul 29, 2019, 05:34 AM UTC

    Upstream carrier confirmed the issue resolved itself and services are now operational. The fault was due to a lockup on a soft switch after a high level of traffic caused heavy resource usage, this caused timeouts on some calls from 1:20pm AEST until about 1:33pm AEST.