CallFire incident

CallFire Legacy Delays

Minor Resolved View vendor source →

CallFire experienced a minor incident on September 18, 2014, lasting 6d 6h. The incident has been resolved; the full update timeline is below.

Started
Sep 18, 2014, 03:42 PM UTC
Resolved
Sep 24, 2014, 09:52 PM UTC
Duration
6d 6h
Detected by Pingoru
Sep 18, 2014, 03:42 PM UTC

Update timeline

  1. monitoring Sep 18, 2014, 03:42 PM UTC

    Our CallFire Legacy platform (also known as CF2) is currently experiencing long agent connect times in the Cloud Call Center, and long intervals between Voice Broadcast dialing.

  2. monitoring Sep 19, 2014, 03:52 PM UTC

    A code fix was deployed in the last 12 hours. As of writing of this update, the issue hasn't cropped up. The system is being actively monitored.

  3. monitoring Sep 19, 2014, 07:20 PM UTC

    So far CCC on CF2 seems to be stable. The Reporting queue is not back logged. Additionally, Ops is actively moving heavy voice broadcast dialers to Tesla. This will help with the load issues. Currently, the system is stable, however we cannot call an all-clear yet.

  4. monitoring Sep 20, 2014, 04:19 PM UTC

    Additional code changes & configuration changes were deployed on Friday (9/19) night. We are continually monitoring the situation.

  5. monitoring Sep 22, 2014, 01:40 PM UTC

    Additional changes to the legacy system were made over the weekend and there were no reports as of the time this update of slowness on the system. Today will be a day of active monitoring for any slow dialing issues be it Cloud Call Center or Voice Broadcast.

  6. resolved Sep 24, 2014, 09:52 PM UTC

    This incident has been resolved.