Freshcaller incident

Degraded Performance in Programmable Voice

Minor Resolved View vendor source →

Freshcaller experienced a minor incident on November 25, 2020 affecting Inbound Calls and Outbound Calls and 1 more component, lasting 3h 36m. The incident has been resolved; the full update timeline is below.

Started
Nov 25, 2020, 06:29 PM UTC
Resolved
Nov 25, 2020, 10:05 PM UTC
Duration
3h 36m
Detected by Pingoru
Nov 25, 2020, 06:29 PM UTC

Affected components

Inbound CallsOutbound CallsInbound CallsOutbound CallsInbound CallsOutbound CallsInbound CallsOutbound Calls

Update timeline

  1. investigating Nov 25, 2020, 06:29 PM UTC

    We have detected a potential issue with Programmable Voice. Our Service provider is working to resolve the issue. We expect to provide another update in 1 hour or as soon as more information becomes available.

  2. investigating Nov 25, 2020, 06:29 PM UTC

    Our monitoring systems have detected a potential issue with Elastic SIP Trunking. Our engineering team has been alerted and is actively investigating. We will update as soon as we have more information.

  3. investigating Nov 25, 2020, 07:00 PM UTC

    We are seeing failures when publishing call logs for Trunking calls. Customers may temporarily be unable to view logs for completed Trunking calls. Engineers are looking into this. We expect to provide another update in 30 minutes or as soon as more information becomes available.

  4. investigating Nov 25, 2020, 07:30 PM UTC

    We are seeing failures in publishing call logs for all calls (Programmable Voice and Trunking). Customers are unable to view logs and voice insights data for completed calls and will receive 5xx responses when attempting to retrieve call records from the API. Engineers are looking into this. We expect to provide another update in 30 minutes or as soon as more information becomes available.

  5. investigating Nov 25, 2020, 08:01 PM UTC

    We have started seeing recovery in publishing call logs for all calls (Programmable Voice and Trunking). We will continue to monitor to ensure a full recovery. We expect to provide another update in 30 minutes or as soon as more information becomes available.

  6. investigating Nov 25, 2020, 08:30 PM UTC

    Failures regarding postflight call logs have stopped. Programmable Voice APIs are no longer failing, and Calls postflight records are in place. However, for trunking calls, trunking records for calls that occurred during the outage may not be available. Those records are currently being rebuilt; it will take a few hours to populate those trunking records. We expect to provide another update in 30 minutes or as soon as more information becomes available.

  7. investigating Nov 25, 2020, 09:03 PM UTC

    We continue to populate trunking records for calls that occurred during the outage. We expect to provide another update in 2 hours or as soon as this process is finished.

  8. investigating Nov 25, 2020, 09:33 PM UTC

    Trunking records and Voice Insights for calls that occurred during the outage are now available in Console. This incident is resolved.

  9. investigating Nov 25, 2020, 10:04 PM UTC

    This incident has been resolved.

  10. resolved Nov 25, 2020, 10:05 PM UTC

    This incident has been resolved.