CallFire incident

CallFire Tesla UI Responsiveness

Major Resolved View vendor source →

CallFire experienced a major incident on September 19, 2014, lasting 10d 23h. The incident has been resolved; the full update timeline is below.

Started
Sep 19, 2014, 09:17 PM UTC
Resolved
Sep 30, 2014, 09:06 PM UTC
Duration
10d 23h
Detected by Pingoru
Sep 19, 2014, 09:17 PM UTC

Update timeline

  1. monitoring Sep 19, 2014, 09:17 PM UTC

    Our CallFire Tesla platform is currently experiencing slow response times when accessing the website. The issue is being actively monitored, and new edge server capacity is being provisioned.

  2. monitoring Sep 19, 2014, 11:34 PM UTC

    An additional UI server has been added to help alleviate the issue. Situation still being monitored.

  3. monitoring Sep 20, 2014, 04:18 PM UTC

    There are sporadic issues of slowness. We are continuing to monitor

  4. monitoring Sep 22, 2014, 01:48 PM UTC

    Two additional edge servers were provisioned to help with the increased traffic. This should alleviate the freezing of UI issues that have been reported last week. This issue is also being actively monitored.

  5. identified Sep 23, 2014, 06:18 PM UTC

    Tesla is having intermittent issues. we are working to resolve. status will get updated soon

  6. monitoring Sep 24, 2014, 08:55 AM UTC

    The Engineering teams completed an exhaustive deploy and worked out many issues and load tested the system with up to 5000 concurrent outbound calls and 1500 concurrent inbound calls. The system is going to be actively monitored for the next 24 hours before closing this incident.

  7. investigating Sep 25, 2014, 05:44 PM UTC

    There are intermittent, sporadic incidents of slow UI response times. Most customers will continue to have have a good experience with the website.

  8. investigating Sep 26, 2014, 03:26 PM UTC

    There are changes in place to greatly reduce the albeit sporadic incidents of slow UI times. The engineering teams are working diligently on a more permanent fix.

  9. monitoring Sep 29, 2014, 03:57 PM UTC

    We have no reports of any major issues today. This incident will be kept open for 24 hours as we continue to monitor.

  10. resolved Sep 30, 2014, 09:06 PM UTC

    This incident has been resolved.