CINC incident

Site Outage

Critical Resolved View vendor source →

CINC experienced a critical incident on December 13, 2018 affecting Agent Dashboard and Homeseeker Website, lasting 2d 1h. The incident has been resolved; the full update timeline is below.

Started
Dec 13, 2018, 04:23 PM UTC
Resolved
Dec 15, 2018, 06:20 PM UTC
Duration
2d 1h
Detected by Pingoru
Dec 13, 2018, 04:23 PM UTC

Affected components

Agent DashboardHomeseeker Website

Update timeline

  1. identified Dec 13, 2018, 04:23 PM UTC

    Our sites are currently down and all hands are on deck to find a solution ASAP.

  2. identified Dec 13, 2018, 04:55 PM UTC

    We are continuing to work towards a solution for the outage. Thank you for your patience.

  3. monitoring Dec 13, 2018, 05:01 PM UTC

    We're starting to see some sites come back up. We're going to continue monitoring until everything is back to normal.

  4. monitoring Dec 13, 2018, 05:17 PM UTC

    Sites are back up, but there is still some severe slowness due. We're continuing to monitor and get everything back to normal.

  5. monitoring Dec 13, 2018, 05:34 PM UTC

    All sites are back up and accessible. There is still some slowness inside the platform. We'll update when everything is back to normal.

  6. monitoring Dec 13, 2018, 08:16 PM UTC

    We are currently experiencing small moments of outage and then the sites go back to normal for 15-20 minutes at a time. We are still working vigorously in the office and with our third-party partners to get everything back to normal as quickly as we can.

  7. monitoring Dec 13, 2018, 11:33 PM UTC

    We are continuing to work towards a solution. Thank you all for your patience through this challenging time.

  8. monitoring Dec 14, 2018, 01:16 PM UTC

    We are currently experiencing full outages again. We're working on switching some server-side things around to get the sites back up. We'll update when we see sites come back up.

  9. monitoring Dec 14, 2018, 01:21 PM UTC

    Sites are up, but still slow. We're expecting slowness all day until we can complete the very necessary updates tonight. We'll update if anything changes.

  10. monitoring Dec 15, 2018, 07:23 AM UTC

    We've made substantial improvements in our affected servers which will bring us back to our normal level of performance. We will continue to monitor through the night and tomorrow to ensure no issues pop up.

  11. resolved Dec 15, 2018, 06:20 PM UTC

    All systems are back to normal.