I.T Communications Limited incident

PBX-CL1-02 down due to software issue

Major Resolved View vendor source →

I.T Communications Limited experienced a major incident on February 25, 2017, lasting 6h 58m. The incident has been resolved; the full update timeline is below.

Started
Feb 25, 2017, 10:56 AM UTC
Resolved
Feb 25, 2017, 05:55 PM UTC
Duration
6h 58m
Detected by Pingoru
Feb 25, 2017, 10:56 AM UTC

Update timeline

  1. identified Feb 25, 2017, 10:56 AM UTC

    We have discovered a failure in the OS software and are currently restoring from backup. Update to follow

  2. identified Feb 25, 2017, 11:13 AM UTC

    We are still working on restoring service. Estimated time is up to 2 hours due to the size of the database. Further update to follow

  3. identified Feb 25, 2017, 11:51 AM UTC

    We are still waiting for the restore to complete. currently at 27%

  4. identified Feb 25, 2017, 11:56 AM UTC

    We are still waiting for the restore to complete. currently at 29%

  5. identified Feb 25, 2017, 12:20 PM UTC

    We are still waiting for the restore to complete. currently at 84%

  6. identified Feb 25, 2017, 12:26 PM UTC

    We are still waiting for the restore to complete. currently at 96%

  7. identified Feb 25, 2017, 12:35 PM UTC

    Restore of data now complete. now looking to restore service.

  8. monitoring Feb 25, 2017, 12:39 PM UTC

    full service is now restored. we are continuing to monitor

  9. resolved Feb 25, 2017, 05:55 PM UTC

    This incident has been resolved.