I.T Communications Limited incident

SoftSwitch PBX CL1 - Server 02 and 03 Issues

Resolved after 1 hour 14 minutes, affecting Soft-Switch PBX - pbx-cl1-01.soft-switch-pbx.uk and Soft-Switch PBX - pbx-cl1-02.soft-switch-pbx.uk and 1 more, on November 5, 2021.

monitoring Nov 05, 2021, 05:37 PM UTC

Pbx01 now back in service.

Started
Nov 05, 2021, 04:40 PM UTC
Resolved
Nov 05, 2021, 05:54 PM UTC
Duration
1h 14m
Detected by Pingoru
Nov 05, 2021, 04:40 PM UTC

Affected components

Soft-Switch PBX - pbx-cl1-01.soft-switch-pbx.ukSoft-Switch PBX - pbx-cl1-02.soft-switch-pbx.ukSoft-Switch PBX - pbx-cl1-03.soft-switch-pbx.uk

Update timeline

  1. monitoring Nov 05, 2021, 04:40 PM UTC

    We found a fault with the Databases which stores all the data for Soft-Swtich Cluster 01 and waiting for the databass to catch up with each other before the service will return back to normal.

  2. monitoring Nov 05, 2021, 04:41 PM UTC

    We are continuing to monitor for any further issues.

  3. monitoring Nov 05, 2021, 04:59 PM UTC

    Soft-Switch Cluster 1 - stores its data on 3 seperate databases. Last night we tried to introduce a 4th Database which failed to sync. A fix was applied however casued the other databases replication to crash. The resync is taking its time due to the size of the databases. We continue to work on this until the service is restored. Updates to follow.

  4. monitoring Nov 05, 2021, 05:02 PM UTC

    We are now in the process of failing the servers over to the good working database so service can be resumed quicker. This will allow us to work on the other databases without continued loss of service.

  5. monitoring Nov 05, 2021, 05:11 PM UTC

    PBX-03 is now back up and working.

  6. monitoring Nov 05, 2021, 05:17 PM UTC

    PBX-02 is now Operational

  7. monitoring Nov 05, 2021, 05:37 PM UTC

    Pbx01 now back in service.

  8. resolved Nov 05, 2021, 05:54 PM UTC

    Server 4 is now added to cluster for WebRTC Full service restored

Your product runs on more than I.T Communications Limited

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →