Tangoe Mobile incident

Application Latency

Resolved after 1 day 5 hours, affecting Tangoe Mobile Portal, on January 3, 2022.

resolved Jan 04, 2022, 09:44 PM UTC

This incident has been resolved. System stability has resumed and we will continue actively monitoring the health of the system.

Started
Jan 03, 2022, 04:37 PM UTC
Resolved
Jan 04, 2022, 09:44 PM UTC
Duration
1d 5h
Detected by Pingoru
Jan 03, 2022, 04:37 PM UTC

Affected components

Tangoe Mobile Portal

Update timeline

  1. investigating Jan 03, 2022, 05:01 PM UTC

    We have received reports of some application latency on the Tangoe Mobile application. Technical teams are investigating to resolve and restore full service as expediently as possible. Updates will follow as available.

  2. monitoring Jan 03, 2022, 09:31 PM UTC

    We have found high CPU usage and DB locks associated with a very large number of automated jobs. This is resulting in occasional timeout errors. We are continuing to monitor should further issues arise.

  3. monitoring Jan 04, 2022, 05:21 AM UTC

    We have seen a relative improvement in application responsiveness. We are continuing to monitor for any additional problems.

  4. monitoring Jan 04, 2022, 03:32 PM UTC

    We are continuing to monitor the CPU usage. At this time performance is normal and running without errors.

  5. monitoring Jan 04, 2022, 04:15 PM UTC

    We are continuing to monitor for any further issues.

  6. monitoring Jan 04, 2022, 04:37 PM UTC

    We are currently monitoring a smaller spike in DB lockouts associated with scheduled worker jobs. The system is continuing to process and synchronize the jobs in queue on the backend.

  7. monitoring Jan 04, 2022, 05:16 PM UTC

    We are currently aware of the 500 errors occurring for certain reporting instruments. These errors are sporadic and we expect resolution later this afternoon. We do not anticipate any further issues as the number of automated jobs in process has been greatly reduced.

  8. resolved Jan 04, 2022, 09:44 PM UTC

    This incident has been resolved. System stability has resumed and we will continue actively monitoring the health of the system.

Your product runs on more than Tangoe Mobile

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →