Tangoe Mobile experienced a minor incident on January 3, 2022 affecting Tangoe Mobile Portal, lasting 1d 5h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 03, 2022, 05:01 PM UTC
We have received reports of some application latency on the Tangoe Mobile application. Technical teams are investigating to resolve and restore full service as expediently as possible. Updates will follow as available.
- monitoring Jan 03, 2022, 09:31 PM UTC
We have found high CPU usage and DB locks associated with a very large number of automated jobs. This is resulting in occasional timeout errors. We are continuing to monitor should further issues arise.
- monitoring Jan 04, 2022, 05:21 AM UTC
We have seen a relative improvement in application responsiveness. We are continuing to monitor for any additional problems.
- monitoring Jan 04, 2022, 03:32 PM UTC
We are continuing to monitor the CPU usage. At this time performance is normal and running without errors.
- monitoring Jan 04, 2022, 04:15 PM UTC
We are continuing to monitor for any further issues.
- monitoring Jan 04, 2022, 04:37 PM UTC
We are currently monitoring a smaller spike in DB lockouts associated with scheduled worker jobs. The system is continuing to process and synchronize the jobs in queue on the backend.
- monitoring Jan 04, 2022, 05:16 PM UTC
We are currently aware of the 500 errors occurring for certain reporting instruments. These errors are sporadic and we expect resolution later this afternoon. We do not anticipate any further issues as the number of automated jobs in process has been greatly reduced.
- resolved Jan 04, 2022, 09:44 PM UTC
This incident has been resolved. System stability has resumed and we will continue actively monitoring the health of the system.