Intellishift incident
IntelliShift - Silent Passenger telematics live feed delay.
Intellishift experienced a minor incident on October 5, 2022 affecting Web Application, lasting 1h 29m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 04, 2022, 04:17 PM UTC
As of 11:00am Eastern IntelliShift Telematics is experiencing a delay of the live feed. Our team is investigating the issue and are working to resolve this issue as soon as possible. During this time Map View - Live Tracking and Reports may be delayed.
- investigating Oct 05, 2022, 09:37 AM UTC
IntelliShift's Live feed is returning to normal, we are monitoring this closely until the live feed has been completely restored.
- resolved Oct 05, 2022, 09:41 AM UTC
IntelliShift's - Silent Passenger Live Feed has been restored and assets are back to reporting in real-time. Reports will reflect the activity of the assets during the period of the delay. We are continuing to monitor this very closely. Thank You.
- postmortem Oct 15, 2022, 07:10 PM UTC
The IntelliShift telematics live feed experience delays in updating the Map View module if the IntelliShift application. The IntelliShift Product Development & Architecture teams were immediately alerted and worked to find the root case and address the issue immediately. This issue occurred as a result of increased latency and performance degradation on several third-party map, geocoding, and address look up web services that the IntelliShift applications utilize to deliver standard product functionality to users. The increase in response times caused delays in the IntelliShift telematics live feed that is used to update the Map View. The IntelliShift Product Development & Architecture teams worked to address this issue by reviewing all systems logs. After seeing the potential root cause, the team began working with the third-party, while adding other services to reduce the impact of the performance degradation that was occurring on the third-party service. Once stabilized, additional monitoring was added to the service to mitigate the impact this could have in the future. To prevent issue from impacting IntelliShift users in the future, the IntelliShift Product Development & Architecture have added additional services to augment the existing service to reduce the reliance on these services going forward.