Lakeside Software experienced a minor incident on July 8, 2025 affecting SysTrack API/UI and SysTrack Endpoint Connections, lasting 4h 34m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jul 08, 2025, 03:39 PM UTC
We are currently investigating this issue.
- identified Jul 08, 2025, 04:27 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Jul 08, 2025, 06:48 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jul 08, 2025, 08:13 PM UTC
This incident has been resolved. If you need further support, please open a ticket at the customer portal (https://customers.lakesidesoftware.com/).
- postmortem Jul 15, 2025, 03:46 PM UTC
# What was the issue? Access to the SysTrack CE America’s cloud was slow due to sensor processing. # What was the root cause? Preliminary root cause, which is still being verified, was an Azure networking issue causing the utilization of our caching service to respond slowly to requests. This slowness caused cascading slowness to other areas of the platform. # What is the Prevention Strategy? 1. Running a large cache until other steps have been taken. **\(Completed\)** 2. Migrations to a different networking path to access the cache **\(Sept 2025\)** 3. Updates to how the system gets the cache information for recovery quicker **\(Oct 2025\)** 4. Updating the Sensor agent upload mechanism to implement resends or retries. **\(TBD\)**