Lakeside Software incident
SysTrack Cloud – Availability Issues
Lakeside Software experienced a critical incident on March 10, 2025 affecting SysTrack API/UI and SysTrack Endpoint Connections, lasting 2h 19m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 10, 2025, 11:33 AM UTC
We are currently investigating reported issues with SysTrack Cloud availability in the below regions. Some users may not be able to use the SysTrack cloud UI, make API calls, or use third-party integrations that utilize the API. We're actively working on identifying the root cause. We apologize for any inconvenience and will provide an update once more details become available.
- monitoring Mar 10, 2025, 12:25 PM UTC
A fix has been implemented and we monitoring the situation.
- resolved Mar 10, 2025, 01:52 PM UTC
This incident has been resolved.
- postmortem Mar 17, 2025, 01:24 PM UTC
# What was the issue? Access to SysTrack CE Cloud in Americas was inaccessible. # What was the root cause? An approved and standard maintenance change led to a halt of processing of new sensors in the Americas region. Consequently, the team had to suspend the intake of additional sensors until the root cause was determined and resolved. # What is the Prevention Strategy? 1. Additional review process for maintenance deployments **\(Completed\)** 2. Additional alerting on the backend for earlier indicators of an issue **\(April 1\)** 3. Updating the Sensor agent upload mechanism to implement resends or retries. **\(TBD\)**