Lakeside Software experienced a critical incident on March 1, 2024 affecting SysTrack API/UI and SysTrack Endpoint Connections, lasting 3h 9m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 01, 2024, 02:17 PM UTC
We are currently investigating this issue.
- identified Mar 01, 2024, 02:20 PM UTC
We identified the problem and are working on remediating the issue.
- monitoring Mar 01, 2024, 03:22 PM UTC
A fix has been implemented and we are going to be actively monitoring the system. Due to the nature of the change, resolving to the cloud site might be delayed for some users depending on region and location.
- monitoring Mar 01, 2024, 03:23 PM UTC
We are continuing to monitor for any further issues.
- resolved Mar 01, 2024, 05:27 PM UTC
This incident has been resolved. We apologize for the inconvenience, If you're still experiencing performance problems, please reach out to [email protected].
- postmortem Mar 07, 2024, 08:14 PM UTC
# What was the issue? All customers in the impacted region experience problems loading the webpage. # What was the root cause? Our load balancer, Microsoft Application Gateway, stopped accepting traffic inbound. # What is the Prevention Strategy? We are still working with our cloud provider \(Azure\) to get a full RCA before we prepare an identification and prevention strategy.