Lakeside Software incident
Lakeside Software Cloud Services OUTAGE - Germany
Lakeside Software experienced a critical incident on January 25, 2023 affecting SysTrack API/UI and SysTrack Endpoint Connections, lasting 2h 18m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 25, 2023, 08:52 AM UTC
Lakeside Software is currently experiencing an outage related to SysTrack Cloud Edition impacting the below regions. The outage is currently impacting core cloud services. During this outage, users will not be able to use the SysTrack cloud UI, make API calls, or use third-party integrations that depend on the API. We're actively working on identifying the root cause and will resolve the problem as soon as possible. We apologize for the inconvenience and will provide an update as soon as all services are operational.
- investigating Jan 25, 2023, 09:03 AM UTC
The Incident is related to Azure Global outage on Networking Services. We are monitoring the status and we are in constant touch with Azure. We will share the next update as soon possible.
- identified Jan 25, 2023, 09:13 AM UTC
We are currently investigating this issue.
- investigating Jan 25, 2023, 09:13 AM UTC
We are currently investigating this issue.
- identified Jan 25, 2023, 09:17 AM UTC
The Incident is related to Azure Global outage on Networking Services. We are monitoring the status and we are in constant touch with Azure. We will share the next update as soon possible.
- monitoring Jan 25, 2023, 09:43 AM UTC
Azure Teams has remediated the issue as per the latest update. Lakeside Services on SysTrack Cloud Edition - Germany are back to online. We are continuing to actively monitor the situation. Tracking ID - VSG1-B90 https://app.azure.com/h/VSG1-B90/5287ef
- monitoring Jan 25, 2023, 09:50 AM UTC
We are continuing to monitor for any further issues.
- resolved Jan 25, 2023, 11:10 AM UTC
This incident has been resolved. After monitoring the situation for the past 1.5 hours, we are confident that Microsoft's Mitigation has worked. If you have any future issues, please contact Lakeside Support at [email protected].
- postmortem Feb 02, 2023, 07:31 PM UTC
# What was the issue? Lakeside Germany website was not loading for all customers. # What was the impact? All customers with Lakeside Cloud Tenant on Lakeside Germany domains were not able to access website and its modules. # Root cause: Microsoft Networking outage [\(Tracking ID VSG1-B90\) ](https://status.azure.com/en-us/status/history/)between 07:05 UTC and 12:43 UTC on 25 January 2023. # Prevention Strategy Implement external https checks for the website that will also trigger alerts for faster customer notifications.