Lakeside Software incident

Americas Elevated Error Rate

Minor Resolved View vendor source →

Lakeside Software experienced a minor incident on January 30, 2023 affecting SysTrack API/UI, lasting 13h 30m. The incident has been resolved; the full update timeline is below.

Started
Jan 30, 2023, 02:05 AM UTC
Resolved
Jan 30, 2023, 03:35 PM UTC
Duration
13h 30m
Detected by Pingoru
Jan 30, 2023, 02:05 AM UTC

Affected components

SysTrack API/UI

Update timeline

  1. investigating Jan 30, 2023, 02:14 PM UTC

    We're experiencing an elevated level of API/Website errors and are currently investigating the issue.

  2. monitoring Jan 30, 2023, 02:38 PM UTC

    We have identified the problem and taken action to remediate the issue. We are no longer experiencing the elevated error. We will continue to monitor the issue for the next hour to ensure it is resolved.

  3. resolved Jan 30, 2023, 03:35 PM UTC

    After monitoring the situation for the past hour, we are confident that the remediation has worked. If you have any future issues, please contact Lakeside Support at [email protected].

  4. postmortem Feb 02, 2023, 07:14 PM UTC

    # What was the issue Website was not loading for some customers. # What was the impact Some traffic of web requests were affected. # Root cause: Some traffic routed to unhealthy web servers. # Prevention Strategy Re-evaluate adequacy of health check for the web servers.