Lakeside Software incident

SysTrack Cloud – Reports of Delayed Performance -AB

Major Resolved View vendor source →

Lakeside Software experienced a major incident on November 19, 2024 affecting SysTrack API/UI, lasting 3h 32m. The incident has been resolved; the full update timeline is below.

Started
Nov 19, 2024, 10:51 AM UTC
Resolved
Nov 19, 2024, 02:23 PM UTC
Duration
3h 32m
Detected by Pingoru
Nov 19, 2024, 10:51 AM UTC

Affected components

SysTrack API/UI

Update timeline

  1. investigating Nov 19, 2024, 10:51 AM UTC

    Some users may not be able to use the SysTrack cloud UI, make API calls, or use third-party integrations that utilize the API. We're actively working on identifying the root cause. We apologize for any inconvenience and will provide an update once more details become available.

  2. identified Nov 19, 2024, 12:09 PM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Nov 19, 2024, 12:39 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. monitoring Nov 19, 2024, 02:01 PM UTC

    We are continuing to monitor for any further issues.

  5. resolved Nov 19, 2024, 02:23 PM UTC

    This incident has been resolved.

  6. postmortem Nov 25, 2024, 09:11 PM UTC

    # What was the issue? End users had intermittent access issues with the product. # What was the root cause? A backend scaling issues cause some connections to respond back. # What is the Prevention Strategy? 1. Add additional compute \(Completed on day of incident\) 2. Split backend components to allow them to scale independently \(Expect ETA 12/15\)