Lakeside Software incident

Application Errors

Minor Resolved View vendor source →

Lakeside Software experienced a minor incident on February 7, 2024 affecting SysTrack API/UI, lasting 1d 1h. The incident has been resolved; the full update timeline is below.

Started
Feb 07, 2024, 12:00 PM UTC
Resolved
Feb 08, 2024, 01:30 PM UTC
Duration
1d 1h
Detected by Pingoru
Feb 07, 2024, 12:00 PM UTC

Affected components

SysTrack API/UI

Update timeline

  1. investigating Feb 07, 2024, 04:59 PM UTC

    We are currently investigating this issue.

  2. identified Feb 07, 2024, 05:04 PM UTC

    We have identified problem as an Azure problem and are working with the vendor to fix the root cause.

  3. identified Feb 07, 2024, 05:08 PM UTC

    We have identified problem as an Azure problem and are working with the vendor to fix the root cause.

  4. identified Feb 07, 2024, 07:50 PM UTC

    We have identified problem as an Azure problem and are working with the vendor to fix the root cause.

  5. identified Feb 07, 2024, 10:03 PM UTC

    We have identified problem as an Azure problem and are working with the vendor to fix the root cause.

  6. identified Feb 08, 2024, 01:33 AM UTC

    We have identified problem as an Azure problem and are working with the vendor to fix the root cause.

  7. identified Feb 08, 2024, 04:11 AM UTC

    We have identified problem as an Azure problem and are working with the vendor to fix the root cause.

  8. identified Feb 08, 2024, 06:45 AM UTC

    We have identified problem as an Azure problem and are working with the vendor to fix the root cause.

  9. identified Feb 08, 2024, 08:43 AM UTC

    We have identified problem as an Azure problem and are working with the vendor to fix the root cause

  10. identified Feb 08, 2024, 10:17 AM UTC

    We have identified problem as an Azure problem and are working with the vendor to fix the root cause.

  11. identified Feb 08, 2024, 12:15 PM UTC

    We have identified problem as an Azure problem and are working with the vendor to fix the root cause.

  12. resolved Feb 08, 2024, 01:30 PM UTC

    This incident has been resolved. Azure has identifying the root cause and mitigated the issue. We apologize for the inconvenience, If you're still experiencing performance problems, please reach out to [email protected].

  13. postmortem Feb 14, 2024, 04:55 PM UTC

    # What was the issue? Some customers experienced errors loading certain pages within the UI. # What was the root cause? Microsoft Azure had an issue \(Tracking ID N\_BZ-RSZ\) with Entra Domain Services not being able to reach the Azure Table storage causing access storage accounts to not be available to the application. # What is the Prevention Strategy? We are still working with our cloud provider \(Azure\) to get a full RCA before we prepare an identification and prevention strategy.