Service Degradation – Some Customers Experiencing Login Issues After Maintenance

Update timeline

1. investigating Mar 08, 2026, 07:10 PM UTC

   Description: During our recent maintenance window, a subset of user accounts were inadvertently disabled. We are actively working to identify affected users and restore access as quickly as possible. Impact: Some customers may experience login issues when accessing their sites. Monitoring and alerting capabilities remain fully operational and are not impacted. Next Steps: Restoration efforts are ongoing. Updates will follow as more information becomes available. We appreciate your patience as we work to restore full functionality.

2. identified Mar 09, 2026, 01:13 PM UTC

   Our team has identified a suspected cause of the interruption of user authorization to sites and is taking steps to remediate the issue. Impact: A significant number of customers are receiving errors when they log into Auvik The following services are not affected: Monitoring, alerting, or PSA integrations Please report any related issues to Auvik Support so we can track and assist further. Next Steps: We are implementing mitigation measures and will provide progress updates as they become available.

3. identified Mar 09, 2026, 03:04 PM UTC

   Our team has identified a suspected cause of the interruption of user authorization to sites and is taking steps to remediate the issue. Impact: A significant number of customers are encountering errors when they log in to Auvik. Access for these users is currently being restored. The following services are not affected: Monitoring, alerting, or PSA integrations Please report any related issues to Auvik Support so we can track and assist further. Next Steps: We are implementing mitigation measures and will provide progress updates as they become available.

4. identified Mar 09, 2026, 05:27 PM UTC

   Our team has identified a suspected cause of the interruption of user authorization to sites and is taking steps to remediate the issue. Impact: A significant number of customers are encountering errors when they log in to Auvik. Access for users on the US1, US3, and US5 clusters has been restored. We are working our way through the remaining clusters The following services are not affected: Monitoring, alerting, or PSA integrations Please report any related issues to Auvik Support so we can track and assist further.

5. identified Mar 09, 2026, 07:03 PM UTC

   Our team has identified a suspected cause of the interruption of user authorization to sites and is taking steps to remediate the issue. Impact: A significant number of customers are encountering errors when they log in to Auvik. Access for users on the US1, US2, US3, US5, AU1, CA1, EU1, EU2 clusters has been restored. We are working our way through the remaining clusters US4 and US6 The following services are not affected: Monitoring, alerting, or PSA integrations Please report any related issues to Auvik Support so we can track and assist further.

6. identified Mar 09, 2026, 07:43 PM UTC

   Our team has identified a suspected cause of the interruption of user authorization to sites and is taking steps to remediate the issue. Impact: A significant number of customers are encountering errors when they log in to Auvik. Access for users on the US1, US2, US3, US4, US5, AU1, CA1, EU1, EU2 clusters has been restored. We are working our way through the remaining cluster, US6. The following services are not affected: Monitoring, alerting, or PSA integrations Please report any related issues to Auvik Support so we can track and assist further.

7. resolved Mar 09, 2026, 09:26 PM UTC

   The incident has been fully resolved, and all services are operating normally. Customers should no longer experience any login issues. If you continue to experience problems, please don't hesitate to contact Auvik Support. We will provide a Root Cause Analysis (RCA) once it is available.

8. postmortem Mar 17, 2026, 02:49 PM UTC

   # Service Degraded - Some Customers Experiencing Login Issues After Maintenance ## Root Cause Analysis ### Duration of the incident Discovered: Mar 7, 2026 18:42 - UTC Resolved: Mar 9, 2026 21:10 - UTC ### Cause Following a platform upgrade, an internal reconciliation process incorrectly identified a subset of user records as deleted. This occurred due to an error condition in a synchronization component that failed to process all expected user data during reconciliation. ### Effect Affected users were able to log in to the platform but were unable to access their expected sites or tenants due to missing user authorizations. In some cases, API access using previously generated API keys was also affected. Monitoring, alerting, and data collection services continued to function normally throughout the incident. ### Action taken Engineering paused the process that spreads the changes across the platform to prevent further impact. User access was then restored using backup data taken prior to the upgrade. This allowed the team to recover user access, permissions, and related access keys. After restoration was completed and access was verified across all environments, the incident was declared resolved. ### Future consideration\(s\) * Implement additional monitoring to detect unexpected changes in user or authorization records. * Add safeguards within reconciliation processes to prevent large-scale unintended deletions. * Improve post-upgrade validation checks to identify abnormal record changes earlier. * Enhance disaster recovery tooling and backup retrieval processes to speed up restoration efforts.