Zero Networks experienced a notice incident on March 19, 2025, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Mar 20, 2025, 12:32 PM UTC
Our engineering team implemented a hotfix to address functionality in one of our recently released features. During the implementation process, some users experienced a period where data in the portal was not refreshing properly and login attempts were unsuccessful. Impact: Affected Systems: Customer portal authentication and data services.
- postmortem Mar 20, 2025, 12:33 PM UTC
**Summary:** Our engineering team implemented a hotfix to address functionality in one of our recently released features. During the implementation process, some users experienced a period where data in the portal was not refreshing properly and login attempts were unsuccessful. **Impact:** Affected Systems: Customer portal authentication and data services. **Root Cause:** API Connection Issue: The API service did not automatically reconnect after pushing the mentioned update, requiring a manual restart of API. **Resolution:** Restarted the API pods to establish a new connection. **Prevention & Lessons Learned:** Automated API Recovery: We've added an API automation that restarts the services to prevent manual intervention. Improved Deployment Coordination: Ensure critical dependencies \(e.g., DB migrations\) are included in deployment plans before syncing updates.