Singlewire Software incident
Some Local User Loaders Updated Users Erroneously
Singlewire Software experienced a major incident on July 31, 2025 affecting Local User Loader, lasting 1d 10h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jul 31, 2025, 01:52 PM UTC
We're experiencing an issue where users may be updated by the local user loader unintentionally, which may cause them to be removed from security groups and distribution lists. Our development team is working to identify the root cause and implement a solution. Any customer using the local user loader may be affected by this issue, so we recommend pausing any local user loader automation and holding off on manual updates to load requests. Cloud user loaders (Entra, School Information Systems) are not impacted by this issue. We'll update this incident when we have identified a root cause or within three hours.
- identified Jul 31, 2025, 02:03 PM UTC
We have identified the problem and are implementing a fix. To minimize additional impact until the fix is complete, we are temporarily disabling all local user loaders.
- identified Jul 31, 2025, 05:02 PM UTC
We have identified the root cause of the issue and are testing a fix for it. All local user loaders remain non-functional as a precaution until we have finished the fix. We are working to identify and restore user data that was erroneously changed or deleted.
- monitoring Jul 31, 2025, 08:52 PM UTC
We have deployed a change that fixes the issue and re-enabled local user loaders. We continue to work towards restoring user data that was erroneously changed or deleted.
- resolved Aug 01, 2025, 11:55 PM UTC
Although the issue with the Local User Loader has been corrected, our Support team will be reaching out to affected customers to help restore their data in the next few days.