Lumos had 48 outages in the last 2 years totaling 56h 31m of downtime — averaging 2 incidents per month.
There were 48 Lumos outages since August 23, 2024 totaling 56h 31m of downtime. Each is summarised below — incident details, duration, and resolution information.
Our team is investigating reports of Lumos failing to load or loading indefinitely. We will share updates here as soon as we have them.
A fix has been implemented, and our team is monitoring stability.
We are continuing to monitor for any further issues.
This has been resolved.
The incident that impacted users earlier in the afternoon on 4/28 (PT) re-triggered during the following time slots: 4/28 - 5:24PM PT to 9:09PM PT 4/29 - 2:15AM PT to 3:30AM PT During these times, access requests processed much slower than normal. The team has identified the root cause and deployed a fix to avoid this slowdown moving forward.
We are currently investigating this issue.
We are continuing to investigate this issue.
The issue has been identified and a fix is being implemented.
This incident has been resolved. The approximate time of impact was 3:46PM PT to 4:57PM PT.
There was a brief issue blocking workflows (the series of customizable steps that are automatically executed for provisioning, deprovisioning, etc.) from being saved after being created or modified. We've identified the root cause and implemented a fix, and behavior has since returned to normal.
For a brief period, Lumos encountered longer than expected loading times and occasional timeouts throughout the entire platform. This issue has since been resolved, and platform performance has been restored.
The team continued monitoring the fix from earlier today regarding platform performance, and we're currently seeing a return of slow loading speeds and platform performance. Our engineers are working to rectify ASAP.
The team has identified the root cause of the platform performance issues and is working on rolling out a fix.
Lumos is back online, and we're monitoring to ensure stability.
The issue has been resolved, and performance has returned to normal.
The team is actively investigating UI issues inclusive but not limited to: login failures, slow loading speeds, and general inconsistent performance. We'll provide updates as soon as we have them.
Lumos is back online, and we're monitoring performance to ensure that there are no further issues.
The team has confirmed this issue is fully resolved and platform performance is functioning as expected.
This is separate (although likely related) to the partial outage with custom attribute syncs that occurred earlier today. Since yesterday at 10:12 PM GMT, there have been additional failures updating the values of custom attributes in Lumos from integrations. We're working to identify the root cause and assess impact, and will share updates here as we work to correct things.
The issue has been identified and a fix is being implemented.
This incident has been resolved.
We're deploying a hotfix to address the root cause and will soon be resuming custom attribute syncs and backfilling any that were missed.
If you have Okta connected to Lumos, you may have noticed a temporary influx of API requests coming in as a consequence of this issue, but no request limits were reached, and the issue has been remediated. Our team was also able to validate that there was no data loss as a result of this issue.
Our engineers are actively investigating an issue causing the AppStore and all apps to fail to use for Admins and Users alike. We'll share updates as soon as we pinpoint a root cause and fix.
The team has identified the root cause. As we work to resolve, please not that all tabs within the platform may be slow to load or fail to load altogether.
The incident has been resolved and all systems are back to being operational.
We're investigating reports of errors when staging new apps in an Access Review.
This incident has been resolved.
During 12:27 - 1:48a EST, the AppStore Settings sidebar failed to load across all customer accounts. The root cause was identified and the issue was quickly mitigated. All is functioning as expected now.
We are investigating an incident affecting the Activity Log where actions taken in Lumos may not be logged. Some customers may be impacted during this time.
Our engineers have identified the root cause of this issue and are actively working to rectify.
We are continuing to work on a fix for this issue.
This incident has been resolved.
We're currently investigating an issue blocking many customers, particularly those who use SSO, from logging into Lumos as a result of WorkOS's ongoing incident (https://status.workos.com/).
The team is aware of the scope of impact and is working to mitigate any login issues as a result of the WorkOS outage.
We've observed reduced login errors from Lumos customers using SSO. The team is closely monitoring the situation and will not declare this resolved until we receive clearance from WorkOS (https://status.workos.com/).
This WorkOS incident is no longer impacting Lumos logins. The impact was present from 12:06p - 12:27p EST.
We are currently investigating this issue.
We have found the issue and are implementing a fix.
This incident has been resolved.
We've identified an internal issue which is causing parts of Lumos to fail if they would write to the database. We've identified the cause and are current in progress of rolling out a fix.
We've rolled out a fix and this issue is resolved.
We have a fix currently in progress. This is impacting the employee AppStore, admin AppStore, and the task center from loading. Also impacting Slack usage. We will post updates when functionality is returned.
This incident has been resolved. Full functionality is returned.
Our engineers have identified the root cause of this issue and are actively working to rectify.
Our team has launched a fix and is actively monitoring success. Deprovisioning relating to offboarding and bulk user removals is now working successfully.
We are currently investigating an issue where Lumos Members are being incorrectly tagged as Service Accounts for all customers. We'll provide updates as soon as our team pinpoints the root cause and rolls out a fix.
The team pinpointed the root cause of this erroneous display and is working to identify a fix.
The fix is rolling out and is being monitored for success.
Some syncs are still underway to ensure all identities are updated appropriately, but the fix is live and you'll see the Identities to update as expected. Rest assured, all data will return to its original state.
The team has identified an issue causing integration syncs to fail, resulting in connectivity errors. We are working to pinpoint the source of the issue and implement a fix as we speak.
The root cause of this issue has been pinpointed, and the team is working to roll out a fix ASAP.
A fix is being rolled out now and we will monitor the success of current integration syncs.
Confirming that syncs are being rerun, and we're monitoring for success.
Syncs are running per usual, and integration health is being restored. All future syncs should be operating as expected!
The issue has been identified and investigated. A fix to resolve this issue is currently being shipped.
This incident has been resolved.
We are currently investigating an issue regarding failures in Access Request submissions, approvals, and rejections. We will share updates as soon as possible.
Access requests are now able to be created, as well as approved and denied.
The "Spend by employee attribute" table on Lumos is currently displaying an inaccurate number of unassigned licenses. We are currently investigating this issue.
We are continuing to investigate this issue.
This incident has been resolved.