Expel Outage History

We are aware of an issue with the Expel Alerts count being inflated for customers. We have identified the root cause and are working on a remediation. SOC operations and alert ingestion are not affected. We will provide another update by 1pm EDT.

We have remediated the issue with Expel Alerts count being inflated for customers by shifting to a different source of statistics. We are working on fixing the underlying issue with our main source of count data. We will provide another update by 1pm EDT.

Update: We have confirmed there are no technical issues with Workbench or with our third party systems that support notifications. Microsoft has since updated its Teams service health to indicate they are investigating a possible issue. We recommend you check your MSFT 365 service health and Microsoft's status page for updates: https://status.cloud.microsoft/m365/referrer=serviceStatusRedirect Until further notice, please continue to login to Workbench for timely updates and/or to engage with Expel's SOC. We will provide notice when Workbench notifications to Teams are operational.

We have identified the root cause as an issue with a 3rd party vendor. They are actively working on resolving the issue. We have confirmed that outbound notifications from Workbench have not been delivered to to User Email, Email Ticketing, Slack, MS Teams, PagerDuty, and OpsGenie since 19:30 UTC. These notifications are queued and will be re-sent when the issue is resolved. Alert ingestion is not affected. We will provide another update by 21:30 UTC.

Outbound notifications from Workbench to User Email, Email Ticketing, Slack, MS Teams, PagerDuty, and OpsGenie continue to be delayed. We will provide an update by 22:00 UTC.

Outbound notifications from Workbench to User Email, Email Ticketing, Slack, MS Teams, PagerDuty, and OpsGenie continue to be delayed. We will provide an update by 22:30 UTC.

We believe we have identified the root cause of duplicate phishing outcome emails and are preparing a hotfix. Next update at 5pm EST.

Workbench cannot currently deliver notifications to Microsoft Teams. The issue began around 9:45am ET. Expel is actively investigating the issue to determine if it is related to the larger Cloudflare outage. We recommend you utilize email notifications in the interim, and can confirm communicating with Expel personnel via Microsoft Teams still works. The issue is specific to Workbench notifications.

We’re upgrading our API connection and that may impact delivery of some alerts to Workbench. We are actively investigating and collaborating on a solution to ensure the consistent and complete delivery of all alert data. Please reach out to Orca if you’d like additional information. Thank you for your patience and understanding as we work through this.

Microsoft has reported an outage affecting O365 services. As a result of this outage, our integration with O365 is currently reporting as unhealthy. We are actively monitoring this issue and will provide an update within 60 minutes, or as soon as more information is available.

Microsoft has reported an outage affecting O365 services. As a result of this outage, our integration with O365 is currently reporting as unhealthy. We are actively monitoring this issue and will provide an update within 60 minutes, or as soon as more information is available.

Microsoft has reported an outage affecting O365 services. As a result of this outage, our integration with O365 is currently reporting as unhealthy. We are actively monitoring this issue and will provide an update within 60 minutes, or as soon as more information is available.

Microsoft has reported an outage affecting O365 services. As a result of this outage, our integration with O365 is currently reporting as unhealthy. We are actively monitoring this issue and will provide an update within 60 minutes, or as soon as more information is available.

The issue has been identified and a fix is being implemented.

We have identified an issue with a 3rd party vendor that is causing a 10-15 minute delay in sending email, slack and MS Teams notifications from Workbench. Alert ingestion is not affected. Please be aware that this delay applies to all client notifications, such as Verify Actions and New Incident Notifications. There is no other impact to SOC operations. Customers can continue to request support as usual (ticket creation is not affected by this delay).

We are continuing to work on a fix for this issue.

There is a known issue with Wiz. This is preventing our ability to triage and investigate Wiz alerts. Wiz have identified the root cause and a fix is being implemented (https://status.wiz.io/). As an interim workaround for impacted customers, we are going to promote/verify any critical or high Wiz alerts that come into the queue

We are continuing to work on a fix for this issue.

Alert ingestion is currently delayed. We have identified the root cause and are working on a remediation. We will provide another update by 3:30pm EDT.

Alert ingestion is still delayed. We have identified the root cause and are working on a remediation. We will provide another update by 4:00pm EDT.

Alert ingestion is still delayed. We have identified the root cause and are working on a remediation. We will provide another update by 4:30pm EDT.

Alert ingestion is still delayed. We have encountered further issues and are working on identifying the root cause. We will provide another update by 5:30pm EDT.

Alert ingestion is still delayed. We have taken steps to mitigate the issue and are monitoring to ensure recovery. We will provide another update by 6:30pm EDT.