Expel experienced a major incident on February 27, 2025 affecting Investigative actions and Alert ingestion, lasting 22h 8m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- monitoring Feb 27, 2025, 05:27 PM UTC
We are monitoring a json decoding error that has bubbled up for all Orca Security devices. We have reached out to the vendor and will post another update at 1:00 EST
- monitoring Feb 27, 2025, 06:20 PM UTC
We are continuing to monitor this outage. We are in talks with the vendor at this time. Will provide another update at 2pm EST
- monitoring Feb 27, 2025, 07:03 PM UTC
We are continuing to monitor this outage. We are in talks with the vendor at this time. Will provide another update at 3pm EST.
- monitoring Feb 27, 2025, 08:54 PM UTC
We are continuing to monitor this outage. We are waiting on the vendor to provide guidance. Will provide another update at 5pm EST.
- monitoring Feb 27, 2025, 10:05 PM UTC
Orca has identified a fix and is going to roll it out and monitor it overnight. We will mark this as resolved when we see devices come back online and monitor.
- monitoring Feb 28, 2025, 03:06 PM UTC
All devices are reporting healthy. We're considering this resolved and will continue to monitor.
- resolved Feb 28, 2025, 04:28 PM UTC
Fix was issued @ 3am EST.