Splunk OnCall incident

VictorOps Service Disruption: All Systems Operational

Critical Resolved View vendor source →

Splunk OnCall experienced a critical incident on April 19, 2020 affecting Notifications - SMS and Alert Ingestion and 1 more component, lasting 1h 55m. The incident has been resolved; the full update timeline is below.

Started
Apr 19, 2020, 11:22 PM UTC
Resolved
Apr 20, 2020, 01:17 AM UTC
Duration
1h 55m
Detected by Pingoru
Apr 19, 2020, 11:22 PM UTC

Affected components

Notifications - SMSAlert IngestionNotifications - Google PushAlert Ingestion - Inbound emailNotifications - Apple PushNotifications - PhoneNotifications - Email

Update timeline

  1. investigating Apr 19, 2020, 11:22 PM UTC

    We are actively investigating a service disruption and will be posting updates as efficiently as possible. If you have any immediate questions, please contact the VictorOps Support Team: [email protected]

  2. monitoring Apr 20, 2020, 12:13 AM UTC

    A fix has been implemented and we are monitoring the results.

  3. monitoring Apr 20, 2020, 12:58 AM UTC

    We are continuing to monitor for any further issues.

  4. monitoring Apr 20, 2020, 01:00 AM UTC

    We are continuing to monitor for any further issues.

  5. resolved Apr 20, 2020, 01:17 AM UTC

    After monitoring our remediation efforts surrounding this incident, we are moving this incident to a resolved status. Brief Incident Overview: From approximately 4:30 PM (Mountain) to 5:35 PM (Mountain) on Sunday (04.19.20), we experienced service disruption challenges that may have adversely affected the overall VictorOps end-user experience. We sincerely apologize for any unintended inconvenience this issue may have caused. If you have any immediate questions surrounding this incident, please contact the VictorOps Support Team at: [email protected]