Rubrik incident

Ransomware Monitoring and Investigation - Snapshots processing slower than usual

Resolved after 2 days 20 hours, on September 5, 2023.

resolved Sep 08, 2023, 06:59 PM UTC

The fixes we have implemented have solved the issue. Although some customers may experience slow snapshot processing for the next two to three days.

Started
Sep 05, 2023, 10:08 PM UTC
Resolved
Sep 08, 2023, 06:59 PM UTC
Duration
2d 20h
Detected by Pingoru
Sep 05, 2023, 10:08 PM UTC

Update timeline

  1. investigating Sep 05, 2023, 10:08 PM UTC

    We are experiencing increased latency for Ransomware Monitoring and Investigation on Rubrik Security Cloud in all regions. Symptom(s): Anomalies may not appear in the dashboard, and there may be a reduction in the count of analyzed snapshots. Incident Start Time: 08/28/2023 10:00 PM PST Workaround: None at this time. Engineering is actively investigating to resolve this issue.

  2. identified Sep 06, 2023, 12:06 AM UTC

    The issue has been identified and a fix is being implemented.

  3. identified Sep 06, 2023, 06:43 AM UTC

    We are continuing to work on a fix for this issue.

  4. identified Sep 06, 2023, 04:21 PM UTC

    We are continuing to work on a fix for this issue.

  5. identified Sep 06, 2023, 07:54 PM UTC

    We are still continuing to work on a fix for this issue. We will give another update at 19:00 PST.

  6. identified Sep 07, 2023, 12:25 AM UTC

    We believe the change will start rolling out by 09/07/2023 01:00 PST, we will be continuously monitoring the service to evaluate the effectiveness of the mitigation.

  7. identified Sep 07, 2023, 03:03 PM UTC

    The change is finished rolling out, we are continuously monitoring the service to evaluate the effectiveness of the mitigation. We will provide another update at 10:00 PST.

  8. identified Sep 07, 2023, 05:25 PM UTC

    We are working on few more fixes for this issue. We will give another update at 18:00 PST.

  9. monitoring Sep 08, 2023, 12:18 AM UTC

    A fix has been implemented and we are monitoring the results.

  10. monitoring Sep 08, 2023, 06:02 AM UTC

    The issue has been fixed and we are seeing signs of improvement. We are further monitoring and will provide an update.

  11. resolved Sep 08, 2023, 06:59 PM UTC

    The fixes we have implemented have solved the issue. Although some customers may experience slow snapshot processing for the next two to three days.

Your product runs on more than Rubrik

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →