Inspectorio incident

Analytics Incident

Minor Resolved View vendor source →

Inspectorio experienced a minor incident on May 30, 2018, lasting 1d 17h. The incident has been resolved; the full update timeline is below.

Started
May 30, 2018, 08:03 AM UTC
Resolved
Jun 01, 2018, 01:50 AM UTC
Duration
1d 17h
Detected by Pingoru
May 30, 2018, 08:03 AM UTC

Update timeline

  1. investigating May 30, 2018, 08:03 AM UTC

    Some issues reported related to data in Dashboard and analytics. We are currently investigating the issue with the highest priority, and will provide more information as soon as we have an update.

  2. identified May 30, 2018, 11:59 AM UTC

    We were able to identify the root cause of this issue. It is connected to a large data migration event that occurred today. The size of this migration led to instability in the database used for the Dashboard and Analytics components which is why data is not being displayed as intended. We have already taken measures to begin correcting this event and are continuing to work on resolving it fully. We will continue to update here regularly as we make progress on restoring service for the Dashboard and Analytics components. Please note, all other components (Purchase Orders, Bookings, Assignments, and Reports) are stable and functioning as intended. Thank you for your continued patience.

  3. identified May 30, 2018, 12:06 PM UTC

    We are continuing to work on a fix for this issue.

  4. identified May 30, 2018, 12:08 PM UTC

    We were able to identify the root cause of this issue. It is connected to a large data migration event that occurred today. The size of this migration led to instability in the database used for the Dashboard and Analytics components which is why data is not being displayed as intended. We have already taken measures to begin correcting this event and are continuing to work on resolving it fully. We will continue to update here regularly as we make progress on restoring service for the Dashboard and Analytics components. Please note, all other components (Purchase Orders, Bookings, Assignments, and Reports) are stable and functioning as intended. Thank you for your continued patience.

  5. identified May 30, 2018, 01:27 PM UTC

    We have confirmed that a small percentage of users are being impacted by this issue. We are continuing to work on resolving the root cause so everyone can fully utilize the Dashboard and Analytics components.

  6. identified May 30, 2018, 02:40 PM UTC

    We are continuing data synchronization in the Dashboard, and Analytics component. Data has been restored for several of the affected accounts already.

  7. identified May 30, 2018, 04:17 PM UTC

    We are continuing to work on a fix for this issue.

  8. monitoring May 30, 2018, 04:21 PM UTC

    The issue should be solved for most users, and we will continue to monitor this closely until normal performance is restored. If you have any issues, please contact [email protected]. Thank you for your continued patience.

  9. monitoring May 31, 2018, 01:55 AM UTC

    We are continuing to monitor for any further issues while continuing to make improvements to ensure normal performance is restored for any users who are still affected.

  10. monitoring May 31, 2018, 08:17 AM UTC

    We are continuing to monitor for any further issues.

  11. monitoring May 31, 2018, 08:17 AM UTC

    We are continuing to monitor for any further issues while continuing to make improvements to ensure normal performance is restored for any users who are still affected.

  12. monitoring May 31, 2018, 01:41 PM UTC

    All databases have been stabilized, and data restored. We will continue to monitor closely, and close the incident soon.

  13. resolved Jun 01, 2018, 01:50 AM UTC

    The Incident has been resolved, and we will continue to monitor the database for the next 48hrs. If you have any issues, please contact [email protected]