Recart incident

Performance degradation on our Dashboard

Minor Resolved View vendor source →

Recart experienced a minor incident on May 10, 2023 affecting Dashboard, lasting 6h 24m. The incident has been resolved; the full update timeline is below.

Started
May 10, 2023, 02:04 PM UTC
Resolved
May 10, 2023, 08:28 PM UTC
Duration
6h 24m
Detected by Pingoru
May 10, 2023, 02:04 PM UTC

Affected components

Dashboard

Update timeline

  1. identified May 10, 2023, 02:04 PM UTC

    We made some optimization to our database clusters. The removal processes started taking more time and generating performance degradation. We are working with our providers on the issue.

  2. identified May 10, 2023, 03:12 PM UTC

    We found a resolution to the performance problem. We already started the mitigation. We expect the issue to be resolved in 1-2 hours.

  3. identified May 10, 2023, 04:30 PM UTC

    The change we applied takes longer than expected, it's around 50% done.

  4. monitoring May 10, 2023, 07:36 PM UTC

    The issue has been resolved, performance problems cannot be seen in the system. We are monitoring further to make sure everything is alright.

  5. resolved May 10, 2023, 08:28 PM UTC

    The issue has been resolved, our monitors show proper response times.