Neon Fundraise incident

Degraded performance on Registrants grid

Resolved after 19 hours 29 minutes, affecting Management interfaces, on November 20, 2023.

monitoring Nov 21, 2023, 02:36 PM UTC

A fix has been implemented and we are monitoring the results.

Started
Nov 20, 2023, 10:45 PM UTC
Resolved
Nov 21, 2023, 06:15 PM UTC
Duration
19h 29m
Detected by Pingoru
Nov 20, 2023, 10:45 PM UTC

Affected components

Management interfaces

Update timeline

  1. investigating Nov 20, 2023, 10:45 PM UTC

    This morning at around 7:20am Pacific, we received alerts that the Registrants grid (in the Admin, under People) was failing to load. After some research, we identified that the issue occurs specifically when sorting the grid ascending by Last Name, and that being the default sort configuration, it was failing for every load. As such, at 12:00pm Pacific, we deployed a hotfix that changes the default sort configuration to registration date, with newest first. That way, the grid doesn't fail every time it is loaded. Still, if an administrator manually changes the sort to Last Name ascending, the grid will fail in that instance. We are still investigating the root cause of the issue and will post an update when we have a fix.

  2. monitoring Nov 21, 2023, 02:36 PM UTC

    A fix has been implemented and we are monitoring the results.

  3. resolved Nov 21, 2023, 06:15 PM UTC

    We deployed some slight changes to the grid that has resolved the issue.

Your product runs on more than Neon Fundraise

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →