Neon Fundraise incident

System Inaccessible or slow

Major Resolved View vendor source →

Neon Fundraise experienced a major incident on January 30, 2025 affecting Sprocket API and Donations and 1 more component, lasting 22h 24m. The incident has been resolved; the full update timeline is below.

Started
Jan 30, 2025, 07:18 PM UTC
Resolved
Jan 31, 2025, 05:42 PM UTC
Duration
22h 24m
Detected by Pingoru
Jan 30, 2025, 07:18 PM UTC

Affected components

Sprocket APIDonationsDNS <small><a href="http://status.aws.amazon.com/" target="_blank">status.aws.amazon.com</a></small>Rallybound Payment ServiceManagement interfacesCDN <small><a href="http://status.aws.amazon.com/" target="_blank">status.aws.amazon.com</a></small>Site BuilderEndurance Challenges Strava IntegrationRegistrationsSupport

Update timeline

  1. investigating Jan 30, 2025, 07:18 PM UTC

    We are currently investigating this issue.

  2. monitoring Jan 30, 2025, 07:54 PM UTC

    We were able to put measures in place to reduce load on the system. Sites and interfaces are now loading as normal. Team is still investigating root cause.

  3. monitoring Jan 30, 2025, 10:37 PM UTC

    We are continuing to see intermittent spikes in CPU which causes the system to be briefly unavailable for a minute or so at a time. The team is still investigating.

  4. resolved Jan 31, 2025, 05:42 PM UTC

    The system was cycled with last night's release, bringing performance back to normal levels.