Benevity Outage History
Benevity is up right nowPingoru has recorded 49 incidents for Benevity: 21 in the last 12 months and 5 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 1 | 140 of 720 (19%) |
| May 2026 | 0 | 0 of 744 (0%) |
| June 2026 | 3 | 145 of 720 (20%) |
| July 2026 | 2 | 5 of 744 (1%) |
| August 2026 | 3 | 100 of 744 (13%) |
| September 2026 | 1 | 1 of 720 (0%) |
| October 2026 | 0 | 0 of 509 (0%) |
Incidents, newest first
Minor September 10, 2026 Employees are unable to upload receipts for match requests 37m Donate and Volunteer Core Services
Started Sep 10, 2026, 06:11 PM UTC· resolved Sep 10, 2026, 06:49 PM UTC
- investigating Sep 10, 2026, 06:11 PM UTC
We're currently investigating an issue that is preventing employees from uploading receipts on Request a Match. This is affecting all clients. Our team is aware and is working diligently to resolve the issue as quickly as possible. We'll continue to provide updates here, so please subscribe to this page for the latest information.
- resolved Sep 10, 2026, 06:49 PM UTC
The issue affecting receipt uploads on Request a Match has been fully resolved, and our team has taken all necessary steps to prevent this from happening again in the future. We apologize for any inconvenience this may have caused.
Notice August 24, 2026 Grants Essential (formerly Benevity Grants) tenants may be experiencing some service degradation related to emails 3h 41m Grants Essential (formerly Benevity Grants)
Started Aug 24, 2026, 04:07 PM UTC· resolved Aug 24, 2026, 07:49 PM UTC
- investigating Aug 24, 2026, 04:07 PM UTC
Grants Essential (formerly Benevity Grants) tenants may be experiencing some service degradation related to emails .
- identified Aug 24, 2026, 04:24 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Aug 24, 2026, 06:11 PM UTC
A fix has been implemented and we are monitoring the results.
- monitoring Aug 24, 2026, 07:37 PM UTC
We are continuing to monitor for any further issues.
- monitoring Aug 24, 2026, 07:49 PM UTC
We are continuing to monitor for any further issues.
- resolved Aug 24, 2026, 07:49 PM UTC
This incident has been resolved.
Minor August 13, 2026 Versaic Program Overview Proposal Links Return 404 Not Found in Production 3d 23h Grants Advanced (formerly Versaic)
Started Aug 13, 2026, 04:49 PM UTC· resolved Aug 17, 2026, 04:02 PM UTC
- identified Aug 13, 2026, 04:49 PM UTC
We are currently experiencing degraded functionality for proposal lists accessed via My workspace > Program Overview within the Versaic Client Portal. Our team is actively investigating and working to restore normal behaviour. We will provide updates as more information becomes available.
- monitoring Aug 14, 2026, 02:57 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved Aug 17, 2026, 04:02 PM UTC
The issue affecting proposal lists in the Versaic Client Portal has been resolved. Normal functionality has been restored. We apologize for any disruption this caused.
Major August 6, 2026 Upon logging in via SSO, the system might incorrectly prompts the user for a password reset and their old password, blocking the login flow. 33m Donate and Volunteer Core Services
Started Aug 06, 2026, 05:05 PM UTC· resolved Aug 06, 2026, 05:39 PM UTC
- identified Aug 06, 2026, 05:05 PM UTC
Upon logging in via SSO, the system might incorrectly prompt the user for a password reset and their old password, blocking the login flow. The team has identified the issue and is working actively on the mitigation.
- monitoring Aug 06, 2026, 05:30 PM UTC
The issue has been mitigated and we are actively monitoring.
- resolved Aug 06, 2026, 05:39 PM UTC
Confirmed and communicated mitigation. Incident resolved.
Minor July 28, 2026 Match Assurance AI Validation Degradation 1h 55m Donate and Volunteer Core Services
Started Jul 28, 2026, 12:46 AM UTC· resolved Jul 28, 2026, 02:41 AM UTC
- identified Jul 28, 2026, 12:46 AM UTC
We have identified the cause of the issue affecting AI-powered validation of employee matching requests (Match Assurance) on the Benevity platform. Our team is working on a fix and we expect to have an update shortly.
- resolved Jul 28, 2026, 02:41 AM UTC
This incident has been resolved.
Minor July 6, 2026 External Match Request AI uploader 2h 49m Donate and Volunteer Core Services
Started Jul 06, 2026, 05:37 PM UTC· resolved Jul 06, 2026, 08:26 PM UTC
- investigating Jul 06, 2026, 05:37 PM UTC
We're investigating an issue causing file uploads to fail for money donations with AI-assisted matching enabled. We've temporarily disabled AI matching for these donations and reverted to standard processing while we work on a fix. No other donation types are affected.
- identified Jul 06, 2026, 06:23 PM UTC
The issue has been identified and a fix is being implemented.
- resolved Jul 06, 2026, 08:26 PM UTC
We've resolved the issue that caused file uploads to fail for money donations using AI-assisted matching. We've confirmed uploads are working as expected and restored AI matching for all clients.
Minor June 25, 2026 Grants Essential Degraded Performance 22h 48m Grants Essential (formerly Benevity Grants)
Started Jun 25, 2026, 09:39 PM UTC· resolved Jun 26, 2026, 08:28 PM UTC
- identified Jun 25, 2026, 09:39 PM UTC
We have identified an issue with Benevity Grants Essential that is causing degraded performance for users. The cause has been identified and we are actively working on a resolution.
- monitoring Jun 25, 2026, 09:53 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jun 26, 2026, 08:28 PM UTC
This incident has been resolved.
Major June 25, 2026 Grants Essential Performance Degraded 4h 1m Grants Essential (formerly Benevity Grants)
Started Jun 25, 2026, 03:35 PM UTC· resolved Jun 25, 2026, 07:36 PM UTC
- monitoring Jun 25, 2026, 03:35 PM UTC
Grants intermittent access issues occurred for a short period of time, are actively watching the issue at this time and still working on a root cause analysis.
- monitoring Jun 25, 2026, 03:37 PM UTC
We are continuing to monitor for any further issues.
- monitoring Jun 25, 2026, 03:50 PM UTC
Grants Essential is currently experiencing degraded performance for some users. We are investigating this issue and will provide updates here as we learn more.
- monitoring Jun 25, 2026, 04:26 PM UTC
We continue to see degraded performance for some users with Grants Essential. We continue to investigate / monitor this issue and will provide updates here as we learn more.
- monitoring Jun 25, 2026, 05:19 PM UTC
We continue to see degraded performance for some users with Grants Essential. We continue to investigate the issue and are taking actions to mitigate.
- monitoring Jun 25, 2026, 06:31 PM UTC
We are continuing to monitor for any further issues.
- monitoring Jun 25, 2026, 06:33 PM UTC
The issue has been identified and we are now monitoring to ensure there are no other issues or fallout.
- resolved Jun 25, 2026, 07:36 PM UTC
This incident has been resolved.
Minor June 17, 2026 Mailbox grid view proposal export experiencing issues 4d 22h Grants Advanced (formerly Versaic)
Started Jun 17, 2026, 04:20 PM UTC· resolved Jun 22, 2026, 02:51 PM UTC
- identified Jun 17, 2026, 04:20 PM UTC
We have identified an issue with the proposal exports from mailbox grid view that is currently affecting Grants Advanced clients using Cause search. Our team is working on a fix and we expect to release the update on Monday, June 22, 2026.
- identified Jun 17, 2026, 04:21 PM UTC
We are continuing to work on a fix for this issue.
- identified Jun 18, 2026, 05:25 PM UTC
A fix for this issue has been created and is pending release on Monday June 22nd, 2026
- identified Jun 22, 2026, 02:26 PM UTC
A fix has been published to the Versaic Prod environment. All exports are now available on the client portal.
- monitoring Jun 22, 2026, 02:38 PM UTC
A fix has been implemented and we are monitoring the results.
- monitoring Jun 22, 2026, 02:51 PM UTC
The issue affecting proposal exports from mailbox grid view for Grants Advanced clients using Cause search has been resolved. An update was released on Monday, June 22, 2026 and exports are now functioning as expected. We apologize for any disruption this caused.
- resolved Jun 22, 2026, 02:51 PM UTC
The issue affecting proposal exports from mailbox grid view for Grants Advanced clients using Cause search has been resolved. An update was released on Monday, June 22, 2026 and exports are now functioning as expected. We apologize for any disruption this caused.
Minor April 15, 2026 Alaya Email Services Unavailable 3h 36m Alaya by Benevity
Started Apr 15, 2026, 02:00 PM UTC· resolved Apr 15, 2026, 05:37 PM UTC
- investigating Apr 15, 2026, 02:00 PM UTC
We are currently investigating an issue in with Alaya emails. Certain platform functionality may not work as a result.
- identified Apr 15, 2026, 02:24 PM UTC
This issue has been identified and the teams are working to implement a fix.
- monitoring Apr 15, 2026, 05:04 PM UTC
The fix has been implemented and has been tested and confirmed as working. Email services in Alaya should now be operating as normal.
- resolved Apr 15, 2026, 05:37 PM UTC
This incident has now been resolved.
Minor March 31, 2026 Versaic Production site experiencing slow performance 5d 20h Grants Advanced (formerly Versaic)
Started Mar 31, 2026, 07:36 PM UTC· resolved Apr 06, 2026, 04:10 PM UTC
- investigating Mar 31, 2026, 07:36 PM UTC
We're currently investigating this issue and will have more details soon. Please subscribe to this page for the latest updates.
- investigating Mar 31, 2026, 08:02 PM UTC
We are currently experiencing performance issues with the Versaic Admin Portal. Applicant sites are also impacted. Some users may encounter slow page load times or intermittent timeouts. Our team is actively investigating and working to restore normal performance. We will provide updates as more information becomes available.
- monitoring Mar 31, 2026, 09:37 PM UTC
We have identified an issue and have deployed a mitigation. We will continue to monitor the situation but all services are now restored.
- resolved Apr 06, 2026, 04:10 PM UTC
This incident has been resolved.
Major March 18, 2026 Causes Portal - Okta Login and Password Reset Issues 2d 2h Benevity Causes and Nonprofits Portal
Started Mar 18, 2026, 07:06 PM UTC· resolved Mar 20, 2026, 09:52 PM UTC
- investigating Mar 18, 2026, 07:06 PM UTC
Our team is investigating an issue with Okta SSO logins and password resets for the Causes Portal. We're working to resolve this as quickly as possible; please subscribe to this page for the latest updates.
- monitoring Mar 18, 2026, 07:54 PM UTC
We’ve identified the cause of the Causes Portal outage and have taken steps to stabilize the platform. User workflows are currently functional, and we are continuing to monitor the situation closely to ensure full resolution.
- resolved Mar 20, 2026, 09:52 PM UTC
The incident has been resolved. We apologize for any inconvenience this may have caused.
Critical February 10, 2026 Donate and Volunteering intermittently not loading 1h 28m Donate and Volunteer Core Services
Started Feb 10, 2026, 09:22 PM UTC· resolved Feb 10, 2026, 10:50 PM UTC
- investigating Feb 10, 2026, 09:22 PM UTC
We are experiencing intermittent outages across all sites. This is being investigated with urgency, and we will update the status page when more information is available.
- identified Feb 10, 2026, 09:34 PM UTC
Our cloud provider is experiencing an Incident, impacting all of our client sites intermittently. We are awaiting further updates from our provider, and are monitoring closely.
- identified Feb 10, 2026, 10:04 PM UTC
Our cloud provider's services are recovering, and site stability is restoring. Users may still see intermittent issues until a full recovery is achieved by the provider - we will continue to monitor until full resolution is achieved.
- resolved Feb 10, 2026, 10:50 PM UTC
Our cloud provider's services have fully recovered, and our sites are fully operational. Thank you for your patience and understanding.
Notice February 9, 2026 Versaic: Direct Page Navigation Feature Temporarily Disabled 9d 1h Grants Advanced (formerly Versaic)
Started Feb 09, 2026, 09:58 PM UTC· resolved Feb 18, 2026, 11:24 PM UTC
- investigating Feb 09, 2026, 09:58 PM UTC
We identified an issue affecting the Direct Page Navigation feature, which allows users to navigate between form pages by clicking page names at the top of the screen. While we work on a fix, we have temporarily disabled the Direct Page Navigation feature. Users can continue to navigate forms using the "Next" and "Previous" buttons without any issues. We apologize for any inconvenience and appreciate your patience.
- investigating Feb 09, 2026, 10:17 PM UTC
Action Required: If any proposals were submitted using the Direct Page Navigation feature and appear to have missing information, please release those proposals for editing so submitters can re-enter their responses.
- identified Feb 12, 2026, 06:13 PM UTC
The issue has been identified and a fix is being implemented.
- resolved Feb 18, 2026, 11:24 PM UTC
This incident has been resolved.
Major February 2, 2026 Forms are not populating computed fields (Smart Expressions) 21h 41m Grants Advanced (formerly Versaic)
Started Feb 02, 2026, 05:55 PM UTC· resolved Feb 03, 2026, 03:36 PM UTC
- identified Feb 02, 2026, 05:55 PM UTC
On February 2nd we’ve identified that forms were not populating computed smart expressions values. Our team is actively implementing a solution to restore full functionality. We appreciate your patience while we work on this. More updates will follow as they become available.
- monitoring Feb 02, 2026, 10:02 PM UTC
The issue affecting forms not populating computed values properly has been resolved, and we are monitoring the issue to ensure quality and stability. If you continue to experience issues, please submit a support request. Thank you for your understanding.
- resolved Feb 03, 2026, 03:36 PM UTC
The issue affecting forms not populating computed values properly has been fully resolved, and our team has taken all necessary steps to prevent this from happening again in the future. We apologize for any inconvenience this may have caused.
Minor February 1, 2026 MFT Folder Clean Up Impacted Payroll Files for Some Clients –
Started Feb 01, 2026, 11:22 PM UTC· resolved Feb 01, 2026, 11:22 PM UTC
- resolved Feb 01, 2026, 11:22 PM UTC
During a routine administrative cleanup of inactive user accounts on the managed file transfer system, an unintended cascading deletion occurred that impacted some payroll files for some clients.
- postmortem Feb 01, 2026, 11:22 PM UTC
## Summary Between December 15 and December 17, 2025, an incident occurred on our Managed File Transfer \(MFT\) system that resulted in the accidental deletion of critical payroll folder structures. This impacted two primary workflows: the receipt of Success Files from clients and the delivery of Deduction Files to clients. While most data was restored via backups, a small window of data was unrecoverable, and some deliveries were delayed. ## Impact The incident created two distinct challenges for our clients: * Issue 1: Payroll Success Files - Due to a gap between the deletion and our last system backup, 28 files uploaded by clients between 12:08 AM and 2:51 PM on December 15 were lost. These clients were contacted directly to resubmit their data. * Issue 2: Payroll Deduction Files - The deletion prevented the system from automatically generating deduction files for some clients. By December 17, our teams manually recovered and redelivered all affected files to ensure client payroll deadlines were met. ## Root Cause During a routine administrative cleanup of inactive user accounts on the MFT system, an unintended cascading deletion occurred. The investigation revealed a hidden dependency: when specific administrative user profiles were removed, the system automatically deleted the directory structures associated with those profiles. Unfortunately, these directories housed critical payroll folders. Because this "link" between the user account and the production folder was not explicitly flagged by the system, the deletion was not caught until automated processing errors were reported. ## Future Mitigation * Increased snapshot frequency for critical payroll directories to reduce the recovery window. * All account cleanups now require a peer-review protocol and a dependency check. * Implement monitoring and alerting for bulk file/folder deletions in payroll directories. ## Timeline of Events * **Dec 15, 12:08 AM:** Accidental deletion occurs during routine account maintenance. * **Dec 16, 9:15 AM:** Incident detected and response teams engaged. * **Dec 16, 2:30 PM:** Core folder structure restored from snapshots; 15-hour data gap identified. * **Dec 17, 12:52 PM:** Deduction file folders restored; manual re-generation of files begins. * **Dec 17, 6:00 PM:** All 67 missing deduction files successfully delivered to clients. * **Dec 18 – Jan 20:** Direct outreach to the affected clients to coordinate success file resubmissions. * **Jan 25, 2026:** Incident officially closed following confirmation of all data reprocessing.
Minor December 11, 2025 Grants Advanced Mail Outage 10m Grants Advanced (formerly Versaic)
Started Dec 11, 2025, 11:27 PM UTC· resolved Dec 11, 2025, 11:38 PM UTC
- investigating Dec 11, 2025, 11:27 PM UTC
There was an outage in mail delivery due to maintenance in our systems.
- investigating Dec 11, 2025, 11:38 PM UTC
All mail that was delayed during the outage has now been sent.
- resolved Dec 11, 2025, 11:38 PM UTC
This incident has been resolved.
Notice November 18, 2025 Authentication errors for some API Clients 22h 52m Benevity Platform
Started Nov 18, 2025, 06:15 PM UTC· resolved Nov 19, 2025, 05:07 PM UTC
- monitoring Nov 18, 2025, 06:15 PM UTC
Between roughly 07:45 and 09:00 MT a Cloudflare outage affected Auth0, which caused some API clients to receive 4xx errors during authentication, and were therefore unable to authenticate and use the APIs. All systems are now working correctly, and we will continue to monitor.
- resolved Nov 19, 2025, 05:07 PM UTC
This incident has been resolved.
Major November 18, 2025 Payment processor outage 9h 54m Donate and Volunteer Core Services
Started Nov 18, 2025, 01:34 PM UTC· resolved Nov 18, 2025, 11:28 PM UTC
- monitoring Nov 18, 2025, 01:34 PM UTC
We are currently investigating an issue affecting credit card transactions. Some customers may experience errors when attempting to donate or set up recurring donations. We have identified the issue originating with one of our upstream payment processing partners. Our team is actively monitoring the situation and working on a resolution to restore full service as quickly as possible.
- monitoring Nov 18, 2025, 01:58 PM UTC
We have identified errors regarding credit card transactions, caused by an outage by one of your upstream partners. We are actively working on technical migrations to re-route payment traffic and restore service to impacted user, while continuing to monitor the external outage.
- monitoring Nov 18, 2025, 03:02 PM UTC
Our upstream payment provider has started to recover, and we are actively seeing signs of recovery within our own systems. However, we will continue to monitor all transaction flows closely until we can ensure that everything is fully back online and stable.
- monitoring Nov 18, 2025, 04:13 PM UTC
To improve payment reliability while we work through ongoing processing issues, we’ve temporarily disabled Apple Pay and saved credit card functionality. Users attempting to make a payment, will need to manually re-enter their credit card details at checkout. We’re closely monitoring the situation and will restore these features as soon as we can ensure system stability.
- monitoring Nov 18, 2025, 11:19 PM UTC
All credit card functionality has now been restored. We will continue to monitor the situation to ensure performance remains high.
- resolved Nov 18, 2025, 11:28 PM UTC
This incident is now resolved.
Minor October 29, 2025 Inconsistent load times for Volunteering pages within our Employee Engagement product 4h 18m Donate and Volunteer Core Services, Grants Advanced (formerly Versaic), Benevity Causes and Nonprofits Portal
Started Oct 29, 2025, 05:26 PM UTC· resolved Oct 29, 2025, 09:44 PM UTC
- investigating Oct 29, 2025, 05:26 PM UTC
We are experiencing inconsistent load times for Volunteering pages within our Employee Engagement product. We are currently investigating the issue.
- identified Oct 29, 2025, 05:48 PM UTC
The issue has been identified, we are working to mitigate the issue and restore service.
- identified Oct 29, 2025, 06:52 PM UTC
We are observing an improvement in service as the underlying cause is remediated. We continue to investigate impact.
- monitoring Oct 29, 2025, 07:09 PM UTC
Impacted services have returned to normal availability. We continue to monitor.
- resolved Oct 29, 2025, 09:44 PM UTC
This incident has been resolved.
Major October 20, 2025 SEV 1 Benevity Ecosystem Service Disruption 1d 4h Donate and Volunteer Core Services, Grants Essential (formerly Benevity Grants), Donate and Volunteer (formerly Benevity Spark), Community Investment - Essential (formerly Benevity Grants) and 11 more
Started Oct 20, 2025, 11:36 AM UTC· resolved Oct 21, 2025, 03:50 PM UTC
- monitoring Oct 20, 2025, 11:36 AM UTC
Due to a service disruption with a key infrastructure provider all Benevity Services are in a degraded or disrupted state. Recovery is underway and services are recovering. Next update in 1 hour.
- monitoring Oct 20, 2025, 12:01 PM UTC
We are continuing to monitor for any further issues.
- monitoring Oct 20, 2025, 12:42 PM UTC
Everything seems to be working as expected but we are continuing to monitor all of our services and assessing the impact of this disruption
- monitoring Oct 20, 2025, 03:06 PM UTC
All services have been restored and are operating normally. Our teams continue to monitor the systems closely as we conduct a full assessment of the disruption's impact.
- monitoring Oct 20, 2025, 04:18 PM UTC
We are still impacted by an ongoing incident with our primary infrastructure provider. Our team is closely monitoring their status and will post updates as they become available.
- monitoring Oct 20, 2025, 05:52 PM UTC
Benevity services are recovering and are stable, and we do not expect any significant impact to end users at this time. Our team continues to monitor the situation closely and is working with the infrastructure provider to ensure full restoration.
- monitoring Oct 20, 2025, 08:34 PM UTC
Most Benevity services have recovered and are functioning as expected. We are continuing to monitor minor degradation in a few areas and are working with our infrastructure provider to ensure full restoration.
- monitoring Oct 21, 2025, 12:05 AM UTC
The majority of Benevity services have recovered and are functioning as expected. We are continuing to monitor minor degradation in a few areas and are working with our service providers to ensure full restoration.
- resolved Oct 21, 2025, 03:50 PM UTC
This incident has been resolved.
- postmortem Nov 03, 2025, 02:39 AM UTC
## Summary Between October 20, 2025, 1:11 AM MT and 4:01 PM MT, Benevity's key infrastructure provider experienced a significant outage in the US-EAST-1 region. This outage cascaded into widespread disruptions across this and other providers and Benevity services. The incident was triggered by DNS resolution failures within the key infrastructure providers systems, which subsequently caused impairments in the launch capabilities of virtual machines and Network Load Balancer health checks. These issues thereafter affected a wide range of services provided by our key infrastructure provider that Benevity services depends on. The outage affected the following key Benevity functionalities: * Payment processing * Giving and volunteering operations \(Matching Offers, Giving Opportunity and Volunteer Opportunity creation with reporting tags\) * Non-profit related systems. * User management and authentication * Search and reporting capabilities * Email notifications and communications * Various backend integrations and data processing jobs, including disbursement of funds. During the incident, our teams actively managed the situation by successfully failing over credit card donation processing to our backup system and proactively pausing donation disbursement functionality to ensure integrity. As services returned, our team managed the backlog and executed retry measures to remediate the impact. While many workflows continued functioning, some end-user experiences were affected during the outage window: * Donation attempts experienced disruptions in select cases * Some automated match requests did not receive auto-approval and required manual review * Some email communications experienced delays or failures that were subsequently remediated * Administrative functions like giving and volunteer opportunity management, and search capabilities experienced degradation. By end of business October 20th, the vast majority of systems were operational. October 21st confirmed full recovery with no outstanding issues. ## Impact During the incident, Benevity experienced service interruptions across several core systems as a result of a widespread outage from our primary infrastructure provider. The failure caused temporary throttling of system resources and delays in launching new compute environments, which in turn affected the performance and availability of key Benevity services between 00:50 MT and 22:58 MT on October 20, 2025. **Giving and Transactional Systems** were impacted early in the event, with credit card donations temporarily unavailable for approximately 90 minutes before successfully failing over to a stable backup flow. While a small number of donation attempts failed during this period, Benevity has verified that all processed transactions were reconciled and that data integrity was fully maintained. Disbursement processing was proactively paused later in the morning to ensure consistency, resulting in a one-day delay in initiating batch disbursements. **API and Administrative Tools** experienced intermittent unavailability during the same window. Clients integrating via Benevity APIs encountered elevated error rates, and administrators faced interruptions when managing Giving and Volunteering Opportunities. Temporary failures in tag management and match request automation required limited rework once systems stabilized. Benevity engaged directly with affected clients to identify and address any incomplete records or pending approvals. **Automated Data and Processing Systems** experienced delays in sending notifications, importing payroll and user data, and generating scheduled reports. All impacted processes were retried and successfully completed later in the day, with no data loss or long-term effects. **Nonprofit Systems** also saw delays in background communications and validation workflows for newly nominated and onboarded nonprofits. These queued processes automatically resumed once infrastructure services recovered. Overall, while several client-facing and back-end services were temporarily unavailable or delayed throughout the day, Benevity’s data integrity remained intact, all queued operations completed successfully once stability was restored, and all client transactions and records were validated post-recovery. ## Root Cause The widespread degradation and disruption of Benevity services stemmed from a major external failure originating with Benevity's key infrastructure provider in its primary operating region. This event was categorized as a Severity 1 \(SEV1\) incident due to its sweeping impact across multiple critical applications. The Core Technical Failure \(Infrastructure Provider\) The infrastructure provider confirmed that the incident was triggered by fundamental networking issues within their platform, specifically related to DNS resolution issues for core data service endpoints. This initial failure led to cascading problems that impacted internal subsystems responsible for managing fundamental cloud resources: * Connectivity issues immediately halted the operation of critical serverless applications and disturbed access to centralized data storage services. * As the provider attempted recovery, they were forced to throttle \(rate limit\) the launch of new virtual servers. This throttling prevented essential scheduled tasks and automated workflows from executing normally. * This outage was not isolated to Benevity; multiple critical business partners and tools relied upon for operations were also degraded by the same infrastructure failure. ## Future Mitigation Benevity views the recent service disruption as a critical learning opportunity to further reinforce our systems against external infrastructure volatility. Our future mitigation strategy centres on three key areas designed to maximize stability, data accuracy, and operational response, ensuring greater resilience for clients. We are implementing improvements to ensure core systems remain stable, even if external services they depend on experience outages. This includes but is not limited to improved operating procedures to disable capabilities, payment system hardening, matching offers graceful degradation, automated email retry, reporting tag loss prevention, vendor accountability, improvements to observability, updated post incident playbooks for service validation. ## Timeline of Events * October 20, 2025 - 00:50 MT - Benevity Incident automatically detected and alerted through monitoring systems. * October 20, 2025 - 01:11 MT - Key infrastructure provider system report increased error rates & latencies. * October 20, 2025 - 01:34 MT - Internal incident formally declared. * October 20, 2025 - 02:28 MT - Credit card donation functionality failed over to backup system. * October 20, 2025 - 02:28 MT - Major Contributing Cause confirmed as key infrastructure provider outage. * October 20, 2025 - 03:22 MT - Key Infrastructure provider reports initial mitigation and early signs of recovery * October 20, 2025 - 03:22 MT - Benevity services start reporting healthy checks, we continue to monitor. * October 20, 2025 - 10:33 MT - Donation Disbursement functionality disabled till reliable service is available. * October 20, 2025 - 14:41 MT - Most Benevity services have recovered and are functioning as expected. * October 20, 2025 - 15:29 MT - Previously failed Spark email notifications are sent. * October 20, 2025 - 17:00 MT - Key Infrastructure provider reporting all by services are resolved. * October 20, 2025 - 18:00 MT - Benevity holds monitoring status for the evening while our providers' systems heal. * October 21, 2025 - 09:24 MT - All Benevity systems are reported to have return to operational status.
Minor October 2, 2025 Canada Post Service Disruption 63d Benevity Global Disbursements
Started Oct 02, 2025, 10:38 PM UTC· resolved Dec 04, 2025, 11:30 PM UTC
- identified Oct 02, 2025, 10:38 PM UTC
As of September 25, 2025, Canada Post announced a service disruption affecting mail, including checks mailed from Canada to Canadian non-profits. To minimize disbursement delays, impacted nonprofits are encouraged to switch to EFT. We recognize the importance of timely funding, especially during Giving Season. We will continue to monitor the situation and provide updates until Canada Post mail service resumes. For more information, please reach out to your Benevity Client Success Manager.
- identified Oct 15, 2025, 04:53 AM UTC
As of October 11, 2025, Canada Post has transitioned to a nationwide rotating strike. Mail pickup and delivery have resumed but delays are expected during this period. We continue to encourage nonprofits to switch to EFT to help minimize disbursement delays. Benevity will monitor the situation closely and provide further updates as needed. For more information, please reach out to your Benevity Client Success Manager.
- resolved Dec 04, 2025, 11:30 PM UTC
Canada Post has resumed regular service and mail timelines are back to normal.
Major October 2, 2025 Credit card transactions are failing to be processed for our Online Giving Foundation in India (OGF-I) 1d 1h Donate and Volunteer Core Services
Started Oct 02, 2025, 02:59 PM UTC· resolved Oct 03, 2025, 04:52 PM UTC
- investigating Oct 02, 2025, 02:59 PM UTC
We are currently experiencing issues with processing local credit card transactions in India through the Online Giving Foundation. Our teams are actively investigating the cause and working to resolve this as quickly as possible. We will provide further updates as more information becomes available.
- identified Oct 02, 2025, 05:29 PM UTC
We have identified the issue affecting local credit card transactions in India through the Online Giving Foundation. Currently, all transactions are failing due to a 3DS authentication issue. Our credit card processor is actively working to resolve the issue, and we will continue to provide updates as progress is made.
- identified Oct 02, 2025, 10:48 PM UTC
Our credit provider is currently implementing a fix to resolve the 3DS authentication issue. We will continue to monitor the situation and provide updates as progress is made
- monitoring Oct 03, 2025, 02:56 PM UTC
The 3DS authentication fix has been successfully implemented, and payment acceptance rates have returned to normal levels. We will continue to closely monitor the issue to ensure ongoing stability.
- resolved Oct 03, 2025, 04:52 PM UTC
Credit card transactions are now processing normally through the Online Giving Foundation in India. This issue has been resolved. Thank you for your patience
Major September 26, 2025 Spark donation form is down 28m Donate and Volunteer Core Services
Started Sep 26, 2025, 03:40 PM UTC· resolved Sep 26, 2025, 04:08 PM UTC
- investigating Sep 26, 2025, 03:40 PM UTC
Spark's donation form is currently down and users are experiencing errors when attempting to donate. We are investigating the issue.
- monitoring Sep 26, 2025, 03:56 PM UTC
A fix has been implemented and donation processing is working again. We are continuing to monitor for any adverse affects.
- resolved Sep 26, 2025, 04:08 PM UTC
This incident has been resolved.
- postmortem Oct 05, 2025, 09:41 PM UTC
## Summary On Friday, September 26, between 9:03 AM and 9:50 AM MT, users in Spark \(Donate and Volunteer\) encountered error messages when attempting to access the Donation Form and Donation History pages. During the same window, a related service that supports volunteering functionality was also unable to communicate with necessary backend services. ## Impact During the incident window, users were unable to access certain giving functionality within Spark, including the Donation Form and Donation History pages. This interruption prevented some users from completing or viewing donations and temporarily affected engagement with giving and volunteering activities. ## Root Cause An infrastructure update related to networking security configurations inadvertently resulted in our Payments processing service to become unreachable by the Spark application, causing end-users to experience an error page when trying to access the Donation Form. ## Future Mitigation We are reviewing our Engineering and Quality Assurance practices around infrastructure updates of this kind to prevent such incidents in the future. This includes enhancing our change management processes, improving validation and testing procedures, and strengthening peer review requirements for configuration updates. These efforts are aimed at reducing the likelihood of similar disruptions, increasing visibility into service dependencies, and improving the overall resilience of our production environment. ## Timeline of Events * Sept 26 @ 9:00am MT - Infrastructure update is made leading to the errors on the Donation Form. * Sept 26 @ 9:04am MT - Monitoring systems detect the service is down and page out to Engineering and SRE teams. * Sept 26 @ 9:45am MT - Root cause is identified. * Sept 26 @ 9:46am MT - Fix is put in place and service is restored.
Major September 25, 2025 Grants Essentials Unavailable for Some Clients –
Started Sep 25, 2025, 02:00 AM UTC· resolved Sep 25, 2025, 02:00 AM UTC
- resolved Sep 25, 2025, 07:15 PM UTC
From approximately 4:07pm MT to 7:53pm MT on September 24, 2025, Benevity Grants Essentials was unavailable to some clients that had a specific SSO configuration.
- postmortem Oct 01, 2025, 02:49 AM UTC
## Summary On Wednesday, September 24, 2025, clients using Auth0 Single Sign-On \(SSO\) to access the grants portal experienced a login interruption between approximately 7:06 PM and 7:49 PM Mountain Time. The interruption occurred due to an internal system update that unintentionally overwrote the correct login configurations. As a result, login requests from identity providers were not recognized, producing a "Callback URL mismatch" error message for users. Our technical team identified the issue and restored the correct settings, after which services returned to normal. The system has remained stable since the resolution. ## Impact Clients using Auth0 SSO were unable to log in to the portal during the disruption window. ## Root Cause An internal system update inadvertently replaced the correct SSO login configurations. This caused login requests from identity providers to be rejected, resulting in the "Callback URL mismatch" error message. ## Future Mitigation To reduce the risk of similar issues and strengthen platform reliability, we are implementing the following measures: * **Enhanced Monitoring & Alerting:** Improved real-time monitoring and alerting on login systems to detect and address anomalies more quickly, often before they affect clients. * **Unified Configuration Management:** Consolidation of system configurations into a single, automated source of truth. This prevents manual changes or outdated settings from being applied during updates, eliminating the root cause of this incident. ## Timeline of Events September 24th, 2025 * 19:06 MT - Login issues were identified and escalated to technical support to investigate. * 19:36 MT A severity one incident was created and investigation into the issue the issue began. * 19:50 MT - The root cause of the issue was identified. * 19:53 MT - A fixed was applied to restore access to all Auth0 users. The incident was resolved and system was fully operable.