Blackthorn incident

Partial Payments app outage - (Auth.net) Payment Gateway

Major Resolved View vendor source →

Blackthorn experienced a major incident on October 30, 2024 affecting Blackthorn Payments and Blackthorn PayLink and 1 more component, lasting 16h 44m. The incident has been resolved; the full update timeline is below.

Started
Oct 30, 2024, 08:20 PM UTC
Resolved
Oct 31, 2024, 01:05 PM UTC
Duration
16h 44m
Detected by Pingoru
Oct 30, 2024, 08:20 PM UTC

Affected components

Blackthorn PaymentsBlackthorn PayLinkBlackthorn DocumentLink

Update timeline

  1. investigating Oct 30, 2024, 08:20 PM UTC

    On 30th October, several customers using the Authorize.net payment gateway reported an issue with capturing payments. The issue was identified and reported to Authorize.net. They are investigating the issue and continue to monitor the situation. We are actively communicating with Authorization.net to get more information. We will continue to monitor the situation and sincerely apologize for any disruption to your business and customers. You can monitor our status page using the below link for reference: https://community.blackthorn.io/s/system-status The issue seems to be resolved for many customers, but we are still monitoring the reported cases. If you are still experiencing the issue, you may try the following steps: 1. Navigate to your Payment Gateway -> Click Connect Gateway. This will reconnect your payment gateway. 2. If the issue persists, regenerate the transaction API key in authorize.net and re-authorize the Payment Gateway once more in Salesforce. 3. If you are still experiencing the issue, please feel free to contact Blackthorn Support. Thank you!

  2. investigating Oct 30, 2024, 08:20 PM UTC

    We are continuing to investigate this issue.

  3. monitoring Oct 31, 2024, 12:10 AM UTC

    We are pleased to inform you that Authorize.net has resolved the issue and has implemented a fix to mitigate the issue that was affecting the oAuth service used to authenticate Application Interface (API) requests to Authorize.net. We will continue to monitor the situation actively and sincerely apologize for any disruption to your business and customers. You can monitor our status page using the below link for reference: https://community.blackthorn.io/s/system-status The issue seems to be resolved for many customers, but we are still monitoring the reported cases. If you are still experiencing the issue, you may try the following steps: 1. Navigate to your Payment Gateway -> Click Connect Gateway. This will reconnect your payment gateway. 2. If the issue persists, regenerate the transaction API key in authorize.net and re-authorize the Payment Gateway once more in Salesforce. 3. If you are still experiencing the issue, please feel free to contact Blackthorn Support.

  4. resolved Oct 31, 2024, 01:05 PM UTC

    The incident has been reportedly fixed. We are still monitoring for other occurrences.

  5. postmortem Nov 05, 2024, 12:05 AM UTC

    On October 30th, several customers using the [Authorize.net](http://authorize.net/) payment gateway reported an issue with capturing payments.The [Authorize.net](http://authorize.net/) team identified that a recent certificate change in the upstream process flow caused this issue. The change has been rolled back by their team. This issue is now resolved. The incident was related to [Authorize.net](http://authorize.net/) API Errors, where merchants encountered the error “E00I24—The provided access token is invalid.” To mitigate the impact on the gateway, the certificate migration was rolled back, and the application was restarted. We will continue to monitor the situation actively and sincerely apologize for any disruption to your business and customers. Thank you