Spreedly incident

Core Transactional API Errors

Minor Resolved View vendor source →

Spreedly experienced a minor incident on December 12, 2024 affecting Core Transactional API, lasting 1h 21m. The incident has been resolved; the full update timeline is below.

Started
Dec 12, 2024, 06:04 PM UTC
Resolved
Dec 12, 2024, 07:25 PM UTC
Duration
1h 21m
Detected by Pingoru
Dec 12, 2024, 06:04 PM UTC

Affected components

Core Transactional API

Update timeline

  1. investigating Dec 12, 2024, 06:04 PM UTC

    Spreedly has discovered an issue that could potentially impact new payment method creation. We are taking all necessary measures to investigate the matter and determine the extent of the situation. We will provide an update as soon as more details are available.

  2. monitoring Dec 12, 2024, 06:32 PM UTC

    From 12:47 to 12:58 PM EST, we encountered an issue that affected customers attempting to tokenize a payment method and transacting with a payment method, resulting in a 500 error. We have identified the problem and implemented a fix. Our team is closely monitoring the situation to ensure there are no further impacts. If you need additional assistance, please reach out to our Support team.

  3. resolved Dec 12, 2024, 07:25 PM UTC

    After closely monitoring and confirming that all systems are stabilized and functioning as expected, this incident is considered resolved. No further customer impact is expected. We apologize for any inconvenience or disruption.

  4. postmortem Dec 17, 2024, 10:59 PM UTC

    # December 12th, 2024 — Payment Method Tokenization Errors Server maintenance to reduce log volume caused an internal service to become unavailable, causing payment method tokenization requests to fail. ## What Happened At approximately 17:45 UTC on December 12th, 2024, a logging configuration change made to reduce log volume resulted in an internal service becoming unavailable. Traffic was shifted to an alternate service during the outage window to reduce impact. At approximately 18:01 the internal service resumed normal operation and service was restored. Payment transactions were negatively impacted during this time. ## Next Steps Spreedly Engineering is investigating the coupling of the configuration change to the internal service and will be taking steps to prevent the issue from occuring in the future.