Orderful incident

Transaction Processing Delays.

Major Resolved View vendor source →

Orderful experienced a major incident on September 18, 2023 affecting API, lasting 8h 6m. The incident has been resolved; the full update timeline is below.

Started
Sep 18, 2023, 06:50 PM UTC
Resolved
Sep 19, 2023, 02:57 AM UTC
Duration
8h 6m
Detected by Pingoru
Sep 18, 2023, 06:50 PM UTC

Affected components

API

Update timeline

  1. investigating Sep 18, 2023, 06:50 PM UTC

    Currently we're seeing delays processing transactions. At this time ingestion doesn't seem to be affected.

  2. investigating Sep 18, 2023, 07:12 PM UTC

    Currently, we're seeing delays in processing transactions. We are still able to receive data Via API, As2, SFTP/FTP, and HTTP

  3. investigating Sep 18, 2023, 07:45 PM UTC

    We have identified the issue and are working with our third-party vendor. We're still experiencing delays in processing transactions. We can still receive data Via API, As2, SFTP/FTP, and HTTP.

  4. monitoring Sep 18, 2023, 08:22 PM UTC

    We are seeing partial restoration of transaction processing services. The backlog is currently being processed and we are working to increase capacity to process it quicker.

  5. monitoring Sep 18, 2023, 08:58 PM UTC

    We continue to see partial restoration of transaction processing services. The backlog is being processed, and we are working to increase capacity to process it quicker.

  6. monitoring Sep 18, 2023, 09:57 PM UTC

    We continue to see partial restoration of transaction processing services. The backlog is being processed, and we are working to increase capacity to process it quicker.

  7. monitoring Sep 19, 2023, 12:18 AM UTC

    Currently, we're seeing Transactions in "Processing". We are working on getting that fixed We are still able to receive data Via API, As2, SFTP/FTP, and HTTP

  8. monitoring Sep 19, 2023, 12:51 AM UTC

    We are still working on getting a fix for Transactions in "Processing". We are still able to receive data Via API, As2, SFTP/FTP, and HTTP

  9. monitoring Sep 19, 2023, 01:35 AM UTC

    We have failed over to a Backup infrastructure, and we are currently processing our backlog of Transactions. While we are waiting for the Third-party provider Systems to be fully restored. We are still able to receive data Via API, As2, SFTP/FTP, and HTTP

  10. monitoring Sep 19, 2023, 02:07 AM UTC

    The backlog of Transaction are still being processed. ETA of Completion is 30 mins

  11. monitoring Sep 19, 2023, 02:22 AM UTC

    The entire backlog of transactions has been processed. All new transactions are flowing as expected.

  12. resolved Sep 19, 2023, 02:57 AM UTC

    The issue has been resolved.