SPS Commerce experienced a major incident on October 20, 2025 affecting Oracle Transportation Management and Document Processing and 1 more component, lasting 16h 6m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Oct 20, 2025, 09:54 AM UTC
We are investigating an open incident with our document processing infrastructure which is potentially causing delays or failures for transactions processing through the system. We will reconcile all data after we have recovered from the incident to ensure all transactions are processed as expected.
- identified Oct 20, 2025, 09:55 AM UTC
We are continuing to work on a fix for this issue.
- identified Oct 20, 2025, 10:23 AM UTC
We are currently investigating an issue impacting multiple platform services, including backend systems and customer-facing applications. The disruption is linked to a broader incident with one of our cloud providers and may result in degraded performance, intermittent errors, or temporary delays across portions of our platform. All data will be reconciled and fully processed once services have stabilized. Our teams continue to monitor the situation closely and will provide updates as recovery progresses.
- identified Oct 20, 2025, 10:25 AM UTC
We are continuing to work on a fix for this issue.
- identified Oct 20, 2025, 10:35 AM UTC
We are currently investigating an issue impacting multiple platform services, including backend systems and customer-facing applications. The disruption is linked to a broader incident with one of our cloud providers and may result in degraded performance, intermittent errors, or temporary delays across portions of our platform. All data will be reconciled and fully processed once services have stabilized. Our teams continue to monitor the situation closely and will provide updates as recovery progresses.
- identified Oct 20, 2025, 11:07 AM UTC
All current inbound and outbound dataflow is processing normally. Documents that were in process during the incident are being recovered. Fulfillment Monitor visibility remains impacted and is actively being investigated.
- identified Oct 20, 2025, 11:18 AM UTC
All current inbound and outbound dataflow is processing normally. Documents that were in process during the incident are being recovered. Fulfillment Monitor visibility remains impacted and is actively being investigated.
- identified Oct 20, 2025, 11:19 AM UTC
All current inbound and outbound dataflow is processing normally. Documents that were in process during the incident are being recovered. Fulfillment Monitor visibility remains impacted and is actively being investigated.
- monitoring Oct 20, 2025, 11:37 AM UTC
All current inbound and outbound dataflow is processing normally. Documents that were in process during the incident are being recovered.
- monitoring Oct 20, 2025, 01:31 PM UTC
All current inbound and outbound dataflow continues to process normally. Recovery of affected documents is continuing as expected, and no new impact has been observed.
- monitoring Oct 20, 2025, 03:32 PM UTC
Data processing remains stable, and reconciliation work is nearing completion. The incident will remain in monitoring while teams continue to observe and investigate higher-than-normal error rates.
- monitoring Oct 20, 2025, 04:47 PM UTC
We are currently observing increased error rates affecting a subset of data processing. Most data continues to process successfully, but we’re seeing intermittent delays in the subset of data affected. The incident remains in active monitoring while mitigation and recovery work continues.
- identified Oct 20, 2025, 05:02 PM UTC
Error rates with our primary service provider have increased, resulting in broader impact to processing. We are reverting the incident status to Identified, response teams remain engaged focusing on mitigation and recovery. Updates will be shared as conditions evolve.
- identified Oct 20, 2025, 06:24 PM UTC
Our teams remain actively engaged on this issue and continue to work toward recovery. Error rates remain elevated, and we’re monitoring system performance closely. We’ll provide another update as soon as we have additional information to share.
- identified Oct 20, 2025, 07:54 PM UTC
Our teams remain actively engaged and continue to monitor system performance closely. Error rates are still elevated, and mitigation work is ongoing. We’ll provide further updates as we confirm sustained improvement.
- identified Oct 20, 2025, 08:58 PM UTC
Error rates have improved and document processing has resumed across affected systems. We are actively working through a backlog of transactions and continue to monitor performance to confirm sustained stability. We'll provide further updates as we process through the backlog.
- identified Oct 20, 2025, 10:28 PM UTC
Error rates remain stable following earlier improvements. Processing continues as our teams work through the backlog of transactions and monitor system performance to confirm sustained recovery.
- monitoring Oct 20, 2025, 11:38 PM UTC
Error rates have normalized, and all systems are now processing data as expected. We’ve transitioned this incident to monitoring while our teams verify stability and reconcile any transactions impacted during the earlier disruption.
- monitoring Oct 21, 2025, 12:03 AM UTC
All systems are currently operating as expected. We continue working to reconcile and verify documents impacted during this incident. Updates will be shared as we complete recovery activities.
- resolved Oct 21, 2025, 02:00 AM UTC
All systems are currently operating as expected. We continue working to reconcile and verify documents impacted during this incident.