Parade Outage History

### Incident Summary Between **February 4 and February 6, 2026 (UTC)**, some customers experienced **intermittent service disruptions** affecting load posting and related workflows that rely on Truckstop syndication. During these periods, scheduled postings failed and downstream systems (including load visibility and carrier engagement) were temporarily unavailable. The issue occurred on an **hourly basis**, lasting approximately **5–10 minutes per occurrence**, and primarily impacted customers actively using Truckstop integrations. ### Customer Impact * Scheduled load postings to Truckstop and other loadboards failed or were delayed * Some loads were not visible to carriers during the affected windows * Downstream applications experienced temporary unavailability * A subset of customers may need to **re-authenticate their Truckstop accounts** due to token invalidation during the incident No data loss occurred. --- ## Timeline (UTC) **February 4, 2026** * **10:46 PM** – Truckstop begins returning unexpected errors for authentication refresh requests * **10:46–11:16 PM** – First reports of recurring, hourly service disruptions detected * **11:26 PM** – Preventative rollback performed on recent customer-facing changes to reduce system load * **11:30 PM** – Monitoring continued; impact assessed as limited overnight **February 5, 2026** * **10:00 AM** – Full investigation resumes * **10:40 AM** – Root cause identified as a failure pattern tied to Truckstop authentication refresh behavior * **11:00 AM** – Initial mitigation attempted by disabling Truckstop for affected accounts * **11:15 AM** – Issue persists * **11:28 AM** – Hourly scheduled posting job disabled to stop recurring outages * **12:15 PM** – System stability confirmed after disabling scheduled postings * **Afternoon–Evening** – Engineering work focused on a permanent fix * **10:05 PM** – Initial fixes deployed to staging * **11:04 PM** – Fixes validated in staging **February 6, 2026** * **12:18 AM** – Fixes deployed to production * **1:08 AM** – Scheduled postings re-enabled; no further outages observed --- ## Root Cause Two issues combined to cause this incident: 1. **External dependency failure** Truckstop experienced an issue that caused authentication refresh requests to fail unexpectedly. 2. **System behavior under failure conditions** When these authentication requests failed, our system retried them in a way that unintentionally created contention during peak scheduled activity. Under normal conditions this behavior is safe, but the external failure caused retries to stack up and temporarily block other operations. This combination turned a recoverable external error into a broader, time-boxed service disruption. --- ## Resolution ### Immediate Actions Taken * Disabled hourly scheduled posting jobs to immediately stop recurring disruptions * Reduced system load while isolating and validating the root cause * Coordinated internally and with Truckstop to confirm external failure conditions ### Permanent Fix * Updated our authentication handling to prevent system-wide contention during external API failures * Ensured external calls no longer block other operations during retries * Added additional safeguards so similar external issues cannot cascade into broader outages These changes were fully deployed and validated in production by **February 6, 2026, 01:08 AM UTC**. --- ## Customer Follow-Up Actions * **Some customers may need to re-authenticate their Truckstop accounts** due to token invalidation during the Truckstop outage * Our team will proactively reach out to affected customers with instructions and support * No action is required for customers who are not contacted --- ## Prevention Going Forward * Improved isolation between external partner failures and core system operations * Additional monitoring and alerting around authentication and scheduled job behavior * Stronger guardrails to ensure retries cannot impact overall system availability

This incident has been resolved.