Zonos experienced a critical incident on August 22, 2023 affecting Shopify Duty Tax and BigCommerce Duty Tax and 1 more component, lasting 37m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 22, 2023, 04:51 PM UTC
We are currently investigating this issue.
- identified Aug 22, 2023, 04:54 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Aug 22, 2023, 05:02 PM UTC
A fix has been implemented and we are monitoring the results.
- monitoring Aug 22, 2023, 05:02 PM UTC
We are continuing to monitor for any further issues.
- resolved Aug 22, 2023, 05:18 PM UTC
This incident has been resolved.
- postmortem Aug 22, 2023, 11:00 PM UTC
**What products were affected and what was the impact?** Our Landed Cost APIs \(REST and GraphQL\) were impacted. During the outage, all API requests were unauthorized. Impact: MAJOR OUTAGE **What timeframe did this issue occur?** | | **Date** | **Time** | | --- | --- | --- | | From: | August 15, 2023 | 10:40 MST | | To: | August 15, 2023 | 11:02 MST | **How was the issue detected?** We were immediately alerted by our monitoring system. **What functionality was affected?** No landed cost quotes could be returned due to authentication failures. **What problems did this cause?** wrong auth URL. used the discovery instead of the public. **What was the resolution of the problem and steps that are being taken for continued follow-up?** We discovered a configuration error in a previous release. We fixed the configuration error, released the fix, and confirmed issue resolution via manual and automated testing. **What mitigation solutions will we put in place to prevent this issue from occurring in the future?** We are improving documentation on environment configuration, and clarifying how to properly configure service discovery. We are also improving testing procedures to catch similar issues before they are introduced into production.