POS Portal experienced a minor incident on November 18, 2020 affecting POS Portal API, lasting 4h 50m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 18, 2020, 01:32 PM UTC
We are currently investigating this issue.
- identified Nov 18, 2020, 03:56 PM UTC
The issue has been identified. The team is working on a solution. The next update will provide an ETA for resolution.
- monitoring Nov 18, 2020, 04:28 PM UTC
The team has implemented a solution and is now monitoring.
- resolved Nov 18, 2020, 06:23 PM UTC
This incident has been resolved.
- postmortem Nov 20, 2020, 02:55 PM UTC
POS Portal deployed a Software Release to its Enterprise Resource Planning \(ERP\) platform on the evening of Tuesday, November 17th as part of its standard monthly release process. A feature in this deployment effected order placement for users the next business day, Wednesday, November 18th. Early Wednesday, November 18th, POS Portal identified an incident in the Release which effected order creation. POS Portal immediately began investigating the incident, and had the functionality fully resolved on November 18th at 8:30amPT. Outage Window: Tuesday November 17th 9pm PT to Wednesday November 18 8:30am PT Services Impacted: Portal Access Root Cause: Release Management Cross Platform dependencies. A feature improvement defect cascaded into ordering platforms, and due to deployment timing, environments were not aligned upon deployment of the Releases. POS Portal conducted a formal Agile Retrospective for proactive evaluation, and has identified 3 improvement opportunities to mitigate risk.