ServiceChannel incident
ServiceChannel System Performance Degradation
Resolved after 38 minutes, affecting Asset Manager, on September 5, 2023.
**Infrastructure/Hardware Instability** **Incident Report** **Date of Incident:**` `09/05/2023 **Time/Date Incident Started:** 09/05/2023, 09:15 am EDT **Time/Date Stability Restored:**` `09/05/2023, 10:19 am EDT **Time/Date Incident Resolved:**` `09/05/2023, 10:25 am EDT **Users Impacted:** All **Frequency:** Intermittent **Impact:** Major **Incident description:** Third party vendor infrastructure/hardware instability **Root Cause Analysis:** A third-party vendor infrastructure issue affected performance and system availability for the underlying data storage layer that services platform resources. **Actions Taken:** 1. Investigated system-generated alerts and identified affected platform functionality. 1. SRE and DBA teams initiated a platform infrastructure redeployment, forcing the new infrastructure to be spun up on unaffected infrastructure/hardware. **Mitigation Measures:** 1. Continue the ongoing investigation into root causes of infrastructure issues within our cloud hosting provider. 1. Continue to implement high availability improvements to prepare the platform to respond better to unexpected hardware issues that are beyond our control.
Affected components
Update timeline
- investigating Sep 05, 2023, 02:05 PM UTC
We are actively investigating degraded system performance. An update will be provided shortly. Thank you for your patience.
- resolved Sep 05, 2023, 02:44 PM UTC
This incident has been resolved. All services are working as expected.
- postmortem Sep 19, 2023, 01:59 PM UTC
**Infrastructure/Hardware Instability** **Incident Report** **Date of Incident:**` `09/05/2023 **Time/Date Incident Started:** 09/05/2023, 09:15 am EDT **Time/Date Stability Restored:**` `09/05/2023, 10:19 am EDT **Time/Date Incident Resolved:**` `09/05/2023, 10:25 am EDT **Users Impacted:** All **Frequency:** Intermittent **Impact:** Major **Incident description:** Third party vendor infrastructure/hardware instability **Root Cause Analysis:** A third-party vendor infrastructure issue affected performance and system availability for the underlying data storage layer that services platform resources. **Actions Taken:** 1. Investigated system-generated alerts and identified affected platform functionality. 1. SRE and DBA teams initiated a platform infrastructure redeployment, forcing the new infrastructure to be spun up on unaffected infrastructure/hardware. **Mitigation Measures:** 1. Continue the ongoing investigation into root causes of infrastructure issues within our cloud hosting provider. 1. Continue to implement high availability improvements to prepare the platform to respond better to unexpected hardware issues that are beyond our control.
Your product runs on more than ServiceChannel
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services