UserVoice incident

Portal Outage

Critical Resolved View vendor source →

UserVoice experienced a critical incident on November 1, 2023 affecting Web Portal (subdomain) and Admin Console and 1 more component, lasting 1h 10m. The incident has been resolved; the full update timeline is below.

Started
Nov 01, 2023, 11:44 PM UTC
Resolved
Nov 02, 2023, 12:55 AM UTC
Duration
1h 10m
Detected by Pingoru
Nov 01, 2023, 11:44 PM UTC

Affected components

Web Portal (subdomain)Admin ConsoleUserVoice APIHelpdesk APIWidgetsEmail (incoming)Email (delivery)SearchUser Analytics

Update timeline

  1. investigating Nov 01, 2023, 11:44 PM UTC

    We are currently investigating this issue.

  2. monitoring Nov 02, 2023, 12:00 AM UTC

    A fix has been implemented and we are monitoring the results.

  3. resolved Nov 02, 2023, 12:55 AM UTC

    A security hardening aspect of our cluster infrastructure inadvertently prevented a healthy restart of our load-balancing services. This caused a propogation failure that prevented traffic from being served. We have resolved the issue and are investigating additional testing resources to verify it can't reoccur.