GetFeedback incident

Campaign overview page not loading + API timeout errors

Notice Resolved View vendor source →

GetFeedback experienced a notice incident on December 2, 2022 affecting GetFeedback Digital (formerly Usabilla) for Websites and GetFeedback Digital (formerly Usabilla) for Apps, lasting 5d 8h. The incident has been resolved; the full update timeline is below.

Started
Dec 02, 2022, 09:37 AM UTC
Resolved
Dec 07, 2022, 06:35 PM UTC
Duration
5d 8h
Detected by Pingoru
Dec 02, 2022, 09:37 AM UTC

Affected components

GetFeedback Digital (formerly Usabilla) for WebsitesGetFeedback Digital (formerly Usabilla) for Apps

Update timeline

  1. monitoring Dec 02, 2022, 09:37 AM UTC

    We are seeing improvements in the performance and are monitoring to ensure stability, however issue is still present. We will update once the issue has been fully resolved.

  2. monitoring Dec 02, 2022, 09:38 AM UTC

    We are continuing to monitor for any further issues.

  3. monitoring Dec 02, 2022, 09:41 AM UTC

    We are continuing to monitor for any further issues.

  4. monitoring Dec 02, 2022, 11:45 AM UTC

    We deployed an updated version of the Public API which should improve performance and stability. We continue to work with our vendor to fully address the issue

  5. monitoring Dec 02, 2022, 12:45 PM UTC

    We've deployed another improvement in an effort to further reduce the slowness and are monitoring to ensure stability.

  6. investigating Dec 05, 2022, 12:05 PM UTC

    After monitoring over the weekend, it appears our efforts have slightly improved performance. We continue to investigate the issue and work with our vendor to address the degraded performance.

  7. investigating Dec 05, 2022, 02:03 PM UTC

    In an effort to further pinpoint the issue, we are temporarily turning off Public API to measure impact . Will be monitoring and turning it back on shortly.

  8. investigating Dec 05, 2022, 03:58 PM UTC

    We have re-enabled our public API and are monitoring to see the effects of the latest changes to address the issue.

  9. identified Dec 06, 2022, 02:50 PM UTC

    A potential root cause for the disruption has been found and we are coordinating with our vendors to remedy the situation. We’re no longer seeing high error rates, and performance is satisfactory. The Public API has some slightly degraded performance but is functional. The total metrics in the campaigns overview page is currently still unavailable, which show the totals as zero in the list and activity chart (customers can see these by clicking into each campaign). We will update with more information when we the remediation changes have been applied

  10. monitoring Dec 07, 2022, 12:01 PM UTC

    We are seeing almost no errors and very snappy performance for the last while. Public API performance has also been consistently stable. We’ll continue to monitor for a while before officially closing the incident, but we’re cautiously optimistic that we’ve resolved the issue.

  11. resolved Dec 07, 2022, 06:35 PM UTC

    After closely monitoring today, we are happy to report we believe the issue has been addressed. All features have been enabled for all costumers this morning. Thank you for your patience while we addressed this issue.