Proxyclick incident

Users may be unable to access the Dashboard and other components

Critical Resolved View vendor source →

Proxyclick experienced a critical incident on November 4, 2021 affecting Dashboard and iPad app and 1 more component, lasting 19m. The incident has been resolved; the full update timeline is below.

Started
Nov 04, 2021, 11:04 AM UTC
Resolved
Nov 04, 2021, 11:24 AM UTC
Duration
19m
Detected by Pingoru
Nov 04, 2021, 11:04 AM UTC

Affected components

DashboardiPad appBrowser-based KioskSMSMailSlackSkype for BusinessAPIWebhooksBox integration

Update timeline

  1. investigating Nov 04, 2021, 11:04 AM UTC

    We are currently investigating the issue

  2. monitoring Nov 04, 2021, 11:12 AM UTC

    A fix has been implemented, we are currently monitoring the situation

  3. monitoring Nov 04, 2021, 11:15 AM UTC

    We are continuing to monitor for any further issues.

  4. resolved Nov 04, 2021, 11:24 AM UTC

    The incident is now resolved. The kiosk status displayed on Settings > Kiosk list may display Unknown with no impact to the performance of the app, this will be restored to accurate status shortly. A root-cause report will be made available as soon as available.

  5. postmortem Nov 04, 2021, 11:46 AM UTC

    Due to a server outage, API calls that are used to manage our kiosk status created a backlog. This backlog caused an impact on the performance of the platform impacting accessibility and performance of the application. Our team responded to the notifications received from our server provider and immediately proceeded to manage the backlog of outstanding API calls to restore functionality. We will proceed to review the time-out configuration and management of these API calls to ensure high availability of the server and minimise any possible impact to the platform.