SynchroNet incident

Slow page loads/timeouts

Minor Resolved View vendor source →

SynchroNet experienced a minor incident on October 9, 2020 affecting Dashboard, lasting 40m. The incident has been resolved; the full update timeline is below.

Started
Oct 09, 2020, 03:49 PM UTC
Resolved
Oct 09, 2020, 04:30 PM UTC
Duration
40m
Detected by Pingoru
Oct 09, 2020, 03:49 PM UTC

Affected components

Dashboard

Update timeline

  1. investigating Oct 09, 2020, 03:49 PM UTC

    We have had a handful of reports of slow page loads and page load timeouts from multiple customers. We are currently investigating this issue to determine if there is an issue that could be impacting all CLICK users.

  2. investigating Oct 09, 2020, 04:30 PM UTC

    We are able to see that some backend processes were experiencing performance issues between approximately 10:00 AM EDT and 11:00 AM EDT this morning. At this time, the issue appears to have stopped. Our logs and metrics indicate no internal source for the issue. We will be opening a support ticket directly with AWS to identify any potential issues on their end that may have caused this incident.

  3. resolved Oct 09, 2020, 04:30 PM UTC

    This incident has been resolved.

  4. postmortem Oct 09, 2020, 07:21 PM UTC

    After contacting Amazon Web Services, we received confirmation of an incident impacting network connectivity for EC2 instances as well as causing a large number of invocation errors for Lambda functions. This was the cause of the slow page loads and timeout errors in the CLICK dashboard.