Zenscrape incident

Degraded Performance

Minor Resolved View vendor source →

Zenscrape experienced a minor incident on November 25, 2020 affecting Search, lasting 17h 23m. The incident has been resolved; the full update timeline is below.

Started
Nov 25, 2020, 01:22 PM UTC
Resolved
Nov 26, 2020, 06:45 AM UTC
Duration
17h 23m
Detected by Pingoru
Nov 25, 2020, 01:22 PM UTC

Affected components

Search

Update timeline

  1. investigating Nov 25, 2020, 01:22 PM UTC

    We are observing performance issues with our search platform and this may result in few customers facing issues while loading Freshservice. We apologize for any inconvenience this may be causing and will provide updates as they become available.

  2. investigating Nov 25, 2020, 03:26 PM UTC

    The latency in loading Freshservice must be resolved now, however we are experiencing degraded performance with the search platform hence auto-refresh has been affected on the Ticket list view page/Dashboard.

  3. investigating Nov 25, 2020, 04:55 PM UTC

    The root cause of the issue has been identified due to the outage faced by AWS service. We are in touch with the AWS team to fix the issue.

  4. investigating Nov 25, 2020, 06:30 PM UTC

    As per the latest updates from AWS Kinesis Data Streams and few other AWS services are severely impaired. Due to this you would be experiencing degraded services. Our developers are in touch with the AWS team to address this issues. Updates to follow.

  5. investigating Nov 25, 2020, 07:40 PM UTC

    We are still working with the AWS team to fix the issues with degraded services. Updates to follow

  6. investigating Nov 25, 2020, 08:40 PM UTC

    We are still working with the AWS team to fix the issues with degraded services. Updates to follow

  7. investigating Nov 25, 2020, 09:40 PM UTC

    We are continuing to work with the AWS team to address the issue with the degraded performance. Updates to follow shortly

  8. investigating Nov 25, 2020, 10:40 PM UTC

    Some of the AWS services are recovering now. We are continuing to work with the AWS team to address the issue with the degraded performance. More updates to follow.

  9. investigating Nov 25, 2020, 11:40 PM UTC

    As per the latest update from AWS, mitigation for the AWS Kinesis Data Streams services is underway while other services are recovering. We are still working with the AWS team to address the issue with the degraded performance. More updates to follow.

  10. investigating Nov 26, 2020, 12:40 AM UTC

    As per the latest update from AWS, services that rely on the AWS Kinesis Data Streams services have begun observing recovery. We are still working closely with the AWS team to address the issue with the degraded performance. Updates to follow.

  11. investigating Nov 26, 2020, 01:40 AM UTC

    We're continuing to work closely with the AWS team to solve the issues with degraded performance. More updates to follow.

  12. investigating Nov 26, 2020, 02:40 AM UTC

    As per the latest update from AWS, issues with AWS Kinesis Data Streams services have been mitigated and the requests are being throttled. Over the next few hours, AWS is expected to relax the throttles and the service will be back to previous levels. We are closely following up with the AWS team. More updates to follow.

  13. in progress Nov 26, 2020, 03:40 AM UTC

    We're still working closely with AWS team to resolve the issues with the degraded performance. More updates to follow.

  14. investigating Nov 26, 2020, 04:40 AM UTC

    We're continuing to work closely with AWS team to resolve the issues with the degraded performance. Further updates to follow.

  15. investigating Nov 26, 2020, 05:40 AM UTC

    Our services are getting recovered slowly. We are closely monitoring and will keep you posted.

  16. investigating Nov 26, 2020, 06:40 AM UTC

    The application services are up and running, and you can access dashboard/search/ticket list view without any disruption. We are closely monitoring the services with AWS. Do reach out to us at [email protected] if you are experiencing any issues.

  17. resolved Nov 26, 2020, 06:45 AM UTC

    The incident has been resolved