CleverTap incident

We are experiencing downtime on our India SDK endpoint

Major Resolved View vendor source →

CleverTap experienced a major incident on December 1, 2022 affecting SDK Ingestion, lasting 7h 45m. The incident has been resolved; the full update timeline is below.

Started
Dec 01, 2022, 04:03 AM UTC
Resolved
Dec 01, 2022, 11:48 AM UTC
Duration
7h 45m
Detected by Pingoru
Dec 01, 2022, 04:03 AM UTC

Affected components

SDK Ingestion

Update timeline

  1. investigating Dec 01, 2022, 04:03 AM UTC

    We are currently investigating an issue that is causing degraded performance/partial outage on our India SDK Endpoint. Please note that our SDK has a built-in retry mechanism and the traffic will come back to normal when the issue is resolved. We will provide further updates in 30 minutes.

  2. investigating Dec 01, 2022, 04:49 AM UTC

    We are continuing to investigate this issue and working with the Amazon AWS team to bring the services back to normal. We appreciate your patience and we will update the status here in some time.

  3. investigating Dec 01, 2022, 05:04 AM UTC

    We are upgrading the severity of the incident to "Major Outage" for SDK ingestion in the India region. We are continuing to work with Amazon AWS team to get the system back to normal. Currently, this issue has affected the ingestion in the India region for SDK traffic. This will also affect the action-based (trigger) campaigns/journeys. We appreciate your patience and we will continue to update the status here.

  4. investigating Dec 01, 2022, 06:17 AM UTC

    We are continuing to investigate this issue further. We are ruling out a few possibilities with the help of Amazon AWS. We will update here as soon as possible.

  5. investigating Dec 01, 2022, 07:27 AM UTC

    We are continuing to investigate this issue further. So far we have eliminated a few reasons. However, we are taking more time to identify the exact issue which is causing the downtime. We appreciate your patience and we will continue to update the progress here.

  6. investigating Dec 01, 2022, 09:25 AM UTC

    We are taking more time to identify the exact issue which is causing the downtime. Thank you for your patience and we will continue to update the progress here.

  7. identified Dec 01, 2022, 10:48 AM UTC

    The issue has been identified and a fix is being implemented. We will update the progress in the next few minutes.

  8. monitoring Dec 01, 2022, 11:24 AM UTC

    We have deployed the changes and currently monitoring the system. We will observe this for some time before closing the incident.

  9. resolved Dec 01, 2022, 11:48 AM UTC

    This incident has been resolved and SDK ingestion has resumed to normal.