Kontent.ai incident

Application outage for all projects located in the US datacenter

Major Resolved View vendor source →

Kontent.ai experienced a major incident on April 13, 2023 affecting Application and Application and 1 more component, lasting 11h 57m. The incident has been resolved; the full update timeline is below.

Started
Apr 13, 2023, 10:32 AM UTC
Resolved
Apr 13, 2023, 10:29 PM UTC
Duration
11h 57m
Detected by Pingoru
Apr 13, 2023, 10:32 AM UTC

Affected components

ApplicationApplicationManagement REST API

Update timeline

  1. identified Apr 13, 2023, 10:32 AM UTC

    We are currently experiencing intermittent UI issues in the North American area. You may experience longer response times of our application UI. No API is affected by this. We have identified one of our backend service providers as the source of the issue and are currently in contact with this provider to find a solution.

  2. identified Apr 13, 2023, 10:32 AM UTC

    We are continuing to work on a fix for this issue.

  3. identified Apr 13, 2023, 12:17 PM UTC

    We are continuing to work on a fix for this issue.

  4. identified Apr 13, 2023, 12:52 PM UTC

    We are continuing to work on a fix for this issue.

  5. identified Apr 13, 2023, 02:03 PM UTC

    The application outage in the US data center no longer appears intermittently, but constantly. After testing we've confirmed the Management API for US projects is also affected by the outage. We are doing everything we can to find a solution and bring the system back online. Thank you for your understanding and patience.

  6. monitoring Apr 13, 2023, 05:59 PM UTC

    Our metrics are showing strong improvement, and at this point users should be able to access projects in the US data center with only minor slowdown or load time. Please keep in mind there may be some delay on API propagation due to traffic, but our initial tests are showing good speed. We will continue to monitor the state of the database which serves the US data center, and work with the service provider to determine the root cause and how this can be prevented in the future. Thank you for your patience and understanding.

  7. resolved Apr 13, 2023, 10:29 PM UTC

    For the last three hours of monitoring, we have had no new reports of any issues, and our metrics suggest that our back end has stabilized. We have received assurance from our back-end service provider that they are aware of the issue and are taking the necessary steps to make sure that it doesn’t happen again. We would like to thank everyone once again for their understanding and patience. If you run into any issues or have any questions, please don’t hesitate to reach out to our support team.