Ekklesia 360 incident

Timeouts in the CMS and Customer Websites

Major Resolved View vendor source →

Ekklesia 360 experienced a major incident on March 18, 2020 affecting CMS Backend and CMS API, lasting 1h 13m. The incident has been resolved; the full update timeline is below.

Started
Mar 18, 2020, 05:52 PM UTC
Resolved
Mar 18, 2020, 07:05 PM UTC
Duration
1h 13m
Detected by Pingoru
Mar 18, 2020, 05:52 PM UTC

Affected components

CMS BackendCMS API

Update timeline

  1. investigating Mar 18, 2020, 05:52 PM UTC

    We are currently experiencing heavy load on our database and as a result, you may experience 504 Gateway Timeout errors when navigating in the CMS. You may also see that certain content on your website does not load. We are prioritizing website traffic over CMS traffic at this time. Our whole team is working on resolving the issue. Thank you for your patience during this time.

  2. investigating Mar 18, 2020, 05:53 PM UTC

    We are continuing to investigate this issue.

  3. investigating Mar 18, 2020, 06:53 PM UTC

    We have temporarily disabled access to the CMS while we bring our Databases back up.

  4. resolved Mar 18, 2020, 07:05 PM UTC

    We have updated our database auto-scaling and our services are back to running normally.