eCatholic experienced a major incident on February 14, 2018, lasting 20h 29m. The incident has been resolved; the full update timeline is below.
Update timeline
- investigating Feb 14, 2018, 04:03 PM UTC
We are aware of an issue that is rendering some websites unable to load. Our development team is currently investigating the issue.
- identified Feb 14, 2018, 04:33 PM UTC
Our developers have identified the problem and are currently working with Rackspace, our data center, to resolve the issue.
- identified Feb 14, 2018, 05:05 PM UTC
Our developers are still working with Rackspace to resolve the error messages visitors are receiving. At this time we do not have an ETA on when the servers will be back up to 100% operational. However, we will continue to post updates here at least every 30 min.
- identified Feb 14, 2018, 05:32 PM UTC
Our most recent attempt at a fix was unsuccessful and we are continuing to work on a solution with Rackspace.
- identified Feb 14, 2018, 06:20 PM UTC
The intermittent issues are due to a massive spike in increased website traffic directly related to Ash Wednesday. Yesterday, we increased our server infrastructure in anticipation of today's traffic, but some of our servers are still experiencing slow load times or occasional loading errors. We are continuing to work with Rackspace to mitigate the issues. Please pray for a quick resolution and thanks for your patience!
- identified Feb 14, 2018, 07:40 PM UTC
We are continuing to work with Rackspace to balance today's massive Ash Wednesday website traffic across our infrastructure. We are monitoring sites closely and doing everything we can to lighten the load to keep traffic flowing. We anticipate slower-than-normal load times (and the occasional loading error) for the rest of today until Ash Wednesday Masses wrap up this evening. If anything changes, we'll post an update. Thank for your patience today.
- monitoring Feb 14, 2018, 11:59 PM UTC
All sites should be pulling up fine (and fast) now. We were able to pinpoint a minor code inefficiency with News and Events modules that was majorly amplified with today's massive influx of website traffic. We have temporarily disabled all News and Events modules from all sites until today's traffic spike subsides. We anticipate that traffic will return to normal later tonight and then we should be able to reenable News and Events modules on all sites at that time. Until then, your website will have a blank area where any News and Events modules normally appear. We will post an update once we are able to reenable all News and Events modules. Thanks for your prayers and patience!
- monitoring Feb 15, 2018, 03:01 AM UTC
Website traffic numbers are still elevated across our infrastructure due to searches for Ash Wednesday Mass times on the West Coast, so we are going to wait until tomorrow morning (CST) to re-enable News and Events modules on all sites. Thanks for your patience today and we will update you in the morning when we reenable News and Events modules on all sites.
- resolved Feb 15, 2018, 12:32 PM UTC
Our development team has resolved the issue and reenabled the News and Events modules as of 4:45am CST this morning. We've been monitoring our servers since and have resolved the issue. Please contact [email protected] if you're still experiencing any issues that we may not be aware of. Thanks again for your patience and prayers!