CaterTrax incident

Intermittent Site Latency and User Login Issues

Minor Resolved View vendor source →

CaterTrax experienced a minor incident on March 6, 2023 affecting TRAX Platform, lasting 10d 21h. The incident has been resolved; the full update timeline is below.

Started
Mar 06, 2023, 02:39 PM UTC
Resolved
Mar 17, 2023, 12:06 PM UTC
Duration
10d 21h
Detected by Pingoru
Mar 06, 2023, 02:39 PM UTC

Affected components

TRAX Platform

Update timeline

  1. investigating Mar 06, 2023, 02:39 PM UTC

    Please be advised that we are aware of and actively troubleshooting an issue that is currently impacting the ability of some users to log into their sites at this time. This issue is being treated as our highest priority and our team is working hard to get the problem resolved as quickly as possible.

  2. identified Mar 06, 2023, 04:55 PM UTC

    Our team has identified an issue that is currently causing intermittent site latency and 504 Gateway Timeout errors for some users. This issue is our highest priority and we are working to resolve it as quickly as possible.

  3. identified Mar 07, 2023, 08:17 PM UTC

    Our team has identified a few issues that are currently causing some intermittent site latency as well as impacting the ability for some users to log into their sites at this time. These issues are our highest priority and we are working to resolve them as quickly as possible.

  4. identified Mar 08, 2023, 06:20 PM UTC

    Our team is continuing to investigate new reports of degraded performance and latency issues. While the system is currently operating within expected response times, our focus continues to be on performance improvements and resolving open issues. Next update 8:00 AM EST tomorrow.

  5. identified Mar 09, 2023, 01:01 PM UTC

    The system continues to be operating within expected response times. While we work on the remaining open issues, we are closely monitoring performance and latency. Next update 8:00 AM EST tomorrow.

  6. monitoring Mar 10, 2023, 12:59 PM UTC

    Our team has identified and mitigated issues causing intermittent performance degradation to the TRAX Platform. While the system is currently operating within expected response times, we will continue to monitor system performance to ensure a consistent experience for you and your customers. Our focus will shift to additional tuning activities. Next update Tuesday 3/14 at 8:00 AM EST.

  7. monitoring Mar 14, 2023, 12:04 PM UTC

    The TRAX Platform remains within expected response times. Additional tuning activities will continue this week as we monitor system performance. Next update Friday 3/17 at 8:00 AM.

  8. resolved Mar 17, 2023, 12:06 PM UTC

    The TRAX Platform has returned to expected response times. Additionally, the intermittent site latency and user login issues have been resolved. This will be the last update.