TestRail incident

Degraded Performance

Major Resolved View vendor source →

TestRail experienced a major incident on February 26, 2024 affecting TestRail Cloud, lasting 1d 3h. The incident has been resolved; the full update timeline is below.

Started
Feb 26, 2024, 04:22 PM UTC
Resolved
Feb 27, 2024, 08:02 PM UTC
Duration
1d 3h
Detected by Pingoru
Feb 26, 2024, 04:22 PM UTC

Affected components

TestRail Cloud

Update timeline

  1. investigating Feb 26, 2024, 04:22 PM UTC

    Some customers may be experiencing degraded performance. Our engineers are investigating.

  2. identified Feb 26, 2024, 06:57 PM UTC

    Our team is actively working to resolve the incident. Customers may see their instances in maintenance mode intermittently, but for no longer than 15 minutes.

  3. identified Feb 26, 2024, 08:19 PM UTC

    Some customers may experience maintenance windows longer than 15 minutes as our team continues to work to resolve the incident.

  4. monitoring Feb 27, 2024, 12:10 AM UTC

    The incident has been resolved, and all customers should now have access to their instances. Please note that we have temporarily increased the API rate limit restrictions, and reduced the background task frequency while we monitor the system. We appreciate your patience during this time, and are committed to maintaining service stability.

  5. monitoring Feb 27, 2024, 09:02 AM UTC

    Some customers are experiencing degraded performance. Our engineers are investigating.

  6. monitoring Feb 27, 2024, 03:50 PM UTC

    We're pleased to report significant performance improvements across our platform, with decreased error rates and API rate limits restored to their normal levels. Our team continues to closely monitor the system to ensure ongoing stability and reliability. Thank you for your patience and support.

  7. resolved Feb 27, 2024, 08:02 PM UTC

    This incident has been resolved.