Welocalize incident

Relay/pantheon/Junction errors

Critical Resolved View vendor source →

Welocalize experienced a critical incident on July 10, 2024 affecting Junction and Relay, lasting 20h 40m. The incident has been resolved; the full update timeline is below.

Started
Jul 10, 2024, 01:18 PM UTC
Resolved
Jul 11, 2024, 09:58 AM UTC
Duration
20h 40m
Detected by Pingoru
Jul 10, 2024, 01:18 PM UTC

Affected components

JunctionRelay

Update timeline

  1. investigating Jul 10, 2024, 01:18 PM UTC

    IT Engineering is aware of the current problems with Relay/Junction/Pantheon and are investigating.

  2. investigating Jul 10, 2024, 02:54 PM UTC

    we are continuing to investigate the cause of the issues

  3. identified Jul 10, 2024, 04:13 PM UTC

    We have identified the underlying cause of the outage and are working with our support partners to resolve the issue. We are not able to provide an ETA for resolution at this point

  4. identified Jul 10, 2024, 06:12 PM UTC

    We continue to work with our support partners to troubleshoot the affected component. We are not able to provide an ETA for resolution at this time.

  5. identified Jul 11, 2024, 12:29 AM UTC

    We have successfully restored some services and are beginning to see error rates clear. We will continue restoration efforts and update when systems are fully available again.

  6. monitoring Jul 11, 2024, 01:41 AM UTC

    All services have been restored. We will continue to monitor systems closely and correct any issues that arise.

  7. resolved Jul 11, 2024, 09:58 AM UTC

    This incident has been resolved.