Immutable incident

Relaying Database Load - Passport and Minting services impacted

Critical Resolved View vendor source →

Immutable experienced a critical incident on June 28, 2024 affecting Passport, lasting 1h 2m. The incident has been resolved; the full update timeline is below.

Started
Jun 28, 2024, 09:37 AM UTC
Resolved
Jun 28, 2024, 10:39 AM UTC
Duration
1h 2m
Detected by Pingoru
Jun 28, 2024, 09:37 AM UTC

Affected components

Passport

Update timeline

  1. identified Jun 28, 2024, 09:37 AM UTC

    We are currently investigating an issue impacting Passport users and Minting API. Passport: Users will experience transactions stuck in a submitted or pending state. Minting API: Mints are currently delayed. Our team is actively working on resolving this and will provide updates as soon as possible. Severity Level: P2 Next Update: 20:00 AEST We apologise for any inconvenience this may cause and appreciate your patience.

  2. identified Jun 28, 2024, 10:05 AM UTC

    The team has implemented a mitigation to restore operations, we are validating systems. Next update in 15minutes.

  3. monitoring Jun 28, 2024, 10:23 AM UTC

    Our team has implemented a fix for the issue and continuing to monitor the system. Severity Level: P2 Next Update: 15min We apologise for any inconvenience this may cause and appreciate your patience.

  4. resolved Jun 28, 2024, 10:39 AM UTC

    We are pleased to inform you that the issue affecting Passport and Minting services has been resolved. Our team has implemented a fix and verified that the service is operating normally. Incident Start Time: 18:37 AEST Incident Resolution Time: 20:30 AEST We apologise for any disruption this may have caused and thank you for your understanding.