Tango Card incident

Reported issues with saml SSO and Bulk orders

Minor Resolved View vendor source →

Tango Card experienced a minor incident on July 30, 2024 affecting Reward Link - Production and Tango Card Website and 1 more component, lasting 3h 59m. The incident has been resolved; the full update timeline is below.

Started
Jul 30, 2024, 09:02 PM UTC
Resolved
Jul 31, 2024, 01:02 AM UTC
Duration
3h 59m
Detected by Pingoru
Jul 30, 2024, 09:02 PM UTC

Affected components

Reward Link - ProductionTango Card WebsiteRewards Genius - ProductionItem

Update timeline

  1. investigating Jul 30, 2024, 09:02 PM UTC

    Tango engineering teams are investigating reported issues with Bulk Orders and saml SSO login. A fix has been applied and we are requesting customer retry.

  2. monitoring Jul 30, 2024, 09:58 PM UTC

    A fix has been implemented and we are monitoring the results.

  3. resolved Jul 31, 2024, 01:02 AM UTC

    This incident has been resolved.

  4. postmortem Jul 31, 2024, 01:18 AM UTC

    [https://tangocard.atlassian.net/browse/INTREQ-8414](https://tangocard.atlassian.net/browse/INTREQ-8414) includes findings, conclusions and decisions. Details are logged to the following slack channels: * tango-incidents-outages * temp-july-eng-on-call-sanity-validation * temp-ecs-hardening * team-eng-tx