LoyaltyLion incident

Partial API outage and delays in data processing

Minor Resolved View vendor source →

LoyaltyLion experienced a minor incident on October 28, 2024 affecting API and Platform and SDK, lasting 16h 8m. The incident has been resolved; the full update timeline is below.

Started
Oct 28, 2024, 05:49 PM UTC
Resolved
Oct 29, 2024, 09:58 AM UTC
Duration
16h 8m
Detected by Pingoru
Oct 28, 2024, 05:49 PM UTC

Affected components

APIPlatform and SDK

Update timeline

  1. identified Oct 28, 2024, 05:49 PM UTC

    An issue in our caching layer led to elevated error rates and response times. We have identified the issue and stabilised our systems. We're now working through the data processing backlog while monitoring.

  2. identified Oct 28, 2024, 08:27 PM UTC

    Our systems remain stable. We are still working through the data processing backlog whilst continuing to monitor.

  3. monitoring Oct 28, 2024, 11:27 PM UTC

    A fix has been implemented in our caching layer. We continue to monitor the results.

  4. resolved Oct 29, 2024, 09:58 AM UTC

    This incident has been resolved. All our systems are operational and the data backlog has been cleared.