LimaCharlie incident

API USA overload, elevated errors

Resolved on March 19, 2025, affecting section-145355 (API USA Telemetry Ingestion) and section-145355 (API USA InsightAPI USA Insight) and 1 more.

resolved Mar 19, 2025, 05:24 PM UTC

We experience an unusually high volume of API calls in USA that over-running current scaling, causing some API calls to fail. Investigation and remediation on the way. 12:12PM MST - Systems have stabilized. We are still heavily engaged in monitoring and root causing the issue. More details as we have them. We are not considering this incident over just yet. 6:28PM - Systems have been stable for some time now. Root cause was issues that arose from migration of a backend system. Issues resolved.

Started
Mar 19, 2025, 05:24 PM UTC
Resolved
Mar 19, 2025, 05:24 PM UTC
Duration
–
Detected by Pingoru
Mar 19, 2025, 05:24 PM UTC

Affected components

section-145355 (API USA Telemetry Ingestion)section-145355 (API USA InsightAPI USA Insight)section-145355 (API USA External Telemetry)section-145355 (API USA Cloud Adapters)section-145355 (API USA Services)section-145355 (API USA Enrollment)section-145355 (API USA Management)section-145355 (API USA Modules)

Update timeline

  1. resolved Mar 19, 2025, 05:24 PM UTC

    We experience an unusually high volume of API calls in USA that over-running current scaling, causing some API calls to fail. Investigation and remediation on the way. 12:12PM MST - Systems have stabilized. We are still heavily engaged in monitoring and root causing the issue. More details as we have them. We are not considering this incident over just yet. 6:28PM - Systems have been stable for some time now. Root cause was issues that arose from migration of a backend system. Issues resolved.

Your product runs on more than LimaCharlie

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →