Unico incident

Webhook Instability Impacting ID Cloud Capabilities

Notice Resolved

Unico experienced a notice incident on July 29, 2026, lasting —. The incident has been resolved; the full update timeline is below.

Started
Jul 29, 2026, 02:04 PM UTC
Resolved
Jul 29, 2026, 02:04 PM UTC
Duration
—
Detected by Pingoru
Jul 29, 2026, 02:04 PM UTC

Update timeline

  1. resolved Aug 28, 2026, 09:28 PM UTC

    Executive Summary & Impact We would like to inform you that the incident temporarily affecting the ID Cloud Webhook notification service has been resolved. Impact Period: From 08:32 to 10:05. Platform Impact: The instability was strictly limited to Webhook notification delivery, resulting in a delay in message distribution. There was no degradation of the main platform, no failures in the liveness engine, and no data loss. Customer Impact: The issue affected a small portion of customers. Clients who have implemented a GET polling flow as a contingency (fallback) experienced no downtime or operational impact. Root Cause & Resolution Root Cause: The instability was triggered by a feature update in the environment. A temporary mismatch in access permissions between the task queue and the destination route caused requests to be denied, leading to a temporary backlog of notifications. Resolutive Action: The technical team completed the traffic redirection to the new environment, restoring proper communication between components and releasing the backlog of Webhooks. Processing has returned to normal, and the dispatch queue has been fully cleared. Commitment & Next Steps The stability of our services is our highest priority. To ensure total transparency and prevent recurrence, our engineering team will focus on the following fronts: Detailed Postmortem: A comprehensive report featuring an in-depth root cause analysis and a preventive action plan will be prepared. Observability Improvements: We will adjust monitoring metrics to ensure greater visibility into inter-service communication errors during transition scenarios. We sincerely apologize for the inconvenience and remain available for any further clarifications. Sincerely, Unico Team

  2. resolved Aug 28, 2026, 09:28 PM UTC

    On July 29, 2026, between 8:32 AM and 10:05 AM \(Brasília time\), the webhook delivery service was degraded for approximately 1 hour and 33 minutes. The incident was caused by a planned infrastructure migration that created a transient incompatibility state: after internal routes were switched to the new cluster, the webhook service began generating authentication tokens from the new cluster, but deliveries were still being directed to the old cluster — which rejected the tokens with an authorization error. Lost events were reprocessed on demand. Multiple products were affected simultaneously — including authentication, payments, and onboarding flows. Users of clients that relied exclusively on webhook notifications to continue their business flows experienced operational interruptions during the period. Clients that had implemented active polling as a fallback mechanism experienced reduced impact. The root cause was identified in the migration planning: the procedure did not map the chained dependency of the webhook flow — where the internal route creates an asynchronous task that, in turn, calls the public route for delivery. By migrating the internal routes and the public routes in separate steps, an incompatibility window was created where tokens generated by the new cluster were rejected by the old cluster's authorization policy, which specifically validated the previous service account identifier. This scenario was not identified during the change's risk assessment, and none of the existing service quality indicators covered the asynchronous delivery layer where the failure materialized. The team identified the token incompatibility as the source of the failures and completed the migration of the public routes to the new cluster, eliminating the divergence. From that point on, the pending deliveries in the queue began processing normally. The impact ceased at 10:05 AM, and traffic on the old cluster reached zero at 10:32 AM. The incident showed that migrations involving chained asynchronous delivery flows require all routes participating in the chain — internal and public — to be migrated in a coordinated manner, without incompatibility windows between clusters. The gradual approach used, while suitable for most migrations, created a prolonged failure period here that could have been avoided with an atomic transition or a much shorter transition window. Additionally, the lack of observability at the asynchronous delivery layer was the reason the incident was not detected internally — existing indicators measured only event acceptance, not actual delivery. Creating dedicated alerts for failures and backlog at this layer is the most urgent monitoring improvement identified.