Unico incident

Instabilidade no Serviço de Captura e Reaproveitamento de Documentos (ID DOCS)

Major Resolved View vendor source →

Unico experienced a major incident on June 2, 2026 affecting API, lasting 29m. The incident has been resolved; the full update timeline is below.

Started
Jun 02, 2026, 06:55 PM UTC
Resolved
Jun 02, 2026, 07:25 PM UTC
Duration
29m
Detected by Pingoru
Jun 02, 2026, 06:55 PM UTC

Affected components

API

Update timeline

  1. identified Jun 02, 2026, 07:19 PM UTC

    Prezado Cliente, Nossa monitoração identificou um impacto no fluxo de criação de processos que utilizam o flow de Captura e Reaproveitamento de Documentos(IDDocs), afetando os processos integrados a essa capacidade, o problema ocorreu por volta das 15:50 finalizando por volta das 16:10 Nosso time de tecnologia está analisando o ambiente e em breve retornaremos com novas atualizações.

  2. monitoring Jun 02, 2026, 07:21 PM UTC

    Prezado Cliente, Nossa equipe identificou as causas do problema e realizou as ações para que este incidente fosse solucionado. Dentro de alguns dias compartilharemos maiores detalhes através de um Postmortem. Pedimos desculpas pelo transtorno e nos colocamos à disposição para sanar dúvidas através dos nossos canais de atendimento. Atenciosamente, Equipe Unico!

  3. resolved Jun 02, 2026, 07:25 PM UTC

    Prezado Cliente, Incidente resolvido. Nossa equipe identificou as causas do problema e realizou as ações para que este incidente fosse solucionado. Dentro de alguns dias compartilharemos maiores detalhes através de um Postmortem. Pedimos desculpas pelo transtorno e nos colocamos à disposição para sanar dúvidas através dos nossos canais de atendimento. Atenciosamente, Equipe Unico!

  4. postmortem Jul 20, 2026, 08:01 PM UTC

    Public Postmortem: Outage in the Process Creation Flow **Incident date:** June 2, 2026 **Total duration:** 9 minutes **Executive Summary** On June 2, 2026, the process creation flow received an atypical volume of batch requests containing inconsistent data and expired credentials. The validation service correctly rejected these requests due to invalid data. However, the circuit breaker mechanism on one of our internal services interpreted this concentrated volume of rejections as a downstream infrastructure failure and opened the circuit globally. As a result, process creation was temporarily blocked platform-wide, extending the impact beyond the original anomalous traffic. **Impact** The impact was strictly limited to the process creation flow. No data was lost or corrupted. * **Duration:** approximately 9 minutes \(from the first alert to full recovery\) * **Availability:** dropped to 95.8% * **Error volume:** approximately 5,100 error events logged * **Scope:** multiple users had their SLA/SLO targets affected during the period **Root Cause** The incident was caused by a combination of three architectural factors: 1. **Incorrect Error Classification:** The orchestration system did not differentiate between request validation errors \(HTTP 4xx\) and actual downstream service failures \(HTTP 5xx\). As a result, legitimate rejections due to invalid data were incorrectly counted as system unavailability. 2. **Lack of Traffic Isolation \(Multitenancy\):** The circuit breaker operated at a global level. The concentrated volume of errors from a single source was enough to open the circuit for the entire user base \(a "noisy neighbor" effect\). 3. **No Rate Limiting at the Edge:** There was no granular, per-origin rate limiter at the entry layer to contain the anomalous request volume before it impacted the internal ecosystem. **Resolution and Mitigation** The engineering team identified the pattern minutes after the first alert. Mitigation was carried out via a feature flag, disabling the circuit breaker's global behavior directly in production, with no need for a new deployment. Once disabled, the error volume normalized immediately and service availability was restored. **Lessons Learned and Next Steps** _What worked well:_ * **Effective Monitoring:** Multi-layer instrumentation \(application, edge, and infrastructure\) made it possible to correlate data and diagnose the issue quickly. * **Engineering Tooling:** The use of feature flags proved essential for record-time mitigation \(under 10 minutes\). As an action plan, we immediately adjusted the orchestration layer to disregard HTTP 4xx errors in the circuit breaker's metrics. In the medium term, we will refactor the protection mechanism to isolate point-in-time traffic spikes and prevent global impact, complementing this response with the already-completed update to our runbooks documenting this scenario. **Commitment to Transparency:** We apologize for the impact caused. This incident highlighted the need to accelerate our traffic isolation architecture, which is now being treated as a top priority by the engineering team. Unico Team.