Firezone incident

Client connections degraded

Resolved on July 21, 2025.

resolved Jul 22, 2025, 06:54 AM UTC

On July 21, at around 5:30a pacific time, database maintenance was performed to remove stale policy authorization records. Due to a bug in the event processing handler for each connected client, this resulted in large numbers of policy updates to be sent to some Firezone clients that were connected at the time. Depending on the level of resource availability on the host machine at the time of occurrence, some Firezone clients were unable to process this burst of events, and subsequently started dropping them. In some cases, this could have led to one or more Resource(s) being unavailable in the client that the user would normally have access to. Once the client reconnected, either through a client restart or re-authentication, the list of Resources would be reset, resolving the issue. A fix for the event processing filter was deployed later this day at around 5:00 pm pacific time, mitigating the potential for similar issues to occur in the future.

Started
Jul 21, 2025, 12:30 PM UTC
Resolved
Jul 21, 2025, 12:30 PM UTC
Duration
–
Detected by Pingoru
Jul 21, 2025, 12:30 PM UTC

Update timeline

  1. resolved Jul 22, 2025, 06:54 AM UTC

    On July 21, at around 5:30a pacific time, database maintenance was performed to remove stale policy authorization records. Due to a bug in the event processing handler for each connected client, this resulted in large numbers of policy updates to be sent to some Firezone clients that were connected at the time. Depending on the level of resource availability on the host machine at the time of occurrence, some Firezone clients were unable to process this burst of events, and subsequently started dropping them. In some cases, this could have led to one or more Resource(s) being unavailable in the client that the user would normally have access to. Once the client reconnected, either through a client restart or re-authentication, the list of Resources would be reset, resolving the issue. A fix for the event processing filter was deployed later this day at around 5:00 pm pacific time, mitigating the potential for similar issues to occur in the future.

Your product runs on more than Firezone

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →