Certain users are unable to see feature flags in prod1 and prod2

Update timeline

1. investigating Jul 14, 2026, 11:25 AM UTC

   Certain users are unable to see feature flags in prod1 and prod2

2. monitoring Jul 14, 2026, 11:56 AM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Jul 14, 2026, 11:57 AM UTC

   This incident has been resolved.

4. postmortem Jul 23, 2026, 10:32 PM UTC

   ## Summary On July 14, 2026, certain Harness Feature Flag \(FF\) Classic customers on the Prod1 and Prod2 production environments were unable to view Feature Flags in the Harness platform. Affected requests returned an authorization error \(HTTP 403\), so Feature Flags were not visible for those users until access was restored. The behaviour was caused by a planned security update that began requiring an additional Feature Flags permission for related read operations. Users whose roles did not yet include that permission were correctly denied access, which appeared as a product failure. Harness temporarily rolled back the Feature Flags service change to restore access, updated the required permissions for affected users and roles, and confirmed that Feature Flag visibility returned to normal. The stronger permission checks remain in place going forward. ## Impact During the customer-reported incident window on July 14, 2026 \(status page approximately 11:25 UTC to 11:57 UTC\): * Certain Feature Flag Classic customers on **Prod1** and **Prod2** were impacted. * Affected users could not view Feature Flags in the Harness UI / API and received **403 Forbidden** responses. * Impact was limited to users and roles that did not yet have the updated Feature Flags permission required by the security change. * A public status page update was posted for Prod1 and Prod2 Feature Flags. There was **no data loss**, no change to stored feature flag configurations, and no impact to Feature Flag evaluation for applications whose SDK credentials and permissions were unaffected. Customers and users with the required permission continued to operate normally. ## Root Cause Harness deployed a planned Feature Flags authorization update that enforces the `ff_targetgroup_view` permission on target-related read paths used when viewing Feature Flags. This enforcement is intentional and remains the expected behaviour. Users and roles that had not yet been granted `ff_targetgroup_view` received 403 responses and could not see Feature Flags. From the customer’s perspective this looked like an outage; it was an authorization denial due to missing required permissions after the security update. ## Mitigation Harness completed the following mitigation steps: * Temporarily rolled back the Feature Flags service in Prod2 and Prod1 \(and aligned Prod0\) to restore access quickly while permissions were corrected. * Updated / granted the required `ff_targetgroup_view` permission for affected users and roles. * Verified that Feature Flag visibility returned to normal and closed the status page incident. These actions restored customer access. The permission enforcement introduced by the security update **remains in effect** going forward; the lasting fix is correct permission assignment, not removal of the check. ## Preventive Actions To avoid a recurrence of this issue, Harness is taking the following actions: * Keeping the stronger Feature Flags permission checks in place as the permanent security posture. * Ensuring role and permission updates for `ff_targetgroup_view` are applied with \(or before\) similar authorization changes in production.