Harness incident

File Store Expression Degraded : Returning Null

Minor Resolved View vendor source →

Harness experienced a minor incident on September 28, 2026, lasting —. The incident has been resolved; the full update timeline is below.

Started
Sep 28, 2026, 11:30 AM UTC
Resolved
Sep 28, 2026, 11:30 AM UTC
Duration
—
Detected by Pingoru
Sep 28, 2026, 11:30 AM UTC

Update timeline

  1. resolved Sep 29, 2026, 01:12 PM UTC

    On 28th September 2026, we observed an issue with copying files from the Harness Filestore during the export step, where the files were being copied as null/empty files. As a result, all pipelines using Filestore APIs were blocked for approximately 1.5 hours. The issue was caused due to a Feature Flag rollout in Prod1 and Prod2, which was reverted once the issue was discovered. All the operations are back to normal.

  2. postmortem Oct 01, 2026, 05:28 PM UTC

    ### Summary On September 28, 2026, some customers on Prod-2 experienced failures in CD pipeline steps that read File Store content using pipeline expressions \(e.g., ``\). Affected expressions returned null instead of the expected file content, causing dependent steps to fail or behave incorrectly. The issue was traced to a platform update that tightened authorization checks on File Store APIs. The update was rolled back, and File Store expressions have been working normally since. ### Customer Impact * Only pipeline steps on Prod-2 that read File Store content via expressions were affected. Pipelines that do not use File Store expressions were not impacted. * Affected expressions resolved to null, causing steps that depend on configuration or settings files to fail or produce incorrect output. * No File Store content was modified. No files, secrets, credentials, or artifacts were exposed, and no data was lost. ### Root Cause As part of an ongoing security initiative, Harness deployed stricter authorization checks across several platform APIs, including the API that serves File Store content. When a pipeline evaluates a File Store expression, one internal Harness service calls another to retrieve the file. The new authorization checks failed, resulting in the requests being rejected and the expressions returning null. ### Mitigation Once the platform update was identified as the cause, it was rolled back across the production environment. File Store expressions have resolved correctly since the rollback. No customer action is required. ### Preventive Measures & Next Steps * **Audit affected APIs:** Review all APIs where the stricter authorization checks were applied to ensure internal service-to-service calls are correctly accounted for. * **Improve test coverage:** Enhance automated test suites to include File Store expression evaluation across pipeline execution paths