Harness incident

Rule Base Segment supported SDKs timeout (FME Only)

Notice Resolved View vendor source →

Harness experienced a notice incident on April 28, 2026, lasting —. The incident has been resolved; the full update timeline is below.

Started
Apr 28, 2026, 09:48 PM UTC
Resolved
Apr 28, 2026, 06:27 PM UTC
Duration
Detected by Pingoru
Apr 28, 2026, 09:48 PM UTC

Update timeline

  1. resolved Apr 28, 2026, 09:48 PM UTC

    Requests made by SDKs with Rule Based Segments support could have gotten a null instead of an empty for responses with no Rule Based Segments, leading to an null pointer exception. Impact observed: 18:27 until 19:05 UTC. The impact for SDKs would have been a failure to initialize or process an update. Any change made to feature flags or RBS in an affected environment would re-generate any specific caches remaining.

  2. postmortem May 08, 2026, 07:19 PM UTC

    ## Summary On April 28, 2026, a change was pushed to the Harness FME SDK API service modifying the response when there are no active Rule-based Segments to return on the SDK. The expected behavior was to return an empty list in this scenario, but a null value was served to SDKs instead, causing an error parsing the response. ## Impact * This issue impacted FME SDKs released May 2025 or after, as these support Rule-based segments. Prior versions were unaffected. * SDK instances were unable to initialize. Even if flags could be parsed, rule-based segments were not. Rendering the SDK unable to mark itself as ready and enable proper evaluations. * Client side SDKs WITH an existing cache were not impacted. The issue was specific to initializations. ## Timeline | Time \(UTC\) | Event | | --- | --- | | April 28, 18:27 | Deployed the defective SDK api version to production | | April 28, 18:35 | FME team detected the issue validating the production deployment | | April 28, 18:40 | Root cause was identified | | April 28, 18:50 | Fixed version validated e2e in lower, merged | | April 28, 19:05 | Service deployed to production successfully | | April 28, 19:08 | Fix validated in prod. | ## Root Cause When the new filter was added to filter non-active rule-based segments on initialization requests \(`rbSince=-1` on request params\), the Collect function of Golang iterables was used which returns `nil` when there are empty results. Prior to this, we just exposed the query result, which was never nil. This caused the unintended type change, leading to the issue on the SDKs related parser. ## Remediation **Fix deployed:** The team fixed the code on the service to ensure that empty lists were properly serialized as a list with no items \(`[]`\) in the responses served to SDKs. We confirmed the fix in staging and deployed to production as soon as possible. ## Action Items * Testing coverage analysis and improvements for Rule-based segments: Identify any remaining edge-cases that could be introduced by mistake and ensure we have the right coverage for all of these. * Improve continuous E2E suites visibility of the SDKs behavior: Our test applications running SDKs in them remained unaffected for the most part, given these are SDKs running emulating continuous real usage. This issue affected only initialization.