CipherStash incident

Memory leak in CipherStash Proxy when inserting encrypted JSON

Minor Resolved View vendor source →

CipherStash experienced a minor incident on January 9, 2026 affecting CipherStash Proxy (CipherStash Proxy), lasting 15h 55m. The incident has been resolved; the full update timeline is below.

Started
Jan 09, 2026, 07:00 PM UTC
Resolved
Jan 10, 2026, 10:55 AM UTC
Duration
15h 55m
Detected by Pingoru
Jan 09, 2026, 07:00 PM UTC

Affected components

CipherStash Proxy (CipherStash Proxy)

Update timeline

  1. investigating Jan 09, 2026, 07:00 PM UTC

    Some customers are experiencing a memory leak in CipherStash Proxy when inserting records into their databases.

  2. investigating Jan 10, 2026, 04:11 AM UTC

    We are able to reliably replicate the issue in Proxy, and have added a test case to the Proxy integration tests that is failing. Our current hypothesis is: Proxy's mapped statement cache is not releasing older cached statements. What is the mapped statement cache in...

  3. investigating Jan 10, 2026, 05:05 AM UTC

    We have disproved the mapped statement cache hypothesis. We are performing additional profiling to determine the source of the leak.

  4. investigating Jan 10, 2026, 06:21 AM UTC

    Our current hypothesis is: the memory leak is in cipherstash-client We are continuing to debug the exact cause, and our current focus is on the JSON indexer

  5. investigating Jan 10, 2026, 07:23 AM UTC

    We have validated the hypothesis, and the leak has been identified in the encrypted JSON index generation in cipherstash-client After plaintext JSON is indexed, the memory is being zeroized, but not freed We are working on a fix for it now, and will make new CipherStash...

  6. investigating Jan 10, 2026, 08:46 AM UTC

    We have implemented a fix to cipherstash-client, and released it in 0.31.1 We have upgraded CipherStash Proxy to use cipherstash-client 0.31.1 We...

  7. resolved Jan 10, 2026, 10:55 AM UTC

    Fix has been released in CipherStash Proxy 2.1.9. Customers using searchable encrypted JSON in CipherStash Proxy will need to upgrade to 2.1.9 to resolve this issue.