Svix incident
Subtle changes in how payloads are sent broke signature verification for consumers verifying incorrectly
Svix experienced a notice incident on April 10, 2023, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Apr 10, 2023, 04:28 PM UTC
We changed the code to send the payload exactly the same way as it's sent to us (before, we were compacting it before sending). This meant that people that relied on the payload to be compact in order to verify webhooks (i.e they were verifying incorrectly) had verification failing. We reverted this immediately once it was reported signatures were failing for customers. While not a bug in Svix, as people verifying webhooks correctly wouldn't have had issues, it still caused disruption to our customers and we've added tests to ensure that we now always compact the payloads going forward. We are also working on making it even harder to get wrong for customers, but please refer to https://docs.svix.com/receiving/verifying-payloads/how for the correct way of verifying webhooks.