Close experienced a minor incident on August 14, 2023 affecting Querying, lasting 1h 56m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 14, 2023, 06:50 PM UTC
We are currently investigating an issue whereby the limit function in our search filters is not being applied to bulk actions.
- investigating Aug 14, 2023, 06:50 PM UTC
We are continuing to investigate this issue.
- identified Aug 14, 2023, 07:45 PM UTC
The issue has been identified and a fix is being implemented.
- resolved Aug 14, 2023, 08:46 PM UTC
This incident has been resolved.
- postmortem Aug 22, 2023, 07:26 AM UTC
Close sincerely apologizes for the interruption of our service. We take the stability of a platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. ## Impact Bulk actions performed in Close between 2023-08-14 05:36 UTC and 2023-08-14 20:16 UTC were potentially delayed, executed on too many entities, or not executed at all. If a bulk action was executed and was using the “limit” search option, said option was not observed and the bulk action was performed on all search results without limit applied. Once we uncovered the issue we cancelled the execution of queued bulk actions to prevent action being taken on unintended entities such as leads or contacts. ## Root Cause and Resolution A bug was introduced in our API that caused the “limit” parameter to be discarded when bulk actions were being performed. Upon finding the bug we deployed an updated version of our API that solved the issue. We have increased the number of test cases covering this functionality to prevent similar issues from happening again. Improvements were also put in place regarding our escalation procedures to ensure issues like these get resolved more promptly in the future. ## Timeline * 2023-08-14 05:36 UTC - A bug is introduced in our API that discards the “limit” parameter when creating bulk actions * 2023-08-14 16:13 UTC - Close Engineering becomes aware of an issue with Bulk Actions * 2023-08-14 19:49 UTC - Close Engineering deploys a change fixing the issue with Bulk Actions * 2023-08-14 20:16 UTC - Bulk actions that weren’t executed yet were paused to avoid unintended action