Close experienced a minor incident on February 14, 2025 affecting Application UI and Indexing and 1 more component, lasting 54m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Feb 14, 2025, 09:11 PM UTC
We have become aware of degraded performance of the Close system. We are actively triaging the issue. Updates will be posted as they become available.
- identified Feb 14, 2025, 09:45 PM UTC
The issue has been identified and a fix is being implemented. Search indexing may be delayed for some customers.
- monitoring Feb 14, 2025, 09:54 PM UTC
A fix has been implemented and we are monitoring the results. Search indexing may be delayed for some users.
- resolved Feb 14, 2025, 10:05 PM UTC
This incident has been resolved.
- postmortem Feb 21, 2025, 07:42 PM UTC
Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. ## Impact On February 14th, 2025 for two and a half hours \(between 19:50 UTC and 22:00 UTC\) some users saw outdated data in search results and smart views. No data was lost during the outage. ## Root Cause and Resolution A software update introduced malformed data to our Search platform that disrupted updates to our Search database. Close Engineering reverted the change and deleted the malformed data, which allowed the platform to resume processing updates to our Search database. The processing backlog was cleared by 22:00 UTC when full functionality was restored. We are implementing a new set of metrics and alerts that will help us identify this kind of failure more quickly. We are also enhancing the Search platform to better identify and discard malformed data to prevent similar incidents from occurring. ## Timeline All times are in UTC and events take place on 2025-02-14. * 19:50: A software update is deployed that sends malformed data to our search platform. * 20:33: Close Engineering is notified of an elevated error rate on the search platform and start investigating. * 21:04: Root cause is identified and work starts on a fix. * 21:40: The issue is resolved and all data is up-to-date.