Close Outage History

Started Sep 29, 2026, 05:48 PM UTC· resolved Sep 29, 2026, 06:15 PM UTC

1. investigating Sep 29, 2026, 05:48 PM UTC

   We are currently investigating issues with parts of the Close application not loading.

2. monitoring Sep 29, 2026, 06:11 PM UTC

   The issue was identified and reverted. We continue to monitor the application to ensure everything is back to normal.

3. resolved Sep 29, 2026, 06:15 PM UTC

   This incident has been resolved.

4. postmortem Oct 05, 2026, 05:25 PM UTC

   Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. ## Impact Close systems were unavailable for 30 minutes from Sep 29, 17:40 UTC to Sep 29, 18:10 UTC. ## Root Cause and Resolution During deployment of a new application version, we ran a multi-phased migration that added a permission to user roles and assigned it to existing custom roles. The timing of the phased migration wasn’t fully thought out; many application servers were still running the previous version which didn’t recognize the new permission in the database. As a result, error rates increased. We restored service by removing the permission from existing roles. To prevent similar incidents, we’re updating our testing and deployment pipeline to identify migrations that require a phased rollout. ## Timeline * 17:40 UTC - Migration was run and PR was merged. * 17:41 UTC - Engineering team alerted of API failures and application unresponsiveness * 17:43 UTC - Engineering team begins incident response, status page updated * 17:44 UTC - Potential root cause identified \(backfill migration was run before code finished deploying\). * 17:44 UTC - We thought we could fall forward, but realized that was going to take longer so we then decided to revert the migration. This added to the outage time. * 18:10 UTC - Engineering team runs a script to drop the new permission from existing custom roles * 18:10 UTC - Application loading again, error rates dropped. Customer impact ends.

Read the full incident report →

Started Aug 11, 2026, 01:12 PM UTC· resolved Aug 11, 2026, 03:15 PM UTC

1. investigating Aug 11, 2026, 01:12 PM UTC

   We are again investigating an elevated number of database failures.

2. investigating Aug 11, 2026, 02:15 PM UTC

   We are continuing to investigate the issue.

3. monitoring Aug 11, 2026, 02:53 PM UTC

   We have identified the problem, adjusted our scaling strategy, and are seeing healthy recovery of all systems.

4. resolved Aug 11, 2026, 03:15 PM UTC

   This incident has been resolved.

5. postmortem Aug 13, 2026, 10:56 PM UTC

   ## Summary & Impact Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously, and this incident is not acceptable to us. Below is an explanation of what happened and how we will prevent another such interruption from occurring. On August 11th, 2026, Close was intermittently unavailable for a total of about 2.5 hours between 08:00 UTC and 17:03 UTC. During those periods, the Close web app, mobile apps, and API returned errors or failed to load, affecting customers in all regions. Because the disruption was intermittent rather than continuous, many customers likely experienced it as a longer period of unreliability across the working day rather than as a handful of discrete outages. There was also added confusion because we marked our status page "resolved" twice during the day, at 09:26 UTC and again at 15:15 UTC, before the underlying cause had been fixed. We are improving our process so that we do not mark an incident as resolved until we have a good understanding of the situation and applied appropriate mitigations, not just because the symptoms have subsided. ## What went wrong? A routine code change to our email and calendar syncing service altered the way it looks up account information in our database. The new version looked equivalent to the code it replaced, but it caused our database to perform dramatically more internal work for each lookup, enough to exhaust the memory on our primary database server and repeatedly crash it. Because each crash was followed by an automatic recovery, the pattern looked like general instability rather than the symptom of a specific change, which is why it took us longer than we would have wished to trace it back and revert it. ## How we’ll fix this We have already reverted the problematic change and made configuration adjustments to our database hosts to reduce the severity of this class of failure. Close is currently stable. Going forward, we will improve the design of our Email & Calendar Syncing service and similar services which touch Organizations, Users, and Memberships. We will also improve our query performance visibility and alerting, so that slow or inefficient queries and new, unexpected query patterns are identified sooner than they were here. ## Technical Details For those interested in the specifics, here is the full sequence of events and the underlying database behavior. ### Timeline \(all times UTC\) * **August 10, 20:34** — The change finishes deploying to production. Our primary database shard briefly becomes unstable and fails over to a redundant node, producing a few minutes of errors. Because the change deployed after peak usage and deployment hours, traffic subsided shortly afterward and the database remained stable overnight. * **August 11, 08:00** — As application usage and engineering deployments begin increasing, error rates on our primary database shard start to climb. * **08:17** — Our monitoring alerts fire. * **08:26** — Close Engineering is investigating and has posted a status page incident. * **08:46** — We restart the affected database processes, allowing the system to recover. * **09:26** — All Close systems have returned to what looked like normal operation, and we mark the status page resolved. * **12:11 – 17:03** — The same loop of elevated errors, instability in the primary shard, failover to a redundant node, and temporary recovery repeats itself several times. * **14:45** — We find that the database is exhausting its allowed number of open files and raise that limit. This improves stability but does not address the underlying cause. * **15:15** — Systems appear to have recovered again, and we mark the incident resolved a second time. * **15:50** — We identify the previous day's deployment as the likely culprit. * **16:51 – 17:18** — We begin reverting the change when the database becomes unstable again. Customer-facing errors subside at 17:03, and the rollback completes at 17:18. ### The underlying database behavior Our Email & Calendar syncing service regularly queries a database for an up-to-date state of Organizations, Users, and their active Memberships. We store this information in our MongoDB cluster, on its primary shard. We were refactoring some of this code as part of normal tech debt work to bring older code in line with our current patterns. Previously, this query used a `$in: […long list of active organizations' IDs…]` clause as the filter for this query. This wasn’t necessarily very efficient, but it was acceptable because the query ran at a relatively low rate \(only happened every 50 seconds for one syncing process, of which we have 120-180 running depending on the demand\) and runs asynchronously. The refactor changed that query to find applicable Memberships via a `$or: [{organization: someOrgId, user: someUserId}, …long list of such conditions for all users with email/calendar accounts…]` clause. This at first looked like it should have performance roughly equal to the previous query. However, we have since discovered that, while our MongoDB cluster handled large `$in` queries decently well, a large set of compound queries all put into a root-level `$or` clause had disastrous performance characteristics. Even ~3-4 such queries per second, all running roughly around the same time alongside other normal requests, could overwhelm even our heavily scaled database node. Our current understanding, based on our review of the MongoDB source code and ad hoc testing, is that the issue relates to how MongoDB plans large root-level `$or` queries. MongoDB may evaluate the clauses of this query shape as separate sub-plans rather than as a single index scan. In our case, a query could contain approximately 50,000 clauses. At three such queries per second, the database was being presented with roughly 150,000 `$or` clauses per second. We believe this generated tens of thousands of index scans and WiredTiger cursors, churned the query-plan cache, and placed extreme pressure on memory and other database resources. This behavior is consistent with the out-of-memory terminations, file-handle exhaustion, and other low-level failures we observed. Reverting the new query shape eliminated the resource spikes and restored database stability. Pinpointing the query responsible for the database load took longer than anticipated because the seemingly small query rate, a flood of other slow queries, and an immediate database recovery obscured the underlying cause.

Read the full incident report →

Started Jun 29, 2026, 05:46 PM UTC· resolved Jun 29, 2026, 07:39 PM UTC

1. investigating Jun 29, 2026, 05:46 PM UTC

   We have become aware of degraded processing of background tasks in the Close system. We are currently investigating the issue.

2. monitoring Jun 29, 2026, 07:25 PM UTC

   We have identified the root cause and deployed a remediation. Background task processing performance is improving. Some customers may still see delayed processing of background tasks. Close Engineering is monitoring for improved performance.

3. resolved Jun 29, 2026, 07:39 PM UTC

   This incident has been resolved. Background tasks are processing normally.

Read the full incident report →

Started Jun 03, 2026, 02:50 PM UTC· resolved Jun 03, 2026, 06:53 PM UTC

1. investigating Jun 03, 2026, 02:50 PM UTC

   Changes done to CRM data are reflected in search with a delay.

2. investigating Jun 03, 2026, 03:21 PM UTC

   We are continuing to investigate the issue.

3. investigating Jun 03, 2026, 04:05 PM UTC

   We are continuing to investigate the issue. Updates to Close made via the web app and API may take longer than normal to appear in search results.

4. investigating Jun 03, 2026, 04:41 PM UTC

   We are continuing to investigate the issue. Updates to Close made via the web app and API may take longer than normal to appear in search results.

5. identified Jun 03, 2026, 05:45 PM UTC

   We have implemented a remediation for the delayed search indexing and are monitoring for improvement. Updates to Close made via the web app and API may take longer than normal to appear in search results.

6. monitoring Jun 03, 2026, 06:17 PM UTC

   A fix has been implemented and we are monitoring the results. Updates to Close made via the web app and API may take longer than normal to appear in search results.

7. resolved Jun 03, 2026, 06:53 PM UTC

   This incident has been resolved. The Close app and API are now functioning normally.

Read the full incident report →

Started May 12, 2026, 09:34 AM UTC· resolved May 12, 2026, 11:17 AM UTC

1. investigating May 12, 2026, 09:34 AM UTC

   Some users are experiencing connection issues and application errors. Our engineers are currently investigating issues loading the Close application, making & receiving calls, and responding to Close API requests.

2. monitoring May 12, 2026, 11:07 AM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved May 12, 2026, 11:17 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Apr 23, 2026, 04:56 PM UTC· resolved Apr 23, 2026, 07:44 PM UTC

1. investigating Apr 23, 2026, 04:56 PM UTC

   We are currently investigating this issue.

2. identified Apr 23, 2026, 05:14 PM UTC

   The issue has been identified and the fix is being implemented.

3. monitoring Apr 23, 2026, 05:56 PM UTC

   A fix has been implemented and we are monitoring the results.

4. monitoring Apr 23, 2026, 06:50 PM UTC

   We've deployed a fix, we're continuing to see improvements in overall system performance, we expect recovery slowly over the next several minutes.

5. resolved Apr 23, 2026, 07:44 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Apr 23, 2026, 04:16 PM UTC· resolved Apr 23, 2026, 09:18 PM UTC

1. investigating Apr 23, 2026, 04:16 PM UTC

   We are currently investigating this issue.

2. investigating Apr 23, 2026, 04:56 PM UTC

   We are experiencing an issue with a 3rd party provider.

3. monitoring Apr 23, 2026, 05:32 PM UTC

   We are experiencing an issue with a 3rd party provider.

4. monitoring Apr 23, 2026, 06:52 PM UTC

   We're continuing to monitor the situation and expect gradual recovery over the next hour.

5. monitoring Apr 23, 2026, 07:45 PM UTC

   We're continuing to monitor the situation and expect gradual recovery over the next hour.

6. resolved Apr 23, 2026, 09:18 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Mar 16, 2026, 04:08 PM UTC· resolved Mar 16, 2026, 04:55 PM UTC

1. investigating Mar 16, 2026, 04:08 PM UTC

   We are currently investigating this issue.

2. investigating Mar 16, 2026, 04:16 PM UTC

   We are investigating the issue and seeing some recovery.

3. monitoring Mar 16, 2026, 04:27 PM UTC

   Database infrastructure is operational.

4. monitoring Mar 16, 2026, 04:45 PM UTC

   Close services continue to recover. Most customers should see normal performance. Close Engineering continues to monitor the recovery.

5. resolved Mar 16, 2026, 04:55 PM UTC

   This incident is resolved. Close has returned to normal performance.

6. postmortem Mar 17, 2026, 03:13 PM UTC

   # Summary Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. The Close App and API were severely disrupted for all users for 18 minutes between 15:57 and 16:15 UTC on Monday March 16, 2026. Automated systems restored functionality by 16:15 UTC. # Incident Timeline | Time \(UTC\) | Event | | --- | --- | | 2026-03-16 | | | 15:57:00 | **Automated Database Maintenance begins** | | 15:58:42 | **Incident detected by monitoring** | | 15:58:49 | **Escalation acknowledged by Close Engineering On-Call** ​ | | 16:02:41 | **Incident accepted by Close Engineering** ​ | | 16:15:00 | **Affected database restored to normal operation** | | 17:09:22 | **Incident resolved** | # Root Cause and Resolution At ~15:57 UTC an automated database maintenance process began that starved our production database cluster of resources. This caused our primary instance and both replica instances of the production cluster to become unavailable for ~18 minutes. The system recovered and began performing normally at 16:15 UTC. To prevent this from happening in the future Close Engineering has provisioned additional resources to the affected database cluster to make it more resilient to sudden demand spikes. Close Engineering will also perform an audit of the maintenance process' configuration to identify opportunities to reduce its impact on system performance.

Read the full incident report →

Started Nov 05, 2025, 06:42 PM UTC· resolved Nov 05, 2025, 07:56 PM UTC

1. investigating Nov 05, 2025, 06:42 PM UTC

   We are receiving an increased number of reports that users are unable to use calling when using the Desktop Application. Calling within the browser works fine. We're investigating.

2. identified Nov 05, 2025, 07:21 PM UTC

   We have identified the issue and are actively working on a fix.

3. resolved Nov 05, 2025, 07:56 PM UTC

   The issue has been resolved. Please reach out to [email protected] if you continue to have calling issues after refreshing.

Read the full incident report →

Started Nov 01, 2025, 08:55 AM UTC· resolved Nov 01, 2025, 10:53 AM UTC

1. investigating Nov 01, 2025, 08:55 AM UTC

   We're currently investigating a slowdown in application performance. You may experience delays in loading pages. The engineering team is currently looking into resolving this as quickly as possible.

2. identified Nov 01, 2025, 10:32 AM UTC

   The issue has been identified and a fix is being implemented.

3. resolved Nov 01, 2025, 10:53 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 23, 2025, 11:40 AM UTC· resolved Oct 23, 2025, 03:52 PM UTC

1. monitoring Oct 23, 2025, 11:40 AM UTC

   We identified an issue causing Lead Pages to load slowly during Dialer sessions. A fix has been implemented and will be deployed shortly. After deployment, we’ll monitor the system to ensure the issue is fully resolved. New updates to follow.

2. resolved Oct 23, 2025, 03:52 PM UTC

   This incident has been resolved.

3. postmortem Oct 27, 2025, 04:51 PM UTC

   Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. ### Impact Between Tuesday 10/21 14:46–17:24 UTC, Wednesday 10/22 7:50–17:00 UTC, and Thursday 10/23 8:00–15:00 UTC, customers relying on real-time updates in Close \(which is critical for proper & timely functioning of our calling features and the Dialer\) experienced delays, outdated data being displayed on their screens, Dialer disconnections, and other unexpected behavior. ### Root Cause and Resolution On Monday 10/20 at 20:09 UTC, a subtle bug was introduced, which resulted in UI clients slowly opening more and more WebSocket subscriptions for our Data Monitoring Service – a component which powers real-time updates for our Activity Reporting pages. Over the next few days, the number of subscriptions kept rising. The rise was slow enough where it didn’t trigger our alerting, but fast enough to eventually start putting a real strain on our infrastructure and saturating our processing capacity. Our first course of action was to scale the supply to meet the demand, but the demand continued to rise more quickly. Because of unrelated rate-limiting changes that looked more directly responsible for the limitation of the supply, our engineering efforts went in a few wrong directions until finally the root cause – the UI bug – was identified and fixed on Thu 10/23 at 5:19 UTC. However, because the fix required clients to reload their UI, intermittent problems continued until enough clients have reloaded, finally resolving the problem on Thu 10/23 at 15:00 UTC. To prevent this problem from ever happening again, we are improving our monitoring, alerting, testing, and in the longer-term the architecture of our WebSocket subscription service. ### Timeline * Mon 10/20 20:09 UTC: A subtle bug is introduced and over the next few days keeps slowly increasing the number of WebSocket-based Data Monitoring Service subscriptions. * Tue 10/21 14:46 UTC: Close receives first signals of Dialer problems where Users hear the callee many seconds before the corresponding Lead Page loads and immediately start looking into the problem. * Tue 10/21 5:24 UTC: First fixes aimed at increasing the allowed rate of processing of the system land in production. The situation appears resolved. * Wed 10/22 7:50 UTC: The problem comes back despite the fixes. Engineers increase the scale of the system to meet the increasing demand, but the demand keeps saturating the scale. Status page is opened at 10:35 UTC \([https://status.close.com/incidents/ydzwc4l6xmhc](https://status.close.com/incidents/ydzwc4l6xmhc)\). Problems continue on and off until they fully subside around 17:00 UTC * Thu 10/23 5:19 UTC: Engineers identify the root cause and fix it. * Thu 10/23 8:00 UTC: Concerning metrics continue to emerge despite the fix. Customer reports follow and Status page is opened at 11:40 UTC \([https://status.close.com/incidents/j49qgd3m049n](https://status.close.com/incidents/j49qgd3m049n)\). Engineers continue working on the impact mitigation and root cause diagnosis until it is found and fixed at 11:47 UTC \(excessive WebSocket Subscriptions for our real-time Data Monitoring Service, coming from UI clients who visited the Activity Reporting page\). However, the fix being picked up requires some idle UI clients to reload, hence the saturated demand continues until 15 UTC despite scaling attempts. * Thu 10/23 15:00 UTC: The problem is officially resolved.

Read the full incident report →

Started Oct 22, 2025, 10:35 AM UTC· resolved Oct 22, 2025, 11:50 AM UTC

1. investigating Oct 22, 2025, 10:35 AM UTC

   We are currently investigating problems with Lead Pages being loaded too slowly while on a Dialer session.

2. monitoring Oct 22, 2025, 11:10 AM UTC

   A fix has been implemented, and we are currently monitoring this. New updates to follow.

3. resolved Oct 22, 2025, 11:50 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 20, 2025, 02:20 PM UTC· resolved Oct 20, 2025, 08:27 PM UTC

1. investigating Oct 20, 2025, 02:20 PM UTC

   There’s currently an outage on AWS’s end in a certain region. While our hosting is mainly in another region, many other services, including calling, are affected.

2. resolved Oct 20, 2025, 08:27 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 20, 2025, 10:38 AM UTC· resolved Oct 20, 2025, 01:12 PM UTC

1. monitoring Oct 20, 2025, 10:38 AM UTC

   There’s currently an outage on AWS’s end in the us-east-1 region. While our hosting is mainly in another region, many other services, including calling, are affected. We are actively monitoring this, and we will keep you updated

2. monitoring Oct 20, 2025, 11:16 AM UTC

   AWS has reported partial recovery on this incident (https://health.aws.amazon.com/health/status), but services have yet to recover fully. Our service provider, Twilio, is also actively monitoring this (https://status.twilio.com/) We will continue to keep you posted.

3. resolved Oct 20, 2025, 01:12 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 14, 2025, 03:26 PM UTC· resolved Oct 14, 2025, 04:00 PM UTC

1. investigating Oct 14, 2025, 03:26 PM UTC

   Some users are experience downtime. Our engineers are currently investigating issues loading the Close application and pulling data from the Close API.

2. investigating Oct 14, 2025, 03:48 PM UTC

   We are currently applying fixes to restore operations and monitoring their impact.

3. monitoring Oct 14, 2025, 03:53 PM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Oct 14, 2025, 04:00 PM UTC

   This incident has been resolved.

5. postmortem Oct 16, 2025, 06:41 PM UTC

   Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. ## Impact A small number of Close API requests began failing on October 14, 2025 for 67 minutes from 13:20Z to 14:27Z. A larger number of failures occurred for 88 minutes from 14:27Z to 15:55Z. The failures impacted our API as well as various components of the Close App. ## Root Cause and Resolution Unfortunately, a cascading set of events contributed to this failure. Three Pull Requests \(PR\) were in flight during this incident. The first and third had defects that impacted our API. Additionally, although it had no defects, the second PR added to the complexity of troubleshooting the core issues caused by the other two. The first PR made changes to our billing logic that inadvertently started making more requests than expected to an external provider. This extra load on the external provider caused a small number of our API requests to timeout. The larger impact was caused by the third PR which made changes to how some parts of our web requests are handled. We have recently updated our dependencies and are in the process of evolving various parts of our codebase to be better aligned with modern async processing techniques. This update appeared fine during normal testing but failed under our production load. The changes caused a very high utilization of our web workers’ thread pools. This in turn caused some of them to become saturated and left in a state where they were unable to process any new requests. This caused a significant number of API requests to time out. It took some time to identify which update was causing the API timeouts and revert its changes. A flaky test further delayed our deployment process. The recovery was prolonged because the failure affected our infrastructure's health check system. The same endpoints that were timing out were needed to validate and control our deployment changes. We are still reviewing all details related to this incident and are taking several actions to improve our response to complex incidents such as these. We are working to ensure the right resources are available to quickly diagnose issues, strengthening our CI/CD processes, and enhancing our internal QA procedures to better catch these complex defects during testing. ## Timeline * 13:20 UTC - Billing PR merged earlier in the morning starts causing a small number of API failures * 13:25 UTC - Engineering team alerted of API failures, response starts * 13:37 UTC - Async PR is merged, fails to deploy due to flaky e2e test * 13:45 UTC - Engineering team alerted of failing CICD deployments * 13:51 UTC - Revert billing PR, deploy fails due to flaky e2e test * 14:04 UTC - Unrelated PR merged which includes the billing change revert * 14:20 UTC - Both PR’s changes deployed * 14:27 UTC - Large spike of API failures followed up with pods failing to start due to status endpoint failing * 14:33 UTC - Start of main customer impact of the Async PR * 15:18 UTC - Revert Async PR * 15:55 UTC - Customer impact ends

Read the full incident report →

Started Sep 24, 2025, 12:01 PM UTC· resolved Sep 24, 2025, 01:31 PM UTC

1. investigating Sep 24, 2025, 12:01 PM UTC

   Some customers in the EMEA region are experiencing connectivity issues to the Close app. We believe this is a widespread issue and are continuing to investigate.

2. monitoring Sep 24, 2025, 12:22 PM UTC

   The issue appears to have resolved itself. We are still monitoring and waiting for our Cloud provider to confirm resolution.

3. monitoring Sep 24, 2025, 01:23 PM UTC

   AWS confirmed they were investigating an issue but haven't yet notified us that it has been resolved. We are still monitoring for new issues.

4. resolved Sep 24, 2025, 01:31 PM UTC

   Connectivity issues appear to have been resolved. Still waiting for AWS to confirm what actions were taken to resolve the issue.

5. postmortem Sep 25, 2025, 10:19 AM UTC

   Our cloud provider was able to confirm they were having issues with an external network provider during this time period that caused connectivity issues between Europe and the us-west-2 region. This aligns with the issues that were reported to our support team. We will review our monitoring and alerting processes to determine if we can be alerted sooner to these types of regional cloud connectivity issues so that we can more quickly notify our customers.

Read the full incident report →

Started Sep 05, 2025, 10:45 AM UTC· resolved Sep 05, 2025, 11:13 AM UTC

1. investigating Sep 05, 2025, 10:45 AM UTC

   We are currently investigating a delay in processing triggers that kick off Workflow actions.

2. identified Sep 05, 2025, 11:01 AM UTC

   We have identified the problem as coming from our compute provider, AWS. An unplanned patch to our Event Stream is currently underway.

3. resolved Sep 05, 2025, 11:13 AM UTC

   Unplanned patches have been applies and our system has processed through the delayed events.

Read the full incident report →

Started Aug 06, 2025, 08:16 AM UTC· resolved Aug 06, 2025, 01:08 PM UTC

1. identified Aug 06, 2025, 08:16 AM UTC

   We've identified a problem with our telephony provider, Twilio, which incorrectly flags some (not all) accounts as compromised and thus disables their ability to make calls. We have escalated the problem and are working urgently with Twilio to resolve the issue.

2. monitoring Aug 06, 2025, 10:55 AM UTC

   Our telephony provider, Twilio, has fixed the accounts affected so far and is actively working on preventing the problem from reoccurring.

3. resolved Aug 06, 2025, 01:08 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jul 22, 2025, 07:34 PM UTC· resolved Jul 22, 2025, 08:18 PM UTC

1. monitoring Jul 22, 2025, 07:34 PM UTC

   Some customers may experience delayed updates to derived fields such as "Last Communication Date". The delay should be resolved soon.

2. resolved Jul 22, 2025, 08:18 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jun 19, 2025, 04:23 AM UTC· resolved Jun 19, 2025, 05:10 AM UTC

1. investigating Jun 19, 2025, 04:23 AM UTC

   Users are currently unable to load the application UI due to an issue with the serving infrastructure.

2. investigating Jun 19, 2025, 05:02 AM UTC

   We're continuing to investigate the issue.

3. monitoring Jun 19, 2025, 05:08 AM UTC

   Application is loading again, we're monitoring the situation.

4. monitoring Jun 19, 2025, 05:09 AM UTC

   We are continuing to monitor for any further issues.

5. resolved Jun 19, 2025, 05:10 AM UTC

   This incident has been resolved.

6. postmortem Jun 19, 2025, 06:01 PM UTC

   Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. ## Impact The Close Web and Desktop applications were intermittently failing to load for from June 18th 22:32 UTC to June 19th 05:10 UTC. The issue started with only a few errors with the frequency gradually increasing until it was resolved. ## Root Cause and Resolution A misconfigured version of the application was partially deployed due to mishandling of failures of a re-run CI/CD job. The issue was resolved by deploying a freshly built version of the application. We are improving our alerting systems to detect such incidents sooner and implementing safeguards into our build process to prevent misconfigured versions from occurring in the first place. ## Timeline * June 18th 22:32 UTC - an already deployed version of the application image gets misconfigured due to an issue in our build system * as time goes on and our infrastructure cycles resources which causes more and more traffic to get served by the misconfigured version, but not often enough to trigger our alerts * June 19th 04:17 UTC - Close Engineering team realizes there’s a problem, begins an investigation and starts taking mitigation steps * June 19th 05:10 UTC - a new version of the application is deployed resolving the issue

Read the full incident report →

Started May 14, 2025, 03:47 PM UTC· resolved May 14, 2025, 04:39 PM UTC

1. investigating May 14, 2025, 03:47 PM UTC

   We've identified an issue with our webhook processing system. Webhooks may be delayed up to twenty minutes. Updates will be posted as they are available.

2. identified May 14, 2025, 03:56 PM UTC

   This issue has been identified and a fix is being implemented.

3. monitoring May 14, 2025, 04:19 PM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved May 14, 2025, 04:39 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Apr 22, 2025, 02:25 PM UTC· resolved Apr 23, 2025, 08:46 AM UTC

1. resolved Apr 28, 2025, 05:54 PM UTC

   This incident has been resolved.

2. postmortem Apr 28, 2025, 05:56 PM UTC

   Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. ## Impact Between April 22nd 14:25 UTC and April 23rd 8:50 UTC our search functionality operated on outdated data for some users, which resulted in potentially incorrect search results. In addition, some webhooks with custom field updates were not delivered. No data was lost during the incident. ## Root Cause and Resolution A software update introduced a bug, where some events for custom field updates were malformed. As a result, the changes to custom fields weren’t reflected in search, and webhooks for these events were not sent. Close Engineering reverted the change, and reindexed all affected leads. We are adding guardrails which will prevent us from sending such malformed events in future. ## Timeline * 22 April 14:25 UTC - A software update is deployed that sends malformed events for some Custom Field updates * 22 April 21:20 UTC - Close Engineering is notified of leads not populating correctly in smart views * 23 April 04:36 UTC - Root cause is identified * 23 April 05:49 UTC - Measures applied to stop further impact * 23 April 08:46 UTC - Completed reindexing of affected leads

Read the full incident report →

Started Mar 25, 2025, 06:21 PM UTC· resolved Mar 25, 2025, 09:44 PM UTC

1. identified Mar 25, 2025, 06:21 PM UTC

   OpenAI are currently experiencing increased error rates on their side, which is impacting Lead Summaries and AI Search in Close. You can follow for updates on their status page here - https://status.openai.com/

2. resolved Mar 25, 2025, 09:44 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Mar 10, 2025, 04:02 PM UTC· resolved Mar 10, 2025, 04:28 PM UTC

1. investigating Mar 10, 2025, 04:02 PM UTC

   We've become aware of degraded performance of our Dialer service. We are investigating the issue. Updates will be posted as they become available.

2. monitoring Mar 10, 2025, 04:21 PM UTC

   We are continuing to investigate the cause of the degraded performance of our Dialer system. Our Dialer system is now functioning normally. We are monitoring performance.

3. resolved Mar 10, 2025, 04:28 PM UTC

   This incident has been resolved.

4. postmortem Mar 12, 2025, 05:39 PM UTC

   Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. ## Impact Dialer functionality was impaired for 58 minutes from 15:20 UTC to 16:18 UTC on March 10th 2025. During this time the Dialer feature could get stuck in “connecting” state. ## Root Cause and Resolution The issue was triggered at 15:20 UTC by a service rebalance that caused a number of client connections to close simultaneously. When these clients attempted to reconnect, the sudden spike in traffic that occurred in peak traffic conditions exceeded system limits, leading to service disruptions. Our team quickly identified the cause and worked to stabilize the system. We restored normal operations by 16:18 UTC. To prevent similar incidents in the future, we are reviewing system thresholds and improving our ability to handle sudden increases in demand. ## Timeline * 15:20 UTC - a service rebalance occurs, starting a wave of new connections being established * 15:21 UTC - a portion of requests starts getting dropped due to rate limits * 15:28 UTC - alerts trigger and our response team began identifying the root cause * 15:36 UTC - the rate of dropped requests subsides, but then increases again soon due to a wave-like pattern of retries * 16:18 UTC - final wave of increased errors finishes and situation returns to normal operational levels

Read the full incident report →

Started Feb 14, 2025, 09:11 PM UTC· resolved Feb 14, 2025, 10:05 PM UTC

1. investigating Feb 14, 2025, 09:11 PM UTC

   We have become aware of degraded performance of the Close system. We are actively triaging the issue. Updates will be posted as they become available.

2. identified Feb 14, 2025, 09:45 PM UTC

   The issue has been identified and a fix is being implemented. Search indexing may be delayed for some customers.

3. monitoring Feb 14, 2025, 09:54 PM UTC

   A fix has been implemented and we are monitoring the results. Search indexing may be delayed for some users.

4. resolved Feb 14, 2025, 10:05 PM UTC

   This incident has been resolved.

5. postmortem Feb 21, 2025, 07:42 PM UTC

   Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. ## Impact On February 14th, 2025 for two and a half hours \(between 19:50 UTC and 22:00 UTC\) some users saw outdated data in search results and smart views. No data was lost during the outage. ## Root Cause and Resolution A software update introduced malformed data to our Search platform that disrupted updates to our Search database. Close Engineering reverted the change and deleted the malformed data, which allowed the platform to resume processing updates to our Search database. The processing backlog was cleared by 22:00 UTC when full functionality was restored. We are implementing a new set of metrics and alerts that will help us identify this kind of failure more quickly. We are also enhancing the Search platform to better identify and discard malformed data to prevent similar incidents from occurring. ## Timeline All times are in UTC and events take place on 2025-02-14. * 19:50: A software update is deployed that sends malformed data to our search platform. * 20:33: Close Engineering is notified of an elevated error rate on the search platform and start investigating. * 21:04: Root cause is identified and work starts on a fix. * 21:40: The issue is resolved and all data is up-to-date.

Read the full incident report →