Close experienced a major incident on September 5, 2023 affecting Application UI and API, lasting 55m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Sep 05, 2023, 04:08 PM UTC
Customers may be experiencing elevated error rates. The root cause has been identified. A fix is being deployed.
- monitoring Sep 05, 2023, 04:17 PM UTC
The fix has been deployed. The Close application is now functioning normally. Close Engineering is continuing to monitor application performance.
- resolved Sep 05, 2023, 05:04 PM UTC
This incident has been resolved.
- postmortem Sep 05, 2023, 09:35 PM UTC
Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. ## Impact Close users experienced elevated error rates from 16:00 until 16:16 UTC on September 5, 2023. During this time requests may have presented an error or timed out. ## Root Cause and Resolution At 16:00 UTC on September 5, 2023 a version of the Close app with a major performance issue was deployed to production. This resulted in degraded performance for all customers from 16:00 until 16:16 UTC. Close Engineering deployed a new version of the the Close app that did not have the performance issue at 16:16 UTC. Application performance then returned to normal. Close Engineering is working on a new set of pre-deployment tests that will prevent similar performance issues from being introduced in the future. ## Timeline * 2023-09-05 16:00 - A new version of the Close app is published. * 2023-09-05 16:03 - Close Engineering becomes aware of degraded performance of the Close application. * 2023-09-05 16:05 - Close Engineering reverts to the last working version of the Close application. * 2023-09-05 16:16 - The working version of the Close application is deployed. Performance recovers.