Service Disruption United States Region

Update timeline

1. investigating Dec 23, 2025, 04:30 PM UTC

   We are currently investigating a service disruption that is causing intermittent errors and slowness for all customers within the US region. This may also affect the functionality of the driver app. Our engineering team is actively working to identify the cause and restore full performance as quickly as possible. We do not have a specific timeline for resolution yet, but we are committed to sharing regular progress. We will provide another update within the next 15 minutes or as soon as more information becomes available. Thank you for your patience as we work to resolve this.

2. monitoring Dec 23, 2025, 04:40 PM UTC

   We have successfully mitigated the service disruption affecting the United States (US) region, and we are now seeing significant recovery across our systems. Most customers should experience immediate improvement in performance The issue began at 8:16AM PT and was successfully mitigated by 8:29AM PT. All systems are now stable, and features are performing as expected. If you continue to see any lingering effects, please perform a hard refresh or restart your app. We appreciate your patience while our team worked to restore service. Our team will continue to monitor the platform closely for stability before officially closing the incident. We sincerely appreciate your patience and will provide a final update once the situation is fully resolved.

3. monitoring Dec 23, 2025, 04:47 PM UTC

   The service disruption affecting the United States (US) region is now fully resolved. Between 8:16 AM and 8:29 AM PT, users may have experienced slowness, intermittent errors, or difficulties while using the driver app. Systems are now stable, and you should see immediate improvement. If you continue to experience any issues, please perform a hard refresh of your browser or app. We appreciate your patience while we worked to restore full service.

4. resolved Dec 23, 2025, 04:48 PM UTC

   The service disruption affecting the United States (US) region is now fully resolved. Between 8:16 AM and 8:29 AM PT, users may have experienced slowness, intermittent errors, or difficulties while using the driver app. Systems are now stable, and you should see immediate improvement. If you continue to experience any issues, please perform a hard refresh of your browser or app. We appreciate your patience while we worked to restore full service.

5. postmortem Dec 24, 2025, 05:43 PM UTC

   # Spare Incident Report December 24, 2025 ## Executive Summary Between December 21 and December 23, 2025, the Spare platform experienced brief service disruptions impacting the US region. Users may have experienced intermittent error messages and system slowness during several distinct time windows. The disruption was caused by resource contention within our primary database infrastructure, which led to connection pool exhaustion. This prevented backend services from successfully processing queries. Our engineering team intervened immediately during the initial event to restore service. Subsequent brief recurrences were handled by automated recovery systems while a permanent fix regarding database configuration was explored and some changes to the system were implemented. ## Incident Details The incident began on Sunday, December 21, at 8:00 am PST when internal monitoring systems alerted our engineering team to a spike in database connection errors and increased latency in the US region. ### **Cause** The primary root cause for this incident was the exhaustion of the database connection pool, preventing the platform from establishing necessary connections to the database. Upon deeper investigation, it was determined that specific database triggers were acquiring exclusive locks on key resources. This locking behavior created a bottleneck, leading to query backups and eventual exhaustion of available connections. ### **Mitigation and Resolution** Our team’s response focused on immediate service restoration, data integrity verification, and root cause elimination: 1. **Immediate Restoration:** During the initial incident, the engineering team restarted backend services to reset database connections and clear internal caches, restoring full service. 2. **Data Integrity Check:** While our priority was service restoration, we conducted a thorough review out of an abundance of caution to ensure data integrity. We confirmed that all data remained secure and accurate throughout the incident. 3. **Root Cause Elimination:** During the subsequent brief recurrences on December 22 and 23, the team gathered necessary diagnostic data to identify the problematic database triggers. These triggers have been removed to eliminate the resource locking issue and prevent recurrence. ## Timeline _All times are in Pacific Time \(PST\)._ * **Dec 21, 08:00 am:** Monitoring systems detect elevated error rates and latency in the US region. * **Dec 21, 08:03 am**: Team begins incident investigation. * **Dec 21, 09:10 am:** Service is fully restored following a restart of backend services. * **Dec 22, 05:30 am:** A recurrence of connection pool exhaustion is detected. * **Dec 22, 05:38 am:** System auto-recovers as backend services stabilize; diagnostic data is gathered. * **Dec 22, 10:15 am:** A second minor recurrence is detected. * **Dec 22, 10:21 am:** System auto-recovers and normal service resumes. * **Dec 22, 1:35 pm:** Root cause identified \(database triggers\) and implementation of fix begins. * **Dec 23, 8:16 am**: A third minor recurrence occurs. * **Dec 23, 8:29 am**: System auto-recovers and normal service resumes. * **Dec 23, 9:26 am**: Fix rolled out to all systems in all regions. ## Next Steps To ensure the continued reliability of our platform and prevent similar resource contention issues, we are undertaking the following: * **Database Optimization:** We have removed the problematic triggers and are reviewing all database locking mechanisms to ensure optimal concurrency. * **Enhanced Monitoring:** We are implementing stricter alerts regarding database connection pool usage to detect potential contention before it impacts user experience. * **Cache Resiliency:** We are reviewing our caching logic to ensure it recovers more gracefully during periods of backend instability. We appreciate your understanding as we worked through this incident. We remain vigilantly focused on continuous improvement of our system’s stability and availability.