Lever experienced a critical incident on July 14, 2026 affecting Hire, lasting 9h 2m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jul 14, 2026, 02:47 PM UTC
Issue: We are investigating what appears to be a platform-wide issue in which offer letters fail to be sent. Customer Impact: Recruiters are currently unable to generate or deliver offer emails to candidates. Current Status: Engineering is actively investigating. No workaround is available at this time.
- monitoring Jul 14, 2026, 04:24 PM UTC
Issue: We identified a platform-wide issue in which offer letters failed to be sent. Customer Impact: Recruiters were unable to generate or deliver offer emails to candidates. Current Status: The immediate issue has been mitigated. We are actively monitoring to confirm full resolution and stability.
- resolved Jul 14, 2026, 11:50 PM UTC
Issue: We identified a platform-wide issue in which offer letters failed to be sent. Customer Impact: Recruiters were unable to generate or deliver offer emails to candidates. Status: Resolved. An RCA will be available within 7-10 business days.
- postmortem Jul 22, 2026, 01:20 PM UTC
**Incident Summary** Between approximately July 13 and July 14, 2026, some customers were intermittently unable to send offer letters, and some document/resume previews failed to generate. While source files were uploaded successfully, the conversion step required for previews and offer generation failed. No data loss occurred. **Detection** The issue was identified on July 14, 2026, at approximately 10:43 AM ET, following reports from customers to our Support team. Engineering confirmed the impact through monitoring, which showed a steady increase in process counts within the document-converter service until it reached a hard system limit, causing subsequent tasks to fail. **Root Cause** The incident was caused by a "process leak" introduced during a recent platform upgrade. A change in how the system handles document conversions meant that each conversion started a new background process that was never properly closed. Over several days, these background processes accumulated until the system reached its maximum capacity. Once this limit was hit, the service could no longer start the new processes required to generate offer letters or document previews. **Resolution** * **Initial Mitigation:** At 11:20 AM ET on July 14, engineers restarted the affected service pods. This cleared the accumulated background processes and immediately restored the ability for customers to send offer letters and view previews. * **Permanent Fix:** A code update was developed and deployed at 8:43 PM ET on July 14. This fix ensures that all background processes are correctly terminated and cleaned up after every conversion, preventing any future accumulation. **Preventative Measures** * **Code Improvements:** Implemented a robust process-management system \(using a "reaper" process\) to ensure all background tasks are automatically closed upon completion. * **Enhanced Monitoring:** New automated alerts have been configured to notify the engineering team if process usage exceeds 50% of the system limit. This will allow us to identify and resolve similar resource leaks days before they impact customers. * **Backlog Processing:** A separate initiative was completed to identify and reprocess any document or resume previews that failed to generate during the outage window to ensure all previews are available to users.