Lever experienced a major incident on January 6, 2026 affecting Hire, lasting 2h 59m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 06, 2026, 10:52 AM UTC
We are investigating an issue where resumes are not loading, offers are not displaying, and errors occur when manually adding resumes to opportunities. Our team is actively working to identify the root cause and restore normal functionality.
- monitoring Jan 06, 2026, 12:47 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jan 06, 2026, 01:51 PM UTC
This incident has been resolved.
- postmortem Jan 21, 2026, 03:26 PM UTC
We understand how important this functionality is to your hiring workflows, and our team moved quickly to investigate and address the issue. We can confirm this issue has been fully resolved, and our team has taken steps to ensure system stability moving forward. ## Customer Impact * On January 6th, 2026, between 15:07 and 18:20 \(UTC\), users were **unable to upload resumes or documents** in Lever ATS. * Upload attempts failed with an authorization-related error. * Any new uploads during the incident window may result in references to non-existent files within the ATS. Unfortunately, these files are not recoverable. * Other areas of the application continued to function normally. ## Root Cause The incident was caused by an updated **configuration in Amazon S3 access policies** following a security-related infrastructure change. As part of a security initiative, an existing IAM policy condition was updated in accordance with an AWS-recommended best practice. While this change was intended to improve security, the revised policy **did not behave as expected in our environment**, resulting in S3 denying permission for file uploads. Because file uploads rely on this S3 permission, customers encountered authorization errors when uploading resumes and documents. ## Resolution Once the issue was identified, the infrastructure team **reverted the policy change** to the previously functioning configuration. After the rollback: * File uploads resumed successfully * Error rates returned to normal levels * Customer impact was fully resolved The system was verified as restored by confirming: * Successful resume uploads from the application * Elimination of S3 authorization errors in logs and dashboards Invalid file reference cleanup * Our Engineering teams are in the process of removing any invalid references to files that were not successfully uploaded during this time ## Preventative Actions To reduce the likelihood of similar issues in the future, we are implementing the following improvements: 1. **Post-Deployment Monitoring** * Enhanced monitoring after infrastructure and security-related changes in production 2. **Automated Alerts** * Alerts for elevated file-upload failures and S3 authorization errors to reduce detection time 3. **Change Validation** * Additional validation and testing of IAM policy changes, especially those involving AWS-recommended updates 4. **Documentation & Review** * Improved documentation and cross-team review for security-driven infrastructure changes These steps will help detect issues sooner and minimize customer impact in the event of future changes.