[Critical] Customers may have issues with Uploads, Downloads, and accessing the All Files page

Update timeline

1. investigating Feb 19, 2025, 08:07 PM UTC

   We are currently investigating this issue.

2. investigating Feb 19, 2025, 08:09 PM UTC

   Starting at approximately 11:20AM PST, customer may be seeing issues with Uploads/Downloads and slowness when accessing their All Files page. We are actively investigating this issue and will be providing updates as we work to resolve. Our team is investigating an issue with Uploads/Downloads/API errors when utilizing Box. Users may see errors or slowness when attempting to upload/download items to Box. We will provide additional information as it becomes available.

3. identified Feb 19, 2025, 08:14 PM UTC

   Our team has identified the underlying cause of this issue and is working to take remediating steps. We will provide additional updates as they become available.

4. monitoring Feb 19, 2025, 08:20 PM UTC

   Our team has taken steps to remediate this issue and is seeing improvement for Uploads/Downloads/API service. We are continuing to monitor for any additional impact.

5. monitoring Feb 19, 2025, 09:24 PM UTC

   Our team has taken steps to remediate this issue and is seeing improvement for Uploads/Downloads/API service. We are continuing to monitor for any additional impact.

6. monitoring Feb 19, 2025, 10:20 PM UTC

   All services should now be returned to full functionality. We are continuing to monitor for any additional impact.

7. monitoring Feb 19, 2025, 11:31 PM UTC

   All services should now be returned to full functionality. We are continuing to monitor for any additional impact.

8. resolved Feb 20, 2025, 12:12 AM UTC

   After further monitoring, this incident is now considered resolved. All services have been restored to full functionality. If you continue to experience any issues, please contact Box Support at https://support.box.com.

9. postmortem Feb 20, 2025, 07:26 PM UTC

   We recently addressed issues affecting Box public APIs, Shared Links and Box Notes. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future. Between 11:18 AM and 12:15 PM PST on Feb 19th, 2025, some users may have experienced difficulties while working in Box. During this period, there was a noticeable degradation in the following services: Public API, Login, Box Sign, Shared Links, and Notes. Due to a gap in the procedure related to the management of application authentication keys, this led to a deletion of keysets that were mistakenly believed to be no longer necessary. The impact of this was detected through a drop in Box’s User Availability metrics at 11:15am PST and by 11:52am PST. This drop was traced to the operation of the deletion of these keys. ‌ **Analysis** The deleted key is important for token-based authentication between the application and the database which stores security rules to validate Box upstream services. The deletion of this key caused the authentication failure of the application with the database and prohibited the application from executing properly. Additionally, after reviewing our protocol around customer awareness of issues, we realize that our timeliness goal for posting to our Status site was not met. ‌ **Corrective Actions** Box has initiated the following corrective actions: * Migrate application authorization to a centralized authentication process / service. * Limit visibility of application account key read/write access on the administrator console. * Update necessary runbooks with correct procedures. * Review and streamline Status posting events to ensure timely and consistent communication. ‌ We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter. Sincerely, The Box Team