ThinkOn experienced a notice incident on November 18, 2024 affecting CMH1 Object Storage, lasting 50d 4h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 18, 2024, 03:24 PM UTC
Our ThinkOn engineers are monitoring the system performance.
- monitoring Nov 19, 2024, 04:21 PM UTC
We still continue to monitor the system performance.
- monitoring Nov 20, 2024, 03:39 PM UTC
We are continuing to monitor for any further issues.
- monitoring Nov 22, 2024, 04:15 PM UTC
We continue to monitor HCP CS service performance at CMH1 site for any further issues.
- monitoring Nov 25, 2024, 03:26 PM UTC
We are continuing to monitor for any further issues.
- monitoring Dec 04, 2024, 03:52 PM UTC
Due to the emergency maintenance we had at 9:30AM to 10:00AM, we ask that you check your jobs as they might be in a hung state because a S3 gateway reboot was required. If the job is hung please restart/retry depending on the application j you are using.
- monitoring Dec 05, 2024, 06:22 PM UTC
Object storage system performance at CMH1 is degraded, so we are currently working with our vendor to address this issue.
- monitoring Dec 05, 2024, 10:52 PM UTC
We are continuing to monitor for any further issues.
- monitoring Dec 19, 2024, 10:52 PM UTC
S3 (Object storage) platform at CMH1 is unavailable. Our Engineers are working with the vendor and there is no ETA at the moment.
- monitoring Jan 07, 2025, 07:27 PM UTC
The S3 (Object Storage) platform at CMH1 is fully operational. Backups and restores for customers using our Object Storage services and DataProtect (Veeam O365) are functioning again.
- resolved Jan 07, 2025, 07:29 PM UTC
This incident has been resolved.