ThinkOn had 50 outages in the last 2 years totaling 471h 17m of downtime — averaging 2.1 incidents per month.
There were 50 ThinkOn outages since November 18, 2024 totaling 471h 17m of downtime. Each is summarised below — incident details, duration, and resolution information.
CGY4 is currently experiencing network connectivity issues.
vCloud Access is Unavailable
This incident has been resolved.
We are currently investigating this issue.
We are continuing to investigate this issue.
This issue has been resolved. We are observing successful job completions.
We are currently investigating the issue. ThinkOn engineers are working with the vendor (veeam support) to resolve the issue.
We are continuing to investigate this issue.
We are continuing to investigate the issue. ThinkOn engineers are working with the vendor (Veeam support) to resolve the issue.
Our engineering team has successfully resolved the issue. If you notice failures after today please reach out to the ThinkOn Service Desk.
We are currently experiencing a service disruption affecting our Veeam M365 service hosted in the CGY4 data centre. Our engineering team is actively investigating the issue and working to restore services as quickly as possible. At this time, you may experience interruptions or delays when accessing or using the service. We understand the importance of this service to your operations and appreciate your patience while we work toward resolution. Thank you for your understanding."
We are continuing to investigate a service disruption affecting our Veeam M365 service in the CGY4 data center. Service interruptions or delays may persist. We appreciate your continued patience as we work toward resolution.
We are continuing to investigate a service disruption affecting our Veeam M365 service in the CGY4 data center. Service interruptions or delays may persist. We appreciate your continued patience as we work toward resolution.
We have isolated the issue and are currently working toward a solution. We’ll provide further updates as soon as we have more information.
We are continuing to work towards a resolution.
Our team has confirmed the issue has been resolved and systems are now operational.
We are currently investigating an issue affecting vCloud accessibility. At this time, we do not believe any workloads are impacted. Our team is working diligently to restore access as quickly as possible.
This incident has been resolved.
We are currently experiencing connectivity issues at our Ottawa Data Center. As a result, some customers may be experiencing service disruptions at this time. Our technical teams are actively investigating and working to restore full service as quickly as possible. We will continue to provide updates as more information becomes available. We appreciate your patience and understanding.
This incident has been resolved.
We are currently experiencing an issue affecting access to our Critical Compute environment, which may impact connectivity to vCloud services. At this time, there is no indication that customer workloads are impacted. Our team is actively investigating and working to restore full functionality as quickly as possible. We understand the importance of these services and appreciate your patience while we resolve the issue. Further updates will be provided as more information becomes available. If you have any urgent concerns, please reach out to our support team.
This incident has been resolved.
We are currently experiencing an issue accessing the vCloud portal. Our team is actively investigating the problem.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently experiencing a network issue impacting Zerto replication, Veeam backup operations, and overall system performance. Engineering teams are actively investigating and working to mitigate the issue.
We are still experiencing slowness with Zerto, VCC, VBO and we are still investigating. Engineering teams are actively investigating and working to mitigate the issue.
Performance has improved across affected services, including Zerto, VCC, VBO, and overall system operations. Customer reports and system metrics indicate services are stabilizing. ThinkOn engineers will continue to monitor closely to ensure stability is maintained. Updates will be provided if required.
This incident has been resolved
We are currently experiencing an issue accessing the Data Protect portal. Our team is actively investigating the problem.
The service disruption with the Data protect portal is now resolved.
Please be advised that we are currently troubleshooting a network situation: We are currently investigating a network issue at CGY4. While we troubleshoot and address the root cause, the connection to a few services may become intermittently, such as the vCloud portal and Zerto replications. We apologize for any inconvenience this may cause and appreciate your patience while we work to resolve the problem. Further updates will be provided as they become available.
This Incident has been resolved.
We are receiving multiple reports of service issues at the MTL2 site. Our team is actively working to identify the scope and root cause. Updates to follow.
We are continuing to investigate this issue.
We are working on resolving the issue.
Services have been restored.
We are currently aware of an issue that has caused network instability within our vCloud environments Our team believes they have resolved this issue - but we are monitoring to ensure stability and performance is as expected.
We are continuing to monitor for any further issues.
This incident has been resolved.
We are currently investigating this issue.
We are continuing to investigate this issue.
Primary link connectivity came back. An investigation has begun as to why the fail over did not function as intended.
We are currently experiencing network connectivity issues that may impact service availability and performance for some customers. Our engineering and operations teams are actively troubleshooting the issue and working to restore normal service levels as quickly as possible. We will continue to provide updates as more information becomes available. Thank you for your patience while we work to resolve this.
The Network service degradation has been resolved.
Object Storage is currently inaccessible.
We are continuing to investigate this issue.
Issue has been resolved. All the OBJECT STORAGE in MEL3 is accessible.
We are currently experiencing an issue where the Data Protect Portal is inaccessible. Our engineering team is actively investigating the root cause and working to restore access as quickly as possible.
This incident has been resolved.
We’ve identified an issue in the underlying infrastructure that is currently causing Virtual Machine creation to fail. Our team is actively working on a fix, and we expect the issue to be resolved shortly. We will provide an update within the next hour.
This incident has been resolved.
We are currently Investigating a Burning Smell coming from our Electrical Rooms. Contractors are currently onsite and are investigating this issue. We have Transferred the Critical Loads to Generator Power. Electricians are being dispatched to review the issue. NO customer equipment or services are impacted and our team will monitor the activity closely.
**UPDATE** The faulty breaker has been replaced successfully. Site loads have been transferred back to hydro power. There was no impact to customer equipment or services during the process.
Normal operations have been fully restored.
We are currently working with a vendor to resolve this issue.
We are currently working with a vendor to resolve this issue.
Update: We are actively collaborating with our vendor to resolve the ongoing issue. The following services are currently impacted: - Virtual Compute & vCloud - Zerto Virtual Replication - Object Storage - Compass Portal We appreciate your patience and will provide further updates as progress is made.
We are actively collaborating with our vendor to resolve the ongoing issue. The following services are currently impacted: - Virtual Compute & vCloud - Zerto Virtual Replication - Object Storage - Compass Portal We appreciate your patience and will provide further updates as progress is made.
We are currently in the process of restoring object storage, vCloud, and other affected services. Our teams are actively working to bring systems back online as quickly and safely as possible. Next Update: Scheduled in 90 minutes Thank you for your continued patience. We’ll share further details as progress continues.
- Storage for dedicated blades has been successfully restored. However, some dedicated blades may require a manual reboot to fully reconnect to storage services. - All affected tenant VMs are now back online and operational. - Our teams are continuing work to restore vCloud access, and we will provide updates as progress continues. Thank you for your patience as we work toward full service recovery.
We’re pleased to report that all systems are now back online, including storage, vCloud, and all affected virtual machines (VMs) and physical servers. What You Should Know: -Some virtual machines and dedicated blades may require a manual reboot to reconnect fully to storage. -If you experience any issues accessing your systems, please contact the Service Desk. We sincerely apologize for the inconvenience and appreciate your patience throughout the recovery process.
We are currently experiencing a network outage at our CGY3 & CGY5 site. Our ThinkOn engineers are working with the vendor to find a resolution for this issue.
Thinkon Team is continuing to troubleshoot with the Vendor.
Thinkon Team continues troubleshooting the issue with the Vendor
ThinkOn engineer worked with the vendor to resolve the issue.
We are currently investigating the issue.
We are currently working with a vendor to resolve the issue.
We continue working with a vendor to resolve the issue.
Our ThinkOn engineers are continuing to work with the vendor to resolve the issue.
Our vendor has dispatched field technicians to the site, and repair work is underway.
Fibre teams have successfully lashed the damaged fibers on both ends One cable has been fully spliced The fibres on the other end are currently being spliced
The fibre has been repaired and services have come back online. We are currently monitoring the network.
This incident has been resolved.
The DC team is currently investigating the power disturbance at the Data Centre.
We would like to inform you that the recent power disturbance at our Kamloops Data Centre has been fully resolved. All systems are now stable and operational.