ftrack experienced a major incident on May 8, 2019 affecting Hosted applications, lasting 9h 5m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 08, 2019, 01:56 AM UTC
Hosted instances in the Asia/Pacific region are experiencing connectivity issue. We're working with our hosting provider to restore full capacity as soon as possible.
- identified May 08, 2019, 03:32 AM UTC
Hosted instances in the Asia/Pacific region are experiencing connectivity issue. We're working to restore connectivity as quickly as possible.
- monitoring May 08, 2019, 05:14 AM UTC
Instances are accessible again and we continue to monitor the situation.
- resolved May 08, 2019, 11:02 AM UTC
This incident has been resolved.
- postmortem May 08, 2019, 03:42 PM UTC
# Connectivity issues in Asia/Pacific region ## Summary Users hosted in the Asia region experienced downtime with their ftrack service. The outage resulted in the service being completely inaccessible for customers hosted in the Asia-Pacific region. The outage affected the service for three and a half hours. ## Cause The outage was caused by a system failure in a service managed by the cloud provider \(Google\). An automatic repair was initiated but failed to repair the service. ## Resolution ftrack was able to work around the issue and made the service operational again. The root cause was later resolved by the cloud provider. ## Post-mortem Google cloud platform has committed to conduct an internal investigation of this issue and make appropriate improvements to its systems to help prevent or minimize future recurrence. ftrack will also investigate and take action to minimize the risk of cloud provider outages in the future. We will also look over our internal processes and policies to ensure information about an outage is available to our customers on our status page \(status.ftrack.com\) as soon as possible during an outage. ## Outage history All times are in CEST. 2019-05-08 01:59 AM - Internal service failure detected. 2019-05-08 02:14 AM - ftrack service outage started. 2019-05-08 05:45 AM - ftrack service available again. 2019-05-08 10:19 AM - The root issue was resolved by the cloud provider \(Google\).