Integrate.io had 30 outages in the last 2 years totaling 1622h 19m of downtime — averaging 1.2 incidents per month.
There were 30 Integrate.io outages since August 14, 2024 totaling 1622h 19m of downtime. Each is summarised below — incident details, duration, and resolution information.
We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
The incident has been resolved.
We are currently investigating this issue
The incident has been resolved.
We are currently investigating the issue.
The issue has been identified and it's due to our upstream provider and the affected scope is for newly created clusters. New cluster creation and job runs should now be working. We are working on unblocking clusters that are stuck on Creating.
The incident has been resolved.
Issue - There’s a Salesforce service disruption. During a service disruption, end users can’t access the service. Please refer to more on Salesforce status - https://status.salesforce.com/incidents/20004038 Affected integrate.io jobs having the Salesforce connectors (Source/destination) are failing with an error message : Error Occurred While Authenticating: { "error" : "system_down", "error_description" : "system may be currently unavailable" }
Please be advised that the Salesforce incident has been fully resolved. You can view the official status update here: https://status.salesforce.com/incidents/20004038 We kindly ask that you check your jobs utilizing Salesforce connectors; all integrations should now be fully functional. Should you have any further questions or continue to experience issues, please do not hesitate to reach out to us at [email protected]
We've updated the credentials for our Google integration as part of a security rotation. If you have a Google Ads, Google Analytics, Google Sheets or Google Drive connection, you may need to reconnect it: - Go to your Connections page - Open the affected connection - Click Reconnect and follow the Google authorization prompt If you have any trouble, our support team is here to help.
This incident has been resolved. If you encounter any issues related to Google Ads, GA analytics, or Google Sheets/Drive, please follow the steps suggested in this incident.
We are currently investigating this issue
The issue has been identified and a fix has been implemented. Please rerun the jobs on a newly created cluster.
The incident has been resolved.
The issue has been identified with our upstream provider and a fix is being implemented.
A fix has been implemented by our upstream provider and services are starting to recover. We are monitoring the results.
The incident has been resolved.
We have identified an issue causing job failures with the BigQuery V1 connector. This was introduced by a recent library upgrade that resulted in a regression affecting the connector. We are actively working on a fix. In the meantime, customers may migrate to the BigQuery V2 connector by following our documentation or by reaching out to our support team for assistance. We apologize for the inconvenience and appreciate your patience.
A fix has been implemented, and we are actively monitoring the results to ensure stability. Please proceed with running the jobs on a newly created cluster. This issue impacts packages that include the ExecuteSQL component and BigQuery V1. We apologize for the inconvenience and appreciate your patience.
This incident has been resolved.
We are currently investigating the issue.
The issue has been identified with the upstream provided and we are continuing to work with them to fix.
We’re seeing an improvement in our applications and currently working on restoring processing engine workloads.
We are continuing to recover the processing engine workloads and jobs should now be working moving forward.
The incident has been resolved. We will be writing a post-mortem report soon.
Jobs are partially failed to be executed on the clusters due to an issue from our upstream providers. The issue was resolved at Nov 21, 07:30 UTC.
Due to AWS's incident the following components are still impacted, SFTP To Go's audit logs and Cron To Go's processing engine For more details please see below AWS status- https://health.aws.amazon.com/health/status
This incident has been resolved.
We are currently investigating this issue.
A fix has been implemented and we are monitoring the results. Job workloads aren't affected and should still run as normal.
The incident has been resolved.
We are currently investigating the issue.
A fix has been implemented and we are monitoring the results. We apologize for the inconvenience caused.
The incident has been resolved.
Hi All, We are experiencing an issue with our Reverse SSH connections, and the jobs are failing. Action: Investigating the issue
The issue has been identified, and please find the fix below to resolve the connectivity issue. Please delete/refresh the entry on your known_hosts file. You can do either of the following: Delete the specific line on ~/.ssh/known_hosts file which contains virginia-tunnel.xplenty.com. Make sure to save. Execute ssh-keygen -R "[virginia-tunnel.xplenty.com]:50683" on the tunnel instance
RCA: It looks like an issue one of our upstream providers. Follow the steps shared to resolve the issue. Detailed RCA will be shared soon.
RCA: Initial findings: There seems to have been an issue with our upstream provider, which we're working with them to identify why. The above solution resolves the issue.
We are currently investigating the issue which is causing the dashboard to return intermittent errors.
A fix has been implemented and we are monitoring the result. We apologize for the inconvenience caused here.
The incident has been resolved. We apologize for the inconvenience caused.
We are currently investigating an issue which caused jobs to be stuck on pending.
A fix has been implemented and we are monitoring the results.
The incident has been resolved.
Customers may experience issues with jobs that read data from Bing Ads. Our engineers are investigating the issues.
A fix has been implemented. Please reconnect your Bing Ads connection and rerun the jobs on a newly provisioned cluster.
The incident has been resolved.
We are seeing some clusters stuck on Creating on the Virginia region and have identified that it's due to an issue with our upstream provider. Will provide updates as soon as we have it.
Clusters are now being created. We are continuing to monitor the issue.
The incident has been resolved.
From Feb 24th, 14:30 UTC to 19:41 UTC, an issue with our job notification service caused jobs to skip the Running state and display a blank runtime. While Dataflow packages may have run normally (pending further confirmation), Workflows were affected, leading to job failures. The issue has now been resolved. If needed, please rerun the jobs or use "Run One-Off" on the schedule page. A post-mortem will be released soon. We sincerely apologize for the inconvenience.
From 10:25 UTC to 13:00 UTC, job failures occurred in packages using the file storage destination component due to a faulty deployment. The issue has been resolved. We apologize for any inconvenience this may have caused.
We are aware of an intermittent issue affecting the dashboard. If you encounter this problem, please try refreshing the page. Our development team is actively investigating the root cause and will implement a fix as soon as possible. We apologize for the inconvenience and appreciate your patience.
This incident has been resolved.
Our API is currently down due to an emergency maintenance of our upstream provider. We will update shortly
This incident has been resolved.
There was an issue with our SSH server in the Oregon region, and the dev team applied the fix as of 21:40 UTC
We are currently investigating this issue.
The issue was due to our upstream provider. The incident has been resolved.