Instaclustr Outage History

Instaclustr is up right now

Instaclustr had 37 outages in the last 2 years totaling 173h 5m of downtime — averaging 1.5 incidents per month.

There were 37 Instaclustr outages since June 12, 2024 totaling 173h 5m of downtime. Each is summarised below — incident details, duration, and resolution information.

Source: https://status.instaclustr.com

Major November 25, 2024

ClickHouse provisioning temporarily unavailable

Detected by Pingoru
Nov 25, 2024, 04:12 AM UTC
Resolved
Nov 25, 2024, 05:41 AM UTC
Duration
1h 29m
Affected: Management ConsoleCluster Management API
Timeline · 3 updates
  1. investigating Nov 25, 2024, 04:12 AM UTC

    Provisioning of ClickHouse clusters is currently unavailable. This affects provisioning through the Console and through the Instaclustr Provisioning API. Provisioning of all other offerings is unaffected and no impact on existing running clusters including ClickHouse.

  2. identified Nov 25, 2024, 04:18 AM UTC

    We have identified the root cause and the team is currently working on a fix. Further updates will be provided as they become available.

  3. resolved Nov 25, 2024, 05:41 AM UTC

    This incident has been resolved.

Read the full incident report →

Major November 22, 2024

Customers unable to create new clusters via console in certain configurations

Detected by Pingoru
Nov 22, 2024, 12:20 AM UTC
Resolved
Nov 22, 2024, 03:24 AM UTC
Duration
3h 3m
Affected: Management Console
Timeline · 3 updates
  1. identified Nov 22, 2024, 12:20 AM UTC

    Users of the Google Cloud Platform (GCP) Marketplace are currently unable to create new clusters via the management console, existing clusters and other cloud services (Amazon Web Services, Azure and Google Cloud) are unaffected. Users of GCP Marketplace can still use our Cluster Management API or Terraform provider to provision new clusters. We will provide updates as soon as they become available. If you have an urgent matter, please reach out to us via the support portal (https://support.instaclustr.com) or email ([email protected]).

  2. identified Nov 22, 2024, 12:58 AM UTC

    We have identified further configurations that cannot be provisioned via the managed console, these are: - Provisioning Kafka with Tiered Storage is not possible (all cloud providers) - Provisioning OpenSearch with Searchable Snapshots on Google Cloud Platform is not possible - Provisioning OpenSearch with Cross-cluster Replication is not possible (all cloud providers) We will provide updates as soon as they become available. If you have an urgent matter, please reach out to us via the support portal (https://support.instaclustr.com) or email ([email protected]).

  3. resolved Nov 22, 2024, 03:24 AM UTC

    This incident has been resolved.

Read the full incident report →

Notice September 16, 2024

Malformed Email Notifications

Detected by Pingoru
Sep 16, 2024, 11:26 PM UTC
Resolved
Sep 18, 2024, 01:26 AM UTC
Duration
1d 2h
Affected: Management Console
Timeline · 3 updates
  1. identified Sep 16, 2024, 11:26 PM UTC

    We have identified an issue where some email notifications are displaying malformed messages. As a result, customers may receive email notifications that are not properly formatted, potentially affecting readability and the ability to understand the content. New and existing customer clusters are unaffected. The Management Console, Cluster Management API, Monitoring API, Prometheus API and Support Portal are still operational. Additional details will be provided as they become available. If you are affected by this issue, please contact us via the support portal (support.instaclustr.com) or email ([email protected]) for assistance and we will provide a workaround.

  2. identified Sep 17, 2024, 06:05 AM UTC

    We have identified the root cause and the team is currently working on a fix. Further updates will be provided as they become available.

  3. resolved Sep 18, 2024, 01:26 AM UTC

    This incident has been resolved.

Read the full incident report →

Notice September 16, 2024

Delays in Sending Customer Notification Emails

Detected by Pingoru
Sep 16, 2024, 05:39 AM UTC
Resolved
Sep 16, 2024, 06:59 AM UTC
Duration
1h 20m
Affected: Management Console
Timeline · 2 updates
  1. identified Sep 16, 2024, 05:39 AM UTC

    We are currently investigating issues with the outbound email function in our Management Console. As a result, our system is experiencing delays in sending customer notifications via email. This includes account verification and password reset emails. Our team is actively investigating the issue and we apologise for any inconvenience caused. New and existing customer clusters are unaffected. The Management Console, Cluster Management API, Monitoring API, Prometheus API and Support Portal are still operational. Additional details will be provided as they become available. If you are affected by this issue, please contact us via the support portal (support.instaclustr.com) or email ([email protected]) for assistance and we will provide a workaround.

  2. resolved Sep 16, 2024, 06:59 AM UTC

    This incident has been resolved.

Read the full incident report →

Minor August 27, 2024

Private network Redis cluster provisioning issues

Detected by Pingoru
Aug 27, 2024, 06:03 AM UTC
Resolved
Aug 28, 2024, 11:43 PM UTC
Duration
1d 17h
Affected: Management ConsoleCluster Management API
Timeline · 3 updates
  1. investigating Aug 27, 2024, 06:03 AM UTC

    Creating a Redis cluster with the Enterprise feature Private Network Cluster, will fail to provision the cluster. We are currently investigating the issue. There is no impact to existing Redis clusters. Additional details will be provided as they become available. If you have an urgent matter, please reach out to us via the support portal (support.instaclustr.com) or email ([email protected]).

  2. identified Aug 28, 2024, 03:04 AM UTC

    We have identified the root cause of the issue and are currently testing a fix. We anticipate releasing the fix within the next few hours. Thank you for your patience and understanding as we work to resolve this matter. We will provide another update as soon as the fix is deployed. If you have an urgent matter, please reach out to us via the support portal (support.instaclustr.com) or email ([email protected]).

  3. resolved Aug 28, 2024, 11:43 PM UTC

    We are pleased to inform you that the issue affecting the provisioning of Redis clusters with the Enterprise feature "Private Network Cluster" has been resolved. The fix has been successfully deployed and tested. We appreciate your patience and understanding as we worked to resolve this matter. If you experience any further issues or have any questions, please do not hesitate to reach out to us via the support portal (support.instaclustr.com) or email ([email protected]).

Read the full incident report →

Notice August 20, 2024

Support Email Service Degradation

Detected by Pingoru
Aug 20, 2024, 11:25 PM UTC
Resolved
Aug 21, 2024, 06:43 AM UTC
Duration
7h 18m
Timeline · 2 updates
  1. identified Aug 20, 2024, 11:25 PM UTC

    We are aware that some users may be experiencing issues sending emails to our support email ([email protected]) due to a problem with an upstream provider. Our team is currently investigating the problem and we apologise for any inconvenience caused. New and existing customer clusters are unaffected. Additional details will be provided as they become available. If you have an urgent matter, please create a ticket through support.instaclustr.com.

  2. resolved Aug 21, 2024, 06:43 AM UTC

    This incident has been resolved.

Read the full incident report →

Notice August 20, 2024

Support Email Unavailable

Detected by Pingoru
Aug 20, 2024, 05:51 AM UTC
Resolved
Aug 20, 2024, 11:34 AM UTC
Duration
5h 42m
Timeline · 3 updates
  1. investigating Aug 20, 2024, 05:51 AM UTC

    We are aware that some users may be experiencing issues sending emails to our support email ([email protected]) due to a problem with an upstream provider. Our team is currently investigating the problem and we apologise for any inconvenience caused. New and existing customer clusters are unaffected. Additional details will be provided as they become available. If you have an urgent matter, please create a ticket through support.instaclustr.com.

  2. identified Aug 20, 2024, 06:01 AM UTC

    The issue has been identified and we're waiting on a fix being rolled out by our upstream provider.

  3. resolved Aug 20, 2024, 11:34 AM UTC

    The fix from the upstream provider has been rolled out, and the issue has been resolved.

Read the full incident report →

Minor August 20, 2024

Service Disruptions for Deprecated Provisioning API v1

Detected by Pingoru
Aug 20, 2024, 05:10 AM UTC
Resolved
Aug 22, 2024, 06:27 AM UTC
Duration
2d 1h
Affected: Cluster Management API
Timeline · 3 updates
  1. investigating Aug 20, 2024, 05:10 AM UTC

    We are currently facing ongoing degradation of service on our deprecated provisioning v1 API, which is affecting both direct API calls and interactions with Terraform v1. Our team is actively working to resolve the issue and we apologise for any inconvenience caused. Existing customer clusters are unaffected. The Management Console and Cluster Management API are still operational. We will provide updates as soon as they become available. If you have an urgent matter, please reach out to us via the support portal (support.instaclustr.com).

  2. investigating Aug 22, 2024, 03:08 AM UTC

    We have increased our rate limits for our provisioning v1 API to 350 requests/minute and are currently working towards adjusting it further with the goal of restoring to original rate limits. Rate limits for Terraform Provider v1 usage have been completely restored to the original limits of 750 requests/minute. Existing customer clusters and the Management Console continue to be unaffected. We will provide further updates as we continue to restore our rate limits. If you have an urgent matter, please reach out to us via the support portal (support.instaclustr.com).

  3. resolved Aug 22, 2024, 06:27 AM UTC

    This incident has been resolved. Rate limits for the provisioning v1 API have been restored to the original limit of 750 requests/minute. All Instaclustr APIs are operating as expected with standard API rate limits.

Read the full incident report →

Minor August 19, 2024

Service Disruptions for Management Console and Cluster Management API

Detected by Pingoru
Aug 19, 2024, 03:04 AM UTC
Resolved
Aug 19, 2024, 07:24 AM UTC
Duration
4h 20m
Affected: Management ConsoleCluster Management API
Timeline · 3 updates
  1. investigating Aug 19, 2024, 03:04 AM UTC

    We are currently investigating elevated latencies affecting our Management Console and Cluster Management API. Our team is actively working to resolve the issue and we apologise for any inconvenience caused. Existing customer clusters are unaffected. Our Monitoring API and Prometheus API is unaffected. We will provide updates as soon as they become available. If you have an urgent matter, please reach out to us via the support portal (support.instaclustr.com) or email ([email protected]).

  2. investigating Aug 19, 2024, 03:09 AM UTC

    We are continuing to investigate this issue.

  3. resolved Aug 19, 2024, 07:24 AM UTC

    A fix has been applied, and latencies on both the Management Console and Cluster Management API have returned to baseline levels.

Read the full incident report →

Major July 19, 2024

Azure Central US Region Operational Issues

Detected by Pingoru
Jul 19, 2024, 12:25 AM UTC
Resolved
Jul 19, 2024, 06:27 AM UTC
Duration
6h 1m
Affected: Management ConsoleMonitoring APICluster Management APIPrometheus API
Timeline · 3 updates
  1. identified Jul 19, 2024, 12:25 AM UTC

    We are currently experiencing service degradation and potential disruptions in the Azure Central US region. This incident is a result of an ongoing issue with the cloud provider's infrastructure. You will be unable to provision new clusters or nodes in Azure Central US region. All nodes in the Azure Central US region are unavailable. Node resize and replace, Monitoring API and Prometheus API for nodes in Azure Central US region are also affected. Individual impacted customers are being contacted via support tickets. Clusters in other Azure regions and cloud providers are unaffected. We are actively monitoring the situation and will provide additional details as they become available. Azure status is tracked at https://azure.status.microsoft/en-us/status/. If you have an urgent matter, please reach out to us via the support portal (support.instaclustr.com) or email ([email protected]).

  2. monitoring Jul 19, 2024, 03:05 AM UTC

    We are continuing to see recovery in our Provisioning systems as indicated by updates from Azure. We are actively monitoring the situation and will provide additional details as they become available. Azure status is tracked at https://azure.status.microsoft/en-us/status/. Please reach out to us via the support portal (support.instaclustr.com) or email ([email protected]) if you have any further questions.

  3. resolved Jul 19, 2024, 06:27 AM UTC

    This incident has been resolved as indicated by updates from Azure.

Read the full incident report →

Major July 2, 2024

Failure when adding a data centre to an existing Redis cluster in GCP or Azure

Detected by Pingoru
Jul 02, 2024, 05:28 AM UTC
Resolved
Jul 03, 2024, 06:35 AM UTC
Duration
1d 1h
Affected: Management ConsoleCluster Management API
Timeline · 2 updates
  1. identified Jul 02, 2024, 05:28 AM UTC

    We are currently experiencing issues with adding a data centre through the Cluster Management API and Management Console for Redis clusters that are hosted with GCP or Azure. Add data centre requests for Redis clusters will be accepted but the nodes for the new data centre will not reach running. We have identified a manual workaround for the issue, please reach out to us via the support portal (support.instaclustr.com) or email ([email protected]) to action this. Our team is actively working on resolving the underlying problem and we apologise for any inconvenience caused. Additional details will be provided as they become available. If you have an urgent matter, please reach out to us via the support portal (support.instaclustr.com) or email ([email protected]).

  2. resolved Jul 03, 2024, 06:35 AM UTC

    This incident has been resolved.

Read the full incident report →

Major June 12, 2024

Private Network Cluster Provisioning Failure for AWS

Detected by Pingoru
Jun 12, 2024, 03:27 AM UTC
Resolved
Jun 12, 2024, 05:14 AM UTC
Duration
1h 46m
Affected: Management ConsoleCluster Management API
Timeline · 2 updates
  1. investigating Jun 12, 2024, 03:27 AM UTC

    We are currently experiencing issues with provisioning Private Network clusters on AWS through the Cluster Management API and Management Console. Provisioning requests will be accepted but the cluster nodes will not reach running. Our team is actively working on resolving this problem and we apologise for any inconvenience caused. At this stage, existing customer clusters appear to be unaffected, and the Monitoring API and Prometheus API are operational. Additional details will be provided as they become available. If you have an urgent matter, please reach out to us via the support portal (support.instaclustr.com) or email ([email protected]).

  2. resolved Jun 12, 2024, 05:14 AM UTC

    Provisioning of new AWS Private Network clusters is working again now.

Read the full incident report →