Neo4j Aura incident

Operations requiring system database fail when not connected directly to the system database

Minor Resolved View vendor source →

Neo4j Aura experienced a minor incident on May 31, 2024 affecting AuraDB Virtual Dedicated Cloud on AWS (*.databases.neo4j.io) and AuraDB Professional on AWS (*.databases.neo4j.io) and 1 more component, lasting 3d 11h. The incident has been resolved; the full update timeline is below.

Started
May 31, 2024, 10:03 PM UTC
Resolved
Jun 04, 2024, 09:56 AM UTC
Duration
3d 11h
Detected by Pingoru
May 31, 2024, 10:03 PM UTC

Affected components

AuraDB Virtual Dedicated Cloud on AWS (*.databases.neo4j.io)AuraDB Professional on AWS (*.databases.neo4j.io)AuraDS on AWS (*.databases.neo4j.io)AuraDS Enterprise on AWS (*.databases.neo4j.io)AuraDB Virtual Dedicated Cloud on Azure (*.databases.neo4j.io)AuraDB Professional on Azure (*.databases.neo4j.io)AuraDS Enterprise on Azure (*.databases.neo4j.io)AuraDS on Azure (*.databases.neo4j.io)AuraDB Virtual Dedicated Cloud on GCP (*.databases.neo4j.io)AuraDB Professional on GCP (*.databases.neo4j.io)

Update timeline

  1. investigating May 31, 2024, 10:03 PM UTC

    We have identified a problem with the routing of some internal messages which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves updating the system database while connected to the neo4j database. Such commands and queries might involve user management as an example. The workaround is to connect to the system database directly and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify system as the database name to connect to. We are currently investigating the root cause and will update this incident with progress.

  2. investigating Jun 01, 2024, 04:04 AM UTC

    We have identified a problem with the routing of some internal messages which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves updating the system database while connected to the neo4j database. Such commands and queries might involve user management as an example. The workaround is to connect to the system database directly and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify system as the database name to connect to. We are currently investigating the root cause and will update this incident with progress.

  3. investigating Jun 01, 2024, 10:17 AM UTC

    We have identified a problem with the routing of some internal messages which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves using the system database while connected to the neo4j database. Such commands and queries might involve user management as an example. The workaround is to connect to the system database directly and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify "system" as the database name to connect to in the driver's code. We are currently investigating the root cause and will update this incident with progress.

  4. identified Jun 01, 2024, 05:36 PM UTC

    We have identified a problem with the routing of some internal messages, which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves using the system database while connected to the neo4j database. For example, such commands and queries might involve user management. The workaround is to directly connect to the system database and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify "system" as the database name to connect to in the driver's code. We have identified the root cause and are working on rolling out a fix.

  5. identified Jun 02, 2024, 01:00 AM UTC

    We have identified a problem with the routing of some internal messages, which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves using the system database while connected to the neo4j database. For example, such commands and queries might involve database user management. The workaround is to directly connect to the system database and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify "system" as the database name to connect to in the driver's code. We have identified the root cause and are working on rolling out a fix.

  6. identified Jun 02, 2024, 07:02 AM UTC

    We have identified a problem with the routing of some internal messages, which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves using the system database while connected to the neo4j database. For example, such commands and queries might involve database user management. The workaround is to directly connect to the system database and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify "system" as the database name to connect to in the driver's code. We have identified the root cause and are working on rolling out a fix.

  7. identified Jun 02, 2024, 01:28 PM UTC

    We have identified a problem with the routing of some internal messages, which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves using the system database while connected to the neo4j database. For example, such commands and queries might involve database user management. The workaround is to directly connect to the system database and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify "system" as the database name to connect to in the driver's code. The root cause has been identified, and we are working on rolling out a fix.

  8. identified Jun 02, 2024, 07:00 PM UTC

    We have identified a problem with the routing of some internal messages, which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves using the system database while connected to the neo4j database. For example, such commands and queries might involve database user management. The workaround is to directly connect to the system database and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify "system" as the database name to connect to in the driver's code. The root cause has been identified, and we are working on rolling out a fix.

  9. identified Jun 03, 2024, 01:00 AM UTC

    We have identified a problem with the routing of some internal messages, which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves using the system database while connected to the neo4j database. For example, such commands and queries might involve database user management. The workaround is to directly connect to the system database and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify "system" as the database name to connect to in the driver's code. The root cause has been identified, and we are working on rolling out a fix.

  10. identified Jun 03, 2024, 07:02 AM UTC

    We have identified a problem with the routing of some internal messages, which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves using the system database while connected to the neo4j database. For example, such commands and queries might involve database user management. The workaround is to directly connect to the system database and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify "system" as the database name to connect to in the driver's code. The root cause has been identified, and we are working on rolling out a fix.

  11. identified Jun 03, 2024, 01:19 PM UTC

    We have identified a problem with the routing of some internal messages, which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves using the system database while connected to the neo4j database. For example, such commands and queries might involve database user management. The workaround is to directly connect to the system database and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify "system" as the database name to connect to in the driver's code. The root cause has been identified, and we are working on rolling out a fix.

  12. identified Jun 04, 2024, 01:08 AM UTC

    We have identified a problem with the routing of some internal messages, which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves using the system database while connected to the neo4j database. For example, such commands and queries might involve database user management. The workaround is to directly connect to the system database and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify "system" as the database name to connect to in the driver's code. The root cause has been identified, and we are working on rolling out a fix.

  13. identified Jun 04, 2024, 07:00 AM UTC

    We have identified a problem with the routing of some internal messages, which impacts some operations for Aura Enterprise and Aura Professional customers. The scope of operations would be anything that involves using the system database while connected to the neo4j database. For example, such commands and queries might involve database user management. The workaround is to directly connect to the system database and run these commands. If using Workspace or Browser, you can switch to the system database with the command ":USE system". If using the Neo4j drivers, specify "system" as the database name to connect to in the driver's code. The root cause has been identified, and we are working on rolling out a fix.

  14. resolved Jun 04, 2024, 09:56 AM UTC

    We have successfully rolled out a fix and this issue is now fixed.