Meteor incident

Downtime at our database service provider.

Major Resolved View vendor source →

Meteor experienced a major incident on October 31, 2023 affecting Galaxy management interface and Meteor Accounts server and 1 more component, lasting 1d 6h. The incident has been resolved; the full update timeline is below.

Started
Oct 31, 2023, 06:34 PM UTC
Resolved
Nov 02, 2023, 12:45 AM UTC
Duration
1d 6h
Detected by Pingoru
Oct 31, 2023, 06:34 PM UTC

Affected components

Galaxy management interfaceMeteor Accounts serverMeteor APMGalaxy Beta

Update timeline

  1. investigating Oct 31, 2023, 06:34 PM UTC

    We are currently experiencing access issues with Galaxy, Meteor Accounts, APM, Atmosphere and Dashboard due to downtime from our database service provider. Customer applications hosted on Galaxy will not be affected. However, deploys are currently not functioning properly. We would also like to inform you that apps integrated with accounts.meteor.com may not be able to use sign-in and sign-up processes. Our team is actively addressing these issues to restore normal operations as soon as possible. We appreciate your patience and understanding during this time.

  2. investigating Oct 31, 2023, 06:34 PM UTC

    We are continuing to investigate this issue.

  3. identified Nov 01, 2023, 12:59 AM UTC

    We regret to inform you that we recently encountered some access issues with our Galaxy, APM, Meteor Accounts, Atmosphere, and Dashboard services due to a downtime experienced by our database service provider. I'm pleased to report that while the applications hosted on Galaxy remained unaffected, there was a disruption in deploy functionalities. Additionally, applications integrated with accounts.meteor.com experienced difficulties with login processes. In order to restore normal operations, we initiated a backup restoration process. However, please be aware that this might lead to some temporary absence of certain packages. As soon as our database provider is back up, we will perform a comprehensive data resynchronization to rectify any discrepancies. Please rest assured that our dedicated team is working diligently to ensure a seamless transition back to full functionality.

  4. monitoring Nov 01, 2023, 12:59 AM UTC

    A fix has been implemented and we are monitoring the results.

  5. resolved Nov 02, 2023, 02:54 PM UTC

    This incident is now resolved. We have successfully restored all the affected services, and the data resynchronization process was completed successfully.