Phantombuster incident

Database outage

Critical Resolved View vendor source →

Phantombuster experienced a critical incident on May 21, 2025 affecting Phantoms and API, lasting 1h 55m. The incident has been resolved; the full update timeline is below.

Started
May 21, 2025, 09:02 PM UTC
Resolved
May 21, 2025, 10:58 PM UTC
Duration
1h 55m
Detected by Pingoru
May 21, 2025, 09:02 PM UTC

Affected components

PhantomsAPI

Update timeline

  1. identified May 21, 2025, 09:02 PM UTC

    At approximately 16:30 UTC, an unusually high volume of specific write queries from our application began to strain our main database. This load overwhelmed our internal data replication system. As a result, a backlog of data awaiting replication quickly grew, consuming all available space on the main database's system disk and forcing it offline. We are currently restoring the database from a snapshot; however, this process is taking longer than expected because the snapshot includes this large volume of non-replicated data, which is inherently slower to process during recovery.

  2. identified May 21, 2025, 09:12 PM UTC

    The data restoration process is nearly complete. We estimate that the system will be fully back to normal operation within the next hour. Thank you for your patience as we bring services back online.

  3. resolved May 21, 2025, 10:58 PM UTC

    Our systems have fully recovered and are back to normal operation. The data restoration process has completed successfully, and all services—API and Phantoms—are now functioning as expected. We’ve addressed the issue caused by an unusually high volume of write queries that led to a backlog in our internal replication system. We sincerely thank you for your patience throughout the resolution process. If you continue to experience any issues, please don’t hesitate to reach out to our support team.