Stellar incident

Latency issues in horizon.stellar.org cluster

Notice Resolved View vendor source →

Stellar experienced a notice incident on October 29, 2019 affecting SDF Public Network Horizon, lasting 3h 11m. The incident has been resolved; the full update timeline is below.

Started
Oct 29, 2019, 07:29 PM UTC
Resolved
Oct 29, 2019, 10:40 PM UTC
Duration
3h 11m
Detected by Pingoru
Oct 29, 2019, 07:29 PM UTC

Affected components

SDF Public Network Horizon

Update timeline

  1. investigating Oct 29, 2019, 07:29 PM UTC

    We're experiencing an elevated level of API errors and are currently looking into the issue.

  2. monitoring Oct 29, 2019, 07:38 PM UTC

    We have failed over to our standby cluster, the new cluster is successfully ingesting and serving requests. We will continue to monitor the situation.

  3. investigating Oct 29, 2019, 07:50 PM UTC

    We are investigating renewed API failure rates.

  4. monitoring Oct 29, 2019, 08:20 PM UTC

    Service has been restored and we are still investigating.

  5. resolved Oct 29, 2019, 10:40 PM UTC

    This issue has now been resolved, all systems are fully operational.