Ably had 32 outages in the last 2 years — averaging 1.3 incidents per month.
There were 32 Ably outages since June 18, 2024. Each is summarised below — incident details, duration, and resolution information.
Customers using our presence feature may have seen sporadic spurious leave + immediate re-enter events for presence members over the last two days, since Tuesday. We have rolled back to a version of the service which does not have the bug, and we apologise for any inconvenience.
The ably.com website, dashboard and control API are experiencing disruption, intermittently since 0749 UTC on 15 February. We are investigating and will post here with further updates. 15th Feb 09:03 AM We are still working to understand the root cause of this problem. We don't have an ETA for resolution yet. 15th Feb 09:39 AM We are contacting our upstream service providers to assist in investigation of this issue. We don't have an ETA for resolution yet. 15th Feb 10:15 AM We are migrating to a replacement database instance. We should be able to confirm shortly whether or not this will resolve the issue. 15th Feb 10:29 AM All services appear to be up and running again, but we will continue to monitor.
Between 15:40 and 17:40 UTC, customers using the REST API directly may have observed 502 error responses. Our SDKs automatically retry 5xx responses to fallback endpoints, so customers using SDKs should not have noticed anything except increased latency due to the retries. The proximate cause was a faulty router deploy, we are actively investigating the root cause, and will be improving our alerting system, which should have caught this sooner. We apologise for any disruption.
We've been alerted to slightly elevated error rates in eu-central-1 and are investigating. 20th Oct 04:14 AM The issue is resolved and service is back to normal.
On Tuesday 19th March at 08:00 UTC we performed backend maintenance for ably.com and control.ably.net which lasted 30 minutes. During this period these services were unavailable. This did not affect the realtime system. Please contact Ably support if you have any concerns, or about anything else relating to the Ably service.
We are currently investigating an issue affecting https://ably.com. Users may experience intermittent 500 errors. 19th Mar 02:30 PM We have taken mitigating actions and error rates have returned to normal levels.
There was an elevated 5xx error rate for connection creation from the us-east-1 datacenter from 13:24-14:20 UTC, due to a broken router process, which was then removed from service. The error rate in that region peaked at 4%. No other regions were affected. Any customers using an Ably SDK will not have noticed anything other than potentially increased latencies, since Ably SDKs automatically retry to a fallback region on 5xx. However, customers in east coast or central US who are using SSE, or other protocols using our protocol translation functionality (such as mqtt, pusher, or pubnub), may have noticed some connection attempts fail with 5xx.