PubNub experienced a notice incident on May 22, 2025 affecting Publish/Subscribe Service and North America Points of Presence and 1 more component, lasting —. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- resolved May 22, 2025, 09:34 PM UTC
Beginning at 20:51 UTC, we detected increased errors and latency for traffic within a single availability zone in the US East region. Our engineers investigated the issue and successfully restored service, which has remained stable since 21:04 UTC.
- postmortem May 28, 2025, 10:15 PM UTC
### **Problem Description, Impact, and Resolution** On May 23, 2025 at 20:51 UTC, PubNub experienced increased errors and latency for a subset of traffic within a single availability zone in the US East region. The root cause was an operational error during an infrastructure change. We unintentionally drained traffic from newly deployed load balancers. As a result, the system routed traffic incorrectly, causing elevated error rates and latency for some users. Once the issue was identified, traffic was rerouted, and service performance returned to normal levels. The incident was resolved by 21:04 UTC the same day. ### **Mitigation Steps and Recommended Future Preventative Measures** To prevent the issue from recurring in the future, the change procedure has been updated to reference isolated, non-production resources during preparation stages. If run prematurely, the script will now operate on an empty set, ensuring no production traffic is impacted.