PubNub incident
Potential for some missed messages for subscribers in US East PoP
PubNub experienced a minor incident on August 25, 2026 affecting Publish/Subscribe Service and North America Points of Presence, lasting 1h 28m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 25, 2026, 01:22 PM UTC
As of 12:15 UTC, Users in our US East PoP are currently experiencing an issue with Publish and Subscribe service that is causing some messages to not appear in calls to that service. No errors are being returned from the service, despite messages not being returned correctly. We are investigating the issue.
- monitoring Aug 25, 2026, 02:10 PM UTC
As of 13:55 UTC, the issue affecting Publish and Subscribe services in the US East PoP has been mitigated. Synthetic monitoring has returned to normal, and no missed messages have been detected. The incident is currently believed to have been related to a monitoring failure. Further investigation is ongoing, and the service continues to be monitored closely.
- resolved Aug 25, 2026, 02:51 PM UTC
With no further issues observed for the past 30 minutes, the incident has been resolved. We will follow up soon with a root cause analysis. If you believe you experienced an impact related to this incident, please report it to PubNub Support at [email protected].
- postmortem Aug 28, 2026, 05:46 PM UTC
### **Problem Description, Impact, and Resolution** On Tuesday, August 25th, 2026, at 12:10 UTC, our internal monitoring alerted us to an issue where a very small subset of messages within one availability zone of one region \(our US East point of presence\) may not have been immediately delivered to subscribers in that same region. All messages were persisted normally within PubNub Persistence service, **thus no message data was lost.** Traffic to or from any other region or AZ was not affected, and no other PubNub services were affected. No customers reported impact. ### **Root Cause** During planned internal load testing, a group of internal routing servers was removed from service. Due to a configuration flaw, their network addresses were not fully retired and remained cached by our publishing layer. This presented no problem for the most part. However, one address was subsequently reassigned to an unrelated internal component that responded normally on the same protocol and port. Because that response appeared successful, the publishing layer received no error, and messages sent to that address were routed incorrectly. A rolling restart of the affected servers cleared the outdated addresses at 13:25 UTC. ### **Mitigation Steps and Recommended Future Preventative Measures** * Once the affected servers were identified, a rolling restart was performed to clear the outdated routing addresses. * Following an investigation that confirmed the root cause, a fix ensuring outdated internal routing addresses are correctly retired was deployed to all publishing servers worldwide on August 25th, 2026 at 22:00 UTC.