PubNub experienced a notice incident on September 7, 2025 affecting Presence Service, lasting 17m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- monitoring Sep 07, 2025, 06:45 PM UTC
We experienced increased latencies and error rates in our Tokyo, San Jose, and Virginia points-of-presence from 18:14 - 18:17 UTC. A fix was deployed and we are monitoring the situation.
- resolved Sep 07, 2025, 07:02 PM UTC
The issues affected the Presence service from 18:14 - 18:17 UTC. We will follow up with a post-mortem soon. We apologize for the impact this may have had on your service. Please reach out to us by contacting PubNub Support ([email protected]) if you wish to discuss the impact on your service.
- postmortem Sep 12, 2025, 11:19 PM UTC
### **Problem Description, Impact, and Resolution** At 18:14 UTC on September 7, 2025 we observed increased error rates and latency for our Presence service in our San Jose, Virginia, and Tokyo regions. We increased capacity in those regions and the issue was resolved at 18:17 UTC. This issue was a recurrence of the issue [experienced on September 2, 2025](https://status.pubnub.com/incidents/1n8xk6w5y9lk), where a bug in one of our APIs allowed a request to execute an operation that exceeded assumed limits in extreme cases, causing out-of-memory conditions for the Presence service. ### **Mitigation Steps and Recommended Future Preventative Measures** In the previous instance of this issue, we placed restrictions on the API in question; those changes were not restrictive enough, which allowed for this recurrence. We have corrected that oversight, as well as increased memory capacities in this area of our system as an additional safeguard.