PubNub incident

Elevated rate of latency in US East Point of Presence

Notice Resolved View vendor source →

PubNub experienced a notice incident on May 2, 2023, lasting —. The incident has been resolved; the full update timeline is below.

Started
May 02, 2023, 05:15 PM UTC
Resolved
May 02, 2023, 02:30 PM UTC
Duration
—
Detected by Pingoru
May 02, 2023, 05:15 PM UTC

Update timeline

  1. resolved May 02, 2023, 05:15 PM UTC

    On Tuesday, May 2, 2023 between 14:20 UTC and 14:28 UTC, customers in our US-East PoP may have experienced some errors, timeouts, or delays across multiple services. RCA will be provided soon.

  2. postmortem May 08, 2023, 02:37 PM UTC

    ### **Problem Description, Impact, and Resolution** On Tuesday May 2, 2023 at 14:26 UTC observed errors and latency on all services in US East PoP. We modified the configuration to scale faster during high load occurrences, and the issue was resolved on May 2, 2023 at 14:34 UTC. This issue occurred because the monitoring could not ramp up needed services quickly enough. ‌ ### **Mitigation Steps and Recommended Future Preventative Measures** To prevent a similar issue from occurring in the future, we modified our monitoring to increase resources more rapidly when needed. These changes were made during the incident and brought resolution to this incident.