PubNub incident

Events and Actions Outage in US East PoP. Events are not Being Dispatched.

Minor Resolved View vendor source →

PubNub experienced a minor incident on May 2, 2023, lasting 52m. The incident has been resolved; the full update timeline is below.

Started
May 02, 2023, 10:01 PM UTC
Resolved
May 02, 2023, 10:53 PM UTC
Duration
52m
Detected by Pingoru
May 02, 2023, 10:01 PM UTC

Update timeline

  1. investigating May 02, 2023, 10:01 PM UTC

    Events and Actions messages are not currently being dispatched for customers in our US-East PoP.

  2. monitoring May 02, 2023, 10:18 PM UTC

    The Events and Actions service in US-East has been restored and any messages that would trigger actions have now been processed.

  3. resolved May 02, 2023, 10:53 PM UTC

    This service incident has been resolved since 22:00 UTC

  4. postmortem May 05, 2023, 07:56 PM UTC

    ### **Problem Description, Impact, and Resolution** On Tuesday May 2, 2023 at 21:23 UTC, we observed that Events and Actions messages were not being processed on our US PoP or EU Central PoP. Shortly after observing the issue, we redeployed a processing schema, and the issue was resolved on May 2, 2023 at 22:00 UTC. ### **Mitigation Steps and Recommended Future Preventative Measures** To prevent a similar issue from occurring in the future, we added the needed monitoring to alert us when Events and Actions messages are not being processed properly so we can redeploy the processing schema as needed.