Hi Marley incident

Outbound messaging filtering from Verizon

Resolved after 1 hour 2 minutes, affecting Messaging, on September 28, 2023.

postmortem Sep 29, 2023, 01:42 PM UTC

Outbound message filtering was confirmed to be resolved by 4:30pm EST yesterday, 9/28. Messages were replayed at roughly 6:30pm EST. Our SMS partner confirmed that this was not specific to Hi Marley and the filtering was happening across all of their application-to-person messaging using 10-digit long codes in the US on traffic to Verizon and AT&T’s networks. In total, roughly 4% of outbound traffic was affected in a random pattern, there weren’t particular customers or types of messages that were more likely to be filtered than others.

Started
Sep 28, 2023, 07:35 PM UTC
Resolved
Sep 28, 2023, 08:38 PM UTC
Duration
1h 2m
Detected by Pingoru
Sep 28, 2023, 07:35 PM UTC

Affected components

Messaging

Update timeline

  1. identified Sep 28, 2023, 07:35 PM UTC

    Our monitoring has detected outbound message filtering by Verizon Wireless. We've reported this to the carrier with high priority, and our team is preparing the necessary script to replay these outbound messages once Verizon confirms the filtering has ended. We will provide confirmation as soon as the replay process is successfully completed

  2. resolved Sep 28, 2023, 08:38 PM UTC

    We've received confirmation from the mobile carrier that the filtering has been resolved. We'll continue to monitor to confirm that, and will process the replays once we can confidently confirm the replaying is resolved.

  3. postmortem Sep 29, 2023, 01:42 PM UTC

    Outbound message filtering was confirmed to be resolved by 4:30pm EST yesterday, 9/28. Messages were replayed at roughly 6:30pm EST. Our SMS partner confirmed that this was not specific to Hi Marley and the filtering was happening across all of their application-to-person messaging using 10-digit long codes in the US on traffic to Verizon and AT&T’s networks. In total, roughly 4% of outbound traffic was affected in a random pattern, there weren’t particular customers or types of messages that were more likely to be filtered than others.

Your product runs on more than Hi Marley

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →