Routific incident

SMS notification instability

Minor Resolved View vendor source →

Routific experienced a minor incident on September 17, 2024 affecting Routific SaaS product, lasting 1d 16h. The incident has been resolved; the full update timeline is below.

Started
Sep 17, 2024, 06:57 PM UTC
Resolved
Sep 19, 2024, 11:57 AM UTC
Duration
1d 16h
Detected by Pingoru
Sep 17, 2024, 06:57 PM UTC

Affected components

Routific SaaS product

Update timeline

  1. identified Sep 17, 2024, 06:57 PM UTC

    We had an issue yesterday with our SMS provider that caused a potential degradation in the sending of SMS messages. We are still investigating the potential ongoing impact to notifications.

  2. monitoring Sep 18, 2024, 02:06 AM UTC

    We have isolated two issues around this, the first happened between 20:20 - 21:00 UTC and the second happened between16:30 -16:55 UTC. We discovered that some messages did go through and only a subset of notifications was affected. We are still actively monitoring the situation and working with our provider to better mitigate issues like this in the future.

  3. resolved Sep 19, 2024, 11:57 AM UTC

    There has been no new occurrences of this issue and all SMS services should be functioning as expected. We plan to improve our ability to monitor and react to future outages like this in an attempt to limit or prevent the effects on our users.