Routific Outage History

Routific is up right now

Routific had 36 outages in the last 2 years totaling 148h 15m of downtime — averaging 1.5 incidents per month.

There were 36 Routific outages since July 26, 2024 totaling 148h 15m of downtime. Each is summarised below — incident details, duration, and resolution information.

Source: https://status.routific.com

Critical January 22, 2025

New Routific outage

Detected by Pingoru
Jan 22, 2025, 06:37 PM UTC
Resolved
Jan 23, 2025, 03:07 AM UTC
Duration
8h 29m
Affected: New Routific
Timeline · 6 updates
  1. investigating Jan 22, 2025, 06:37 PM UTC

    We are actively working to restore user access to the new Routific. The issue is affecting most users and our engineering team is working on a fix at the moment. More updates will be provided in the next 30 minutes.

  2. investigating Jan 22, 2025, 07:11 PM UTC

    We are actively working to restore user access to the new Routific. This issue is our team's highest priority. We thank you for your patience as we work to resolve this as soon as possible. The next update will come within 30 minutes.

  3. investigating Jan 22, 2025, 07:19 PM UTC

    Access to the new Routific has been restored. Thank you for your patience! We are still investigating the cause of the outage. We will report our findings as soon as possible.

  4. monitoring Jan 22, 2025, 07:22 PM UTC

    The new Routific is stable and operational. Our team will continue monitoring and provide a report on the cause of the issue soon.

  5. resolved Jan 23, 2025, 03:07 AM UTC

    The new Routific continues to be stable and fully operational. Our team has identified the root cause of the issue. A bug in our internal messaging system caused an overload, resulting in about 45 minutes of downtime. We have since fixed the issue and will improve our monitoring systems to identify these types of issues earlier to avoid future disruptions.

  6. postmortem Jan 24, 2025, 05:01 PM UTC

    On January 22, 2025 at approximately 10:16 AM PST, Routific experienced an outage that prevented users from accessing the new Routific. The disruption lasted around 45 minutes before full service was restored. Our investigation determined that a bug in our internal messaging system caused a backlog of unacknowledged messages, which overwhelmed the system and led to the outage. Our engineering team quickly identified the root cause and deployed a fix to clear the message backlog and stabilize the platform. We have also improved our monitoring and alerting systems to detect similar issues sooner and avoid future disruptions. Thank you for your patience as we worked to resolve this matter.

Read the full incident report →

Minor November 19, 2024

Driver dispatch partial outage

Detected by Pingoru
Nov 19, 2024, 02:00 AM UTC
Resolved
Nov 19, 2024, 02:00 AM UTC
Duration
—
Timeline · 2 updates
  1. resolved Nov 19, 2024, 02:00 AM UTC

    This incident has been resolved.

  2. postmortem Nov 19, 2024, 08:11 PM UTC

    On Monday, November 18 starting around 4:37 p.m. PT, some users were not able to access their routes on Routific Driver via the link provided in the SMS dispatch. This partial outage lasted approximately one hour before it was resolved. The issue was caused by our URL shortener malfunctioning. When our internal monitoring systems alerted our engineering team of the issue, we were able to reboot our systems allowing the URL shorteners to work as expected. Our team is now working to push an update to improve the stability of our URL shortener.

Read the full incident report →

Major November 8, 2024

Google Play Store app download

Detected by Pingoru
Nov 08, 2024, 05:11 AM UTC
Resolved
Nov 08, 2024, 04:53 PM UTC
Duration
11h 42m
Affected: Driver Mobile App
Timeline · 6 updates
  1. identified Nov 07, 2024, 06:03 PM UTC

    We are actively working to restore the ability to download our app from the Play Store. Please note that this issue only affects new downloads and does not impact users who currently have our application installed on their devices.

  2. identified Nov 07, 2024, 07:09 PM UTC

    We are still actively working to restore the ability to download our app from the Google Play Store.

  3. identified Nov 07, 2024, 10:50 PM UTC

    Our Routific mobile apps have been submitted to the Google Play Store and they’re waiting to be reviewed by the Google Play Store team. Please note that this process may take longer than 24 hours. This issue only affects users attempting to download our apps and does not impact users who already have our apps installed on their devices. We will provide another status update on Friday around 9:00 AM PST.

  4. identified Nov 08, 2024, 05:11 AM UTC

    The ability to download Routific mobile apps from the Google Play Store has been partially restored. Direct download is now available via the email links sent out to your drivers when inviting them. The ability to search for the apps directly on the Google Play Store is still unavailable. This issue does not impact users who already have our apps installed on their devices or who access the app via the email invite link. The direct links are as follows: Routific Driver (compatible with the old Routific) https://play.google.com/store/apps/details?id=com.routific.driverapp&hl=en_CA Routific (compatible with the new Routific) https://play.google.com/store/apps/details?id=com.routific.mobile We will provide another status update on Friday around 9 AM PST.

  5. monitoring Nov 08, 2024, 04:01 PM UTC

    A fix has been implemented and we are monitoring the results.

  6. resolved Nov 08, 2024, 04:53 PM UTC

    Routific mobile apps are now fully accessible via the Google Play Store. You can search for the Routific apps in the Play Store and download via the direct links we send out during the driver invite process.

Read the full incident report →

Notice October 30, 2024

SMS Connectivity Issues in United States and Canada

Detected by Pingoru
Oct 30, 2024, 08:37 PM UTC
Resolved
Oct 31, 2024, 03:39 PM UTC
Duration
19h 1m
Affected: Routific Notifications
Timeline · 2 updates
  1. monitoring Oct 30, 2024, 08:37 PM UTC

    Our SMS service provider is reporting some SMS notifications being blocked in the U.S. and Canada. We are in contact with them to address the issue and hope to restore deliverability as soon as possible.

  2. resolved Oct 31, 2024, 03:39 PM UTC

    This incident has been resolved.

Read the full incident report →

Notice October 22, 2024

AT&T blocking some SMS messages in the United States

Detected by Pingoru
Oct 22, 2024, 11:22 PM UTC
Resolved
Oct 23, 2024, 04:03 PM UTC
Duration
16h 40m
Affected: Routific SaaS productTwilio SMS
Timeline · 2 updates
  1. identified Oct 22, 2024, 11:22 PM UTC

    We have identified an issue with AT&T in the United States that is preventing some drivers from receiving the SMS link with their route. We are investigating with our SMS provider and will update as soon as we know more. If your driver hasn’t received a notification after you’ve dispatched a route, there is a workaround: Click “Copy Link” on the Dispatch screen and paste the link in an email or text message to send to your drivers. This link will open up the Routific web app on their phone, which can also be used to complete deliveries.

  2. resolved Oct 23, 2024, 04:03 PM UTC

    We are no longer experiencing failures to the AT&T network in the United States. After dispatching your project, drivers under this carrier will receive the SMS notification with the link to access their routes without any issues.

Read the full incident report →

Major October 11, 2024

Temporarily unable to access routific.com

Detected by Pingoru
Oct 11, 2024, 10:51 PM UTC
Resolved
Oct 12, 2024, 05:18 AM UTC
Duration
6h 26m
Timeline · 3 updates
  1. investigating Oct 11, 2024, 10:51 PM UTC

    Access to our homepage www.routific.com is temporarily blocked due to our third party provider having an outage. Users attempting to access app.routific.com or beta.routific.com to manage their routes are unaffected.

  2. monitoring Oct 11, 2024, 11:07 PM UTC

    The issue has been resolved. We will continue to monitor

  3. resolved Oct 12, 2024, 05:18 AM UTC

    This incident has been resolved.

Read the full incident report →

Major October 11, 2024

Temporarily unable to access our homepage routific.com

Detected by Pingoru
Oct 11, 2024, 08:59 PM UTC
Resolved
Oct 11, 2024, 09:22 PM UTC
Duration
23m
Timeline · 3 updates
  1. investigating Oct 11, 2024, 08:59 PM UTC

    Access to our home page www.routific.com was temporarily blocked due to our third party provider having an outage. Users attempting to access app.routific.com to manage their routes were unaffected.

  2. monitoring Oct 11, 2024, 09:02 PM UTC

    The issue has been resolved and we are continuing the monitor.

  3. resolved Oct 11, 2024, 09:22 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor September 17, 2024

SMS notification instability

Detected by Pingoru
Sep 17, 2024, 06:57 PM UTC
Resolved
Sep 19, 2024, 11:57 AM UTC
Duration
1d 16h
Affected: Routific SaaS product
Timeline · 3 updates
  1. identified Sep 17, 2024, 06:57 PM UTC

    We had an issue yesterday with our SMS provider that caused a potential degradation in the sending of SMS messages. We are still investigating the potential ongoing impact to notifications.

  2. monitoring Sep 18, 2024, 02:06 AM UTC

    We have isolated two issues around this, the first happened between 20:20 - 21:00 UTC and the second happened between16:30 -16:55 UTC. We discovered that some messages did go through and only a subset of notifications was affected. We are still actively monitoring the situation and working with our provider to better mitigate issues like this in the future.

  3. resolved Sep 19, 2024, 11:57 AM UTC

    There has been no new occurrences of this issue and all SMS services should be functioning as expected. We plan to improve our ability to monitor and react to future outages like this in an attempt to limit or prevent the effects on our users.

Read the full incident report →

Major September 12, 2024

Disruption in SMS Customer Notifications for some Users in Australia

Detected by Pingoru
Sep 12, 2024, 01:53 AM UTC
Resolved
Sep 13, 2024, 08:29 PM UTC
Duration
1d 18h
Affected: Routific Notifications
Timeline · 2 updates
  1. investigating Sep 12, 2024, 01:53 AM UTC

    We are currently investigating this issue.

  2. resolved Sep 13, 2024, 08:29 PM UTC

    After thorough investigation, we have determined that this issue was isolated to a specific group of users. We have been in direct contact with all affected users to resolve the issue as quickly as possible.

Read the full incident report →

Minor August 5, 2024

Account management and webhook email notifications not being sent correctly

Detected by Pingoru
Aug 05, 2024, 08:31 PM UTC
Resolved
Aug 05, 2024, 10:28 PM UTC
Duration
1h 57m
Affected: Routific SaaS productMandrill US EastRoutific Notifications
Timeline · 4 updates
  1. investigating Aug 05, 2024, 08:31 PM UTC

    We've identified an integration issue with an external tool responsible for sending email notifications. This is affecting account management and webhook failure email notifications. Consequently, users may not be receiving emails to activate accounts, update password or notifications about failed webhook calls.

  2. identified Aug 05, 2024, 09:41 PM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Aug 05, 2024, 09:56 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. resolved Aug 05, 2024, 10:28 PM UTC

    This incident has been resolved.

Read the full incident report →

Notice July 26, 2024

Routing Service Disruption

Detected by Pingoru
Jul 26, 2024, 12:02 AM UTC
Resolved
Jul 25, 2024, 07:00 PM UTC
Duration
—
Timeline · 1 update
  1. resolved Jul 26, 2024, 12:02 AM UTC

    Around 12:15:46 PST, we detected a connection issue with a downstream provider. This issue resulted in jobs not being processed, consequently timing out. The issue lasted until 12:17:49 PST and self-resolved. We have reached out to the provider to gather more details on what has caused the connection error and will continue to monitor the situation.

Read the full incident report →