Rhumbix Outage History
Rhumbix is up right nowPingoru has recorded 26 incidents for Rhumbix: 6 in the last 12 months and 2 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 0 | 0 of 720 (0%) |
| May 2026 | 0 | 0 of 744 (0%) |
| June 2026 | 0 | 0 of 720 (0%) |
| July 2026 | 2 | 7 of 744 (1%) |
| August 2026 | 1 | 0 of 744 (0%) |
| September 2026 | 0 | 0 of 720 (0%) |
| October 2026 | 1 | 1 of 512 (0%) |
Incidents, newest first
Minor October 21, 2026 Data syncing delays into Field Analytix 40m Rhumbix
Started Oct 21, 2026, 02:43 PM UTC· resolved Oct 21, 2026, 03:23 PM UTC
- investigating Oct 21, 2026, 02:43 PM UTC
We are currently investigating this issue.
- identified Oct 21, 2026, 02:58 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Oct 21, 2026, 03:13 PM UTC
A fix has been implemented and we are monitoring the data sync until it catches up
- resolved Oct 21, 2026, 03:23 PM UTC
This incident has been resolved.
Notice August 4, 2026 Public API Long Running Queries –
Started Aug 04, 2026, 10:30 AM UTC· resolved Aug 04, 2026, 10:30 AM UTC
- resolved Aug 04, 2026, 03:46 PM UTC
Slow queries have been backing up without being killed, causing future queries to be slow, and pile up eventually exhausting all resources.
Minor July 20, 2026 Degraded performance and elevated timeouts 3h 32m Rhumbix
Started Jul 20, 2026, 02:29 PM UTC· resolved Jul 20, 2026, 06:01 PM UTC
- investigating Jul 20, 2026, 02:29 PM UTC
We are currently investigating this issue.
- identified Jul 20, 2026, 03:13 PM UTC
We've identified the cause and have disabled the Work Rules feature temporarily to mitigate the issue. Further updates are expected.
- resolved Jul 20, 2026, 06:01 PM UTC
This incident has been resolved.
Major July 15, 2026 Mobile login issues 3h 10m Rhumbix
Started Jul 15, 2026, 02:15 PM UTC· resolved Jul 15, 2026, 05:25 PM UTC
- investigating Jul 15, 2026, 02:15 PM UTC
There is an issue impacting login from mobile on the latest version of the app. We are investigating.
- identified Jul 15, 2026, 02:56 PM UTC
We have identified the issue and are working on a solution
- monitoring Jul 15, 2026, 03:42 PM UTC
We have implemented a fix at the network level, users may experience temporary connection issues until DNS flushes.
- resolved Jul 15, 2026, 05:25 PM UTC
This incident has been resolved.
Major January 7, 2026 Logins Receiving "Invalid Token" Message 3h 57m Rhumbix
Started Jan 07, 2026, 06:32 PM UTC· resolved Jan 07, 2026, 10:30 PM UTC
- investigating Jan 07, 2026, 06:32 PM UTC
We are getting a large number of reports of users receiving "Invalid token" error messages when attempting to log in. We are investigating the issue.
- identified Jan 07, 2026, 07:26 PM UTC
The issue has been identified and we are working on a fix.
- identified Jan 07, 2026, 07:39 PM UTC
While we continue to work on a fix, the "Invalid token" login issue can be resolved on a user-by-user basis by clearing cookies and cache in the browser. Instructions on how to do so can be found at https://intercom.help/rhumbix-helpcenter/en/articles/6369184-quick-start-guide-clearing-browser-history
- resolved Jan 08, 2026, 12:46 AM UTC
This incident was resolved at 5:10pm ET. We have monitoring the issue and all "Invalid token" errors have been cleared.
- postmortem Jan 08, 2026, 05:02 PM UTC
## **Executive Summary** Two separate authentication-related issues occurred: 1. A partial SSO login disruption caused by an update to an authentication library 2. A separate issue that resulted in “Invalid token” errors for a limited number of users This release was part of our normal deployment cycle and included routine infrastructure and library updates. Based on the information available at the time, there was no indication that these changes posed any greater risk than a standard release. These issues had a related root cause, but were separate and affected different user groups. Both have been fully resolved, and preventive improvements have been implemented. ## **Incident 1: Partial SSO Login Disruption** ### **Impact** * **Affected users:** Some customers using SSO * **Symptoms:** Unable to authenticate via SSO, preventing user login * **Duration:** Approximately 2–3 hours * **Data impact:** None ### **Timeline \(CT\)** * **06:24** – Reports received of SSO login failures * **07:12** – Root cause identified * **10:32** – SSO issue resolved ### **Root Cause** A routine release included an update to an authentication library that enforced stricter validation of the Issuer URI used in SSO configurations and which invalidated existing tokens. While Rhumbix documentation has long reflected the recommended Issuer URI format, some customers were configured using an older format that was still accepted by the previous library. The updated library no longer supported this legacy format. Once affected configurations were updated to the recommended format, SSO access was restored. ### **Corrective & Preventive Actions** * Verified all supported SSO configurations against current library requirements * Updated affected tenant configurations * Confirmed normal SSO behavior across customers * Adding additional validation for authentication-related library updates prior to release ## **Incident 2: Token-Based Authentication “Invalid Token” Errors** ### **Impact** * **Affected users:** A limited subset of non-SSO users * **Symptoms:** “Invalid token” errors during login * **Duration:** Intermittent issues throughout the day * **Scope:** Smaller user population than SSO * **Data impact:** None ### **Timeline \(CT\)** * **11:26** – Reports received of “Invalid token” errors * **11:39** – Root cause identified * **14:30** – Temporary user-level mitigation identified * **17:10** – Issue fully resolved ### **Root Cause** A similar authentication change resulted in the invalidation of certain cached or expired authentication tokens. This surfaced edge cases related to token lifecycle handling that were not previously observable. This issue was independent of SSO and did not affect SSO authentication flows. ### **Corrective & Preventive Actions** * Resolved token validation logic causing erroneous rejections * Cleared invalid tokens and confirmed successful re-authentication * Increased monitoring around token validation errors * Expanding automated testing for token lifecycle scenarios \(expired, revoked, deleted\)
Major January 7, 2026 Partial SSO Outage 2h 2m Rhumbix
Started Jan 07, 2026, 01:10 PM UTC· resolved Jan 07, 2026, 03:12 PM UTC
- investigating Jan 07, 2026, 01:10 PM UTC
We have received reports that some users are unable to login using SSO. We are investigating the issue.
- identified Jan 07, 2026, 02:24 PM UTC
We have identified the issue and are working on a solution.
- resolved Jan 07, 2026, 03:12 PM UTC
Due to an update to a library supporting SSO all SSO configurations required updating the Issuer URI to include the tenant ID in place of "common". All SSO users should now be able to log in.
Minor February 5, 2025 Public API Cost Code Endpoint 1h 38m Rhumbix
Started Feb 05, 2025, 01:50 AM UTC· resolved Feb 05, 2025, 03:29 AM UTC
- identified Feb 05, 2025, 01:50 AM UTC
An issue has been identified in a performance update for the cost codes endpoint resulting in the job_number field being missing from the response, resulting in downstream errors from integrations. A fix is being processed.
- identified Feb 05, 2025, 02:39 AM UTC
The fix caused further issues. We're rolling back the performance changes to a known working state.
- monitoring Feb 05, 2025, 03:24 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved Feb 05, 2025, 03:29 AM UTC
This incident has been resolved.