Is GUIDEcx down?

This incident has been resolved. More details will follow in the RCA.

$18

Type: Incident Aug 31, 15:00:00 GMT+0 - Resolved - Aug 31, 17:25:01 GMT+0 - Postmortem - August 31, 2026 **Summary** On August 31, 2026, an issue occurred with our email notification queue that affected providers using GUIDEcx 1.0\. A legacy email service outage over the weekend caused outbound jobs to accumulate, resulting in duplicate email notifications being sent when the jobs were restarted. The duplicate sending was contained, and email services are operating normally. **Impact** Some providers associated with GUIDEcx 1.0 accounts received multiple duplicate email notifications during the morning hours. The incident was identified and contained earlier this morning. System monitoring confirms that no additional duplicate emails are being generated or sent. **Resolution** Our engineering team halted the queued outbound email jobs to stop additional duplicate messages from sending. Once contained, the team verified queue states and stabilized the email dispatch pipeline to ensure normal message flow resumed safely without further duplicate processing. **Root Cause** Our legacy email service went offline over the weekend, leading to a backlog of queued email jobs. When the service was restarted this morning, the backed-up jobs executed simultaneously, triggering duplicate email transmissions to affected providers on GUIDEcx 1.0. **Corrective and Preventive Actions** * Legacy email stack has been updated to alert if this issue arises again. * GUIDE 2.0 email service already contains duplicate email filtering logic. * Communicate improved stability benefits in GUIDE 2.0 to customers still using GUIDE 1.0 projects. We apologize for any confusion or inconvenience this may have caused your providers and appreciate your patience as our team worked to resolve the issue.

We are currently investigating this incident.

We have identified the issue and the remediation is in progress.

We are continuing to work on a fix for this incident.

While we aren't fully resolved, many users are now able to navigate the app.

We implemented a fix and are currently monitoring the result.

This incident has been resolved.

$18

Type: Incident Duration: 1 hour and 41 minutes Affected Components: API v1, API v2, Advanced Time Tracking, Report Navigator and Report Builder, Compass Customer Portal, Resource Management, Project Management Aug 12, 21:31:46 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Aug 12, 21:53:18 GMT+0 - Resolved - This incident has been resolved. Aug 12, 20:42:17 GMT+0 - Identified - We are continuing to work on a fix for this incident. Aug 12, 20:11:53 GMT+0 - Investigating - We are currently investigating this incident. Aug 12, 21:05:28 GMT+0 - Identified - While we aren't fully resolved, many users are now able to navigate the app. Aug 12, 20:24:19 GMT+0 - Identified - We have identified the issue and the remediation is in progress. Aug 13, 17:46:50 GMT+0 - Postmortem - Post-Mortem: Production Availability Incident August 12, 2026 Summary On August 12, 2026, a production deployment caused an availability issue that affected portions of the application. A deployment configuration problem caused several services to become unavailable. The issue was corrected and services were restored progressively while system health was monitored. Impact Some customers experienced errors or were unable to load parts of the application. The incident lasted approximately 1 hour and 46 minutes, from 2:07 PM to 3:53 PM MDT. Service began recovering after approximately 50 minutes, with additional functionality returning progressively until full resolution. Resolution Our engineering team identified the deployment configuration issue and updated the production deployment to use the correct configuration and application versions. Services were restored gradually while we monitored application health to ensure the changes were stable before returning to full traffic. Root Cause A configuration difference between our deployment environments caused production services to receive an incorrect scaling configuration during a deployment. This resulted in several services becoming unavailable. The configuration worked correctly in our staging environment, which made the issue less apparent before the production deployment. Corrective and Preventive Actions • Standardize deployment configuration between staging and production. • Add automated checks to prevent unexpected service scaling during deployments. • Validate production deployment configuration before changes are released.. • Continue using progressive deployments and health monitoring to reduce risk during releases. We apologize for the disruption and appreciate your patience while our team worked to restore service.

We are currently investigating this incident.

Latency has improved and most users able to navigate the app without issues. We are continuing to work on identifying the root cause.

This incident has been resolved.