Eptura Outage History

Started Dec 09, 2025, 05:33 PM UTC· resolved Dec 11, 2025, 12:39 AM UTC

1. investigating Dec 09, 2025, 05:33 PM UTC

   We’re aware that some customers are currently experiencing disruptions while navigating SiQ. You may encounter certain errors during your session, and we understand how frustrating this can be. Please know that our team is actively working to resolve this as quickly as possible. We’ll share our next update at 9:32 PM UTC. Thank you for your patience and understanding.

2. investigating Dec 09, 2025, 07:11 PM UTC

   We’ve moved into the monitoring phase after our cloud operations team identified the issue and restarted the affected services. We’re closely monitoring to ensure everything remains stable and fully restored. Thank you for your patience as we work through this. We’ll continue monitoring over the next 4 hours and provide another update at that time.

3. monitoring Dec 09, 2025, 07:25 PM UTC

   We’re aware that some customers are currently experiencing disruptions while navigating SiQ. You may encounter certain errors during your session, and we understand how frustrating this can be. Please know that our team is actively working to resolve this as quickly as possible. We’ll share our next update at 9:32 PM UTC. Thank you for your patience and understanding.

4. monitoring Dec 09, 2025, 11:19 PM UTC

   We will continue monitoring for the next 4 hours to ensure full stability. Our priority is confirming that all customers previously impacted are now fully operational and no longer experiencing any issues. Thank you for your patience as we work through this.

5. monitoring Dec 10, 2025, 03:01 AM UTC

   We will continue monitoring throughout the evening to ensure full stability and confirm that customers who previously reported disruptions are no longer experiencing any issues. We remain committed to transparency and will provide the next update by Wednesday, Dec 10th at 3:30 PM UTC.

6. monitoring Dec 10, 2025, 02:52 PM UTC

   Monitoring continues for the S2 incident. We are actively observing system performance to ensure ongoing stability. No issues have been detected.

7. resolved Dec 11, 2025, 12:39 AM UTC

   We will be moving into a Resolved status. Users have confirmed both internally and externally that this issue has been fully resolved. We sincerely appreciate your patience and understanding as we worked through this disruption. Our team is preparing a detailed Root Cause Analysis, which will be published within the next 10 business days. Thank you for your continued trust and partnership.

Read the full incident report →

Started Dec 08, 2025, 02:40 PM UTC· resolved Dec 12, 2025, 02:53 PM UTC

1. investigating Dec 08, 2025, 02:40 PM UTC

   Our team is currently investigating reports of increased loading times and slow response in the Visitor admin dashboard.

2. identified Dec 08, 2025, 03:20 PM UTC

   We are currently reverting the recent release to maintain system stability and prevent any potential disruptions. During this process, customers may notice that new features introduced in the latest release are temporarily unavailable, but core functionality remains unaffected. Our CloudOps team has initiated the rollback and is closely monitoring progress to ensure a smooth transition. Once the rollback is complete and system performance is validated, we will provide confirmation. Thank you for your patience as we work to resolve this promptly.

3. monitoring Dec 08, 2025, 07:26 PM UTC

   The required rollback has been completed. Response times have gone back to a normal expected level. We are continuing to monitor this situation and will provide additional updates as they become available. Thank you for your patience as we work to resolve this issue.

4. investigating Dec 09, 2025, 10:41 AM UTC

   We are aware that the issue has resurfaced, resulting in elevated response times. Our internal team is actively investigating the cause and working to restore normal performance as quickly as possible. We will share further updates as soon as more information becomes available. Thank you for your patience.

5. identified Dec 09, 2025, 01:43 PM UTC

   Our systems are up and running, and the database is healthy. However, we noticed slower API response times due to a configuration limit. We’ve restarted key services to improve performance. We also found an issue in the SCIM service that caused the disruption. Our engineering team is working on the fix. Thank you for your patience as we work to restore full functionality.

6. identified Dec 09, 2025, 03:11 PM UTC

   We have identified the root cause and are now applying a hotfix. The deployment is expected to take approximately one hour. During this time, some services may be temporarily unavailable. We will provide an update once the hotfix has been successfully applied and validated.

7. identified Dec 09, 2025, 08:52 PM UTC

   Investigating – We are continuing to experience degraded performance and intermittent issues accessing the site. Earlier today, we implemented a hotfix, but it did not deliver the expected improvement. Our teams are actively investigating the root cause and working to restore normal performance as quickly as possible. We will share further updates as soon as more information becomes available. Thank you for your patience while we work to restore normal performance.

8. investigating Dec 10, 2025, 12:00 AM UTC

   We are still seeing degraded performance and intermittent access issues on the site. The hotfix applied earlier did not provide the expected improvement. Our teams remain focused on identifying the root cause and implementing a solution to restore normal performance as quickly as possible. We will continue to share updates as soon as we have more information. Thank you for your continued patience while we work to resolve this issue.

9. investigating Dec 10, 2025, 09:27 AM UTC

   Our engineering teams continue to closely monitor the elevated query activity and the resulting performance impact on the site. They are actively analysing system behaviour and validating several paths to stabilise performance. Additional diagnostics are underway, and we will provide further updates as soon as more information becomes available. Thank you for your continued patience as we work to ensure full service reliability.

10. investigating Dec 10, 2025, 07:23 PM UTC

    Our teams continue to actively work on this issue and have escalated it to third-party vendors for additional support. The root cause remains unknown at this time. As a mitigation step, we are adding additional infrastructure. These changes are significant and require careful execution to ensure stability and safety. We expect these efforts to be completed by 04:30 AM UTC. We will provide the next update after that point. Thank you for your continued patience as we work to restore full service reliability

11. identified Dec 11, 2025, 08:12 AM UTC

    Our teams have identified specific database queries contributing to delays within the platform and have shared these findings with our engineering group. Work is underway on targeted improvements aimed at addressing the behavior we are seeing. We will continue to monitor closely as these changes progress and will provide further updates as more information becomes available. Thank you for your patience as we move toward full resolution.

12. monitoring Dec 11, 2025, 12:21 PM UTC

    System performance has improved after increasing API server capacity and enabling multi-core support. An additional API server has been added, and response times remain stable. We are monitoring the system closely and will share further updates as needed. Thank you for your continued patience.

13. resolved Dec 12, 2025, 02:53 PM UTC

    We are pleased to inform you that the issue with the 'admin dashboard slowness' has been resolved. Our team has completed the necessary actions and verified that the service is now functioning normally. A Root Cause Analysis (RCA) will be conducted to understand the incident in detail and will be made available on our Status Page within 10 days. Thank you for your patience and cooperation throughout this process. If you have any further questions or concerns, please feel free to reach out.

14. postmortem Jan 09, 2026, 03:55 PM UTC

    We are grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident. **Type of Event:** S2 - Performance issue impacting dashboard and check-in flows **Services/Modules Impacted:** All services **Issue Summary/Background:** Between December 8 and December 10, 2025, some customers experienced slower load times and occasional timeouts when using the Eptura Visitor dashboard and related workflows. In certain cases, this affected check-in flows and badge printing. While the service remained available, performance did not meet our usual standards during peak usage periods. **Root Cause:** The issue began after a scheduled update and continued during high-traffic periods, even after the update was rolled back. Our investigation revealed that the root cause was a configuration setting that limited Eptura Visitor services to single-core processing per server. Although servers had multiple cores, Node.js processes were not using them, creating a bottleneck under heavy load. **Remediation:** To restore performance and stabilize the platform, we took immediate action: * Rolled back the recent update to reduce risk * Increased server capacity and optimized runtime settings * Enabled multi-core processing and clustering for better load distribution * Conducted a full infrastructure health review to confirm stability These steps successfully restored normal performance and improved baseline response times. **Timeline \(UTC\):** December 8, 2026 14:19 – First customer reporting of slow performance. 18:18 - Eptura support engaged the CloudOps team for investigation. 19:46 - Multiple customers impacted, case count increased. The issue severity was updated to S2. 20:35 – As part of the initial investigation, Eptura initiated a rollback of a recent update as a precautionary measure. 21:45 – The rollback was completed, and Eptura started checking with customers for feedback. 23:25 – Performance showed temporary improvement, and the incident remained under observation. December 9, 2025 03:19 – The incident was escalated internally to bring in additional teams. 19:18 – A corrective fix was deployed. 23:03 – Monitoring showed that the fix did not fully resolve the issue, and investigation continued. December 10, 2025 10:28 – Eptura Engineering and Cloud teams began joint, in-depth troubleshooting. 17:18 – Investigation was expanded to include platform and infrastructure partners. 23:43 – Additional capacity and configuration improvements were implemented to stabilize performance. December 11, 2025 10:05 – We observed improvement in system performance. 14:48 – Additional capacity was added to support sustained recovery. 17:46 – Eptura support started receiving confirmation of resolution. December 12, 2025 19:01 – Monitoring confirmed performance had returned to normal levels. December 15, 2025 11:25 – Continued monitoring confirmed long-term stability. December 22, 2025 11:51 – The incident was formally resolved and closed. **Total Duration of Event:** 75 hours **Preventive Actions:** To prevent similar incidents in the future, we are implementing the following improvements: * Validate multi-core runtime profiles and clustering for all critical Eptura Visitor services * Strengthen scaling and failover policies for peak traffic scenarios * Enhance monitoring and alerting to detect performance degradation earlier All services are operating normally, and we continue to monitor the platform closely. Thank you for your understanding as we work to deliver the reliable experience you expect.

Read the full incident report →

Started Dec 05, 2025, 09:22 AM UTC· resolved Dec 05, 2025, 09:31 AM UTC

1. investigating Dec 05, 2025, 09:22 AM UTC

   We have identified that the issues currently affecting Eptura products are related to a global Cloudflare incident https://www.cloudflarestatus.com/ Our teams are actively monitoring the situation, and we will continue to update this incident.

2. monitoring Dec 05, 2025, 09:27 AM UTC

   Cloudflare has confirmed that their services are now operating normally, and they are no longer observing elevated errors or delays.

3. resolved Dec 05, 2025, 09:31 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Dec 04, 2025, 09:47 PM UTC· resolved Dec 08, 2025, 10:31 PM UTC

1. investigating Dec 04, 2025, 09:47 PM UTC

   We are currently investigating a disruption with Automated Reports: Character Encoding & Quoting. Our Engineering team is actively working to determine the root cause of the disruption and assess its impact. We will provide our next update by 7:46 PM CST. Thank you for your patience as we work to resolve this issue.

2. investigating Dec 05, 2025, 04:02 AM UTC

   We are currently investigating a disruption affecting Automated Reports: Character Encoding and Quoting. Our Engineering team is actively reviewing system behaviour to determine the root cause and understand the extent of the impact. In parallel, the team is also exploring potential workarounds that may help reduce any immediate disruption while the permanent fix is being established. We will continue to share updates as soon as new information becomes available. Thank you for your patience while we work on this.

3. identified Dec 05, 2025, 02:11 PM UTC

   Our Engineering team has completed the investigation into the Automated Reports issue related to character encoding and unexpected double-quoting. Based on the assessment, we will be rolling back to the previous stable version. The rollback is currently being executed in UAT, and the team is performing the required sanity checks tonight. Once validated, we will proceed with the production rollback tomorrow. We will share another update once the rollback is successfully completed in production. Thank you for your patience and cooperation.

4. monitoring Dec 08, 2025, 08:57 AM UTC

   The rollback has been successfully completed in production and our Engineering team is closely monitoring system behavior to ensure continued stability and expected performance across all affected reporting workflows. We will provide additional updates if any further action is required. Thank you for your patience and cooperation.

5. resolved Dec 08, 2025, 10:31 PM UTC

   The rollback was successful. We have several customers confirming. A Root Cause Analysis (RCA) will be conducted to understand the incident in detail and will be made available on our Status Page within 10 days. Thank you for your patience and cooperation throughout this process. If you have any further questions or concerns, please feel free to reach out.

Read the full incident report →

Started Dec 04, 2025, 03:31 PM UTC· resolved Dec 04, 2025, 07:20 PM UTC

1. monitoring Dec 04, 2025, 03:31 PM UTC

   Issue: HTTP Error 503 – The service was unavailable Description: Users experienced service interruptions due to an HTTP 503 error. Status: The issue has been resolved, and the service is now fully operational. We are currently monitoring to ensure stability. Thank you for your patience while we worked to restore service.

2. resolved Dec 04, 2025, 07:20 PM UTC

   This incident has been resolved.

3. postmortem Dec 17, 2025, 03:52 PM UTC

   **Eptura Asset detailed Root Cause Analysis | Dec. 4, 2025** We are grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident. **Description:** On Dec. 4, 2025, some customers in Utah and Iowa experienced intermittent access issues when visiting certain URLs on our website. Affected users encountered HTTP 503 errors, while internal systems remained fully operational. The issue was limited in scope and did not impact customers outside these regions. **Type of Event:** Outage **Services/Modules impacted:** * Services affected: Specific URLs in Utah and Iowa * No widespread outage; only a small subset of customers impacted * Severity adjusted from S1 to S2 after confirming limited scope **Timeline:** * **Dec 4th – 2:35pm Initial Report:** Customers reported HTTP 503 errors * **Dec 4th – 2:48pm Investigation:** Verified regional impact; confirmed URLs worked internally * **Dec 4th – 3:31pm Root Cause Identified:** A server \(legacy-win-app8\) failed to restart after routine maintenance * **Dec 4th – 3:35pm Resolution:** * Rebooted affected server * Rerouted traffic via load balancer * Restored service; some browsers required cache refresh **Total duration of event:** 1 hr **Root cause:** The issue was traced to a technical failure on one of the virtual machines during routine maintenance. This unexpected system error prevented the server from restarting properly, which impacted access for customers in specific regions. The failure was isolated and did not affect other services or regions. **Remediation and preventative action:** * The affected server was restored with support from our infrastructure partner, and traffic was rerouted to ensure continuity. * We are working closely with our cloud provider to complete a detailed analysis and confirm the underlying cause. * Additional safeguards are being implemented, including enhanced monitoring and automated failover processes, to prevent similar issues in the future. **Current status:** All services are fully operational. We are committed to maintaining reliability and have taken steps to strengthen our systems. Thank you for your patience and trust as we continue to improve your experience.

Read the full incident report →

Started Nov 14, 2025, 05:13 PM UTC· resolved Nov 14, 2025, 07:08 PM UTC

1. investigating Nov 14, 2025, 05:13 PM UTC

   We are currently investigating an issue where users are Unable to access the site via SSO. Our Engineering team is currently investigating to determine the cause of the disruption. As a workaround, you can log in using a username and password. Next update will be posted at 3 PM CST.

2. investigating Nov 14, 2025, 06:11 PM UTC

   We have implemented a fix for the issue preventing users from accessing the site via SSO. We are now in the Monitoring Phase, which will continue for the next hour. Thank you for your patience as we work to confirm full resolution.

3. monitoring Nov 14, 2025, 06:11 PM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Nov 14, 2025, 07:08 PM UTC

   As we have not seen further service disruptions after the fix was implemented, we have moved to the Resolved Phase. Thank you for your patience and understanding while we worked to resolve the issue. We sincerely apologize for any inconvenience this may have caused.

Read the full incident report →

Started Nov 11, 2025, 06:19 AM UTC· resolved Nov 11, 2025, 06:19 AM UTC

1. resolved Nov 19, 2025, 11:32 PM UTC

   Some customers experienced issues when reserving desks via the Outlook plugin. Reservations were not being completed, and users received a [Postmaster] Email Delivery Warning instead of the usual confirmation email

2. postmortem Nov 19, 2025, 11:33 PM UTC

   **Teem by Eptura detailed Root Cause Analysis | 11/19/2025** **S2 – Email Notifications** We are grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident. **Description:** Some customers experienced issues when reserving desks via the Outlook plugin. Reservations were not being completed, and users received a \[Postmaster\] Email Delivery Warning instead of the usual confirmation email. **Type of Event:** Functionality Issue **Services/Modules impacted:** Production/ Reservation Confirmation Emails **Timeline \(UTC\):**` ` On November 11, 2025, at 6:19 AM, we received the first report from a customer indicating they had not received a check-in email after reserving a desk. Our support team responded and began troubleshooting steps. Later that afternoon, at 3:54 PM, two additional customers reported similar issues with email notifications. At 4:06 PM, the issue was escalated to our Cloud Operations team for investigation. By 4:26 PM, Cloud Operations acknowledged the problem and began working on a resolution. At 4:36 PM, the root cause was identified as a configuration issue, and corrective action was taken immediately. By 4:51 PM, the issue was fully resolved, and customers were notified that email delivery was restored. **Total Duration of Event:** 10 hours, 32 minutes **Root Cause:** An unexpected configuration issue in our email delivery system temporarily disrupted the sending of reservation confirmation emails. This caused desk reservations to appear incomplete. **Remediation:** The configuration issue has been corrected, and email delivery has been fully restored. Reservations via the Outlook plugin are now functioning as expected. **Preventative Action:** We have implemented measures to prevent similar disruptions in the future. Our team is creating documentation to strengthen monitoring and ensure continuity.

Read the full incident report →

Started Nov 10, 2025, 04:45 AM UTC· resolved Nov 10, 2025, 04:45 AM UTC

1. resolved Nov 19, 2025, 11:29 PM UTC

   Customers experienced issues where floor plans and images did not load when navigating the platform. Additionally, user imports were not processed as expected.

2. postmortem Nov 19, 2025, 11:29 PM UTC

   **SpaceIQ by Eptura detailed Root Cause Analysis | 11/19/2025** **S2 - SiQ Floorplans and Images not loading** We are grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident. **Description:** Customers experienced issues where floor plans and images did not load when navigating the platform. Additionally, user imports were not processed as expected. **Type of Event:** Functionality Issue **Services/Modules impacted:** Production/ Floorplan images & User Imports via SFTP **Timeline \(UTC\):** On 11/09/2025 at 9:45 PM, the first report of floor plans not generating was submitted to support, followed by two additional cases. On 11/10/2025, more cases were created early in the morning related to both floor plan display issues and user imports. By 2:21 PM, internal teams were notified about the disruption, and engineering confirmed the issue was linked to AWS permissions configuration. At 2:33 PM, engineering worked with AWS to resolve the issue, and customers confirmed functionality was restored. **Total Duration of Event:** 16 hours, 48 minutes **Root Cause:** AWS security policies require specific permissions to access media files. To make files publicly accessible, a public URL must be created with appropriate access settings. A configuration issue impacted these permissions, resulting in failures when loading images and floor plans, and preventing SFTP user imports from processing. **Remediation:** Permissions have been updated to restore access and functionality. **Preventative Action:** We have implemented a proactive alert system that notifies stakeholders ahead of any scheduled URL changes. This ensures teams have sufficient time to review and update links, maintaining uninterrupted access and a seamless customer experience.

Read the full incident report →

Started Oct 30, 2025, 02:23 PM UTC· resolved Oct 30, 2025, 04:19 PM UTC

1. investigating Oct 30, 2025, 02:23 PM UTC

   We are currently investigating an issue affecting file processing for several clients in the U.S. Files submitted earlier today were not successfully imported into the system. Our team has identified that the expected files are not present in the usual processing queues, and we are actively reviewing system logs and delivery paths to determine the root cause. We understand the importance of timely file processing and are working to resolve the issue as quickly as possible. We will provide updates as soon as more information becomes available. Thank you for your patience and understanding. Our next update will be 1:22 PM CST or sooner.

2. monitoring Oct 30, 2025, 03:13 PM UTC

   We have implemented a fix for the issue causing delays in Serraview file processing for U.S. customers. Our teams are actively monitoring the system to ensure stability and optimal performance. Cloud Operations and Support are overseeing the process to confirm full resolution. The next update will be provided by 2:15 PM CST.

3. resolved Oct 30, 2025, 04:19 PM UTC

   The issue causing delays in Serraview file processing for U.S. customers has been fully resolved. Customers have confirmed that files are now processing as expected. Our team has verified system stability and will continue to monitor performance to ensure everything remains operational. A Root Cause Analysis (RCA) will be conducted to understand the incident in detail and will be made available on our Status Page within 10 days. Thank you for your patience and cooperation throughout this process. If you have any further questions or concerns, please feel free to reach out.

Read the full incident report →

Started Oct 29, 2025, 04:53 PM UTC· resolved Oct 30, 2025, 03:10 AM UTC

1. investigating Oct 29, 2025, 04:53 PM UTC

   We have received reports of Engage Outlook add-in and Device Hub not loading. While our teams are investigating the issue, it seems to be related to recently reported Microsoft outage affecting multiple systems.

2. investigating Oct 29, 2025, 04:57 PM UTC

   We are continuing to investigate this issue.

3. investigating Oct 29, 2025, 05:40 PM UTC

   We are continuing to investigate this issue.

4. identified Oct 29, 2025, 05:46 PM UTC

   We have identified that the issues currently affecting Eptura Engage and Eptura Visitor are related to a global Microsoft Azure incident. Impacted Services: Engage Outlook Plug-in Engage Device Hub Visitor Outlook Plug-in Additionally, syncing of calendar events between Engage and Microsoft Exchange may experience intermittent disruptions. Our teams are actively monitoring the situation and we will continue to update this incident.

5. identified Oct 29, 2025, 07:02 PM UTC

   We have identified that the issues currently affecting Eptura Engage, Eptura Visitor, and Eptura Platform V2 are related to a global Microsoft Azure incident. Impacted Services: Engage Outlook Plug-in Engage Device Hub Visitor Outlook Plug-in Platform-V2 Core Services Additionally, syncing of calendar events between Engage and Microsoft Exchange may experience intermittent disruptions. Our teams are actively monitoring the situation and we will continue to update this incident.

6. identified Oct 29, 2025, 07:38 PM UTC

   Microsoft has implemented recovery measures for the global Azure incident. As these changes propagate through Azure, we are beginning to see partial recovery of impacted services. Please note that availability remains intermittent for the following Eptura services: Engage Outlook Plug-in Engage Device Hub Visitor Outlook Plug-in Calendar syncing between Engage and Exchange Platform-V2 Core Services Our teams continue to monitor progress and will provide further updates as stability improves. Thank you for your patience during this recovery process.

7. identified Oct 29, 2025, 07:56 PM UTC

   Microsoft has implemented recovery measures for the global Azure incident. As these changes propagate through Azure, we are beginning to see partial recovery of impacted services. Please note that availability remains intermittent for the following Eptura services: Engage Outlook Plug-in Engage Device Hub Visitor Outlook Plug-in Calendar syncing between Engage and Exchange Platform-V2 Core Services Workplace Hummingbird Outlook plug-in Our teams continue to monitor progress and will provide further updates as stability improves. Thank you for your patience during this recovery process.

8. identified Oct 29, 2025, 10:43 PM UTC

   Update – Microsoft has implemented recovery measures for the global Azure incident, and services are stabilizing. Engage and Visitor are now functional and we are awaiting confirmation on the remaining impacted Eptura services listed below: Platform-V2 Core Services Workplace Hummingbird Outlook Plug-in Our teams continue to monitor progress and will provide further updates as stability improves. Thank you for your patience during this recovery process.

9. monitoring Oct 29, 2025, 10:46 PM UTC

   A fix has been implemented and we are monitoring the results.

10. resolved Oct 30, 2025, 03:10 AM UTC

    Microsoft has confirmed that the global Azure incident causing service disruptions has been fully resolved. As this was the underlying cause of the recent issues, we are now marking this incident as resolved. We appreciate your patience and understanding while this issue was being investigated and remediated.

Read the full incident report →

Started Oct 28, 2025, 09:55 PM UTC· resolved Oct 29, 2025, 09:54 PM UTC

1. investigating Oct 28, 2025, 09:55 PM UTC

   We are currently investigating an issue with the Asset Work Companion/Eptura Asset App not being visible in the Apple Store. Our Product team is currently investigating to determine the cause and to get the app available for download as soon as possible. We appreciate your patience as we work through this issue. Next update will be posted at 1700 UTC October 29th 2025.

2. investigating Oct 29, 2025, 05:03 PM UTC

   We are continuing to investigate the issue preventing the Asset Work Companion (Eptura Asset App) from appearing in the Apple App Store. Our team is working to identify the root cause and restore availability as soon as possible. We appreciate your patience and understanding as we work through this. Next update will be posted by 1700 UTC on October 30, 2025, or sooner if more information becomes available.

3. identified Oct 29, 2025, 05:31 PM UTC

   .

4. resolved Oct 29, 2025, 09:54 PM UTC

   Our Product team has confirmed that the Eptura Asset Companion app is now visible in the Apple store again. We have moved to the Resolved Phase. A Preliminary RCA will be posted in this incident in 2 business days. Please stay subscribed to the page to receive post automatically.

5. postmortem Nov 14, 2025, 05:43 PM UTC

   **Eptura Asset detailed Root Cause Analysis | Oct. 27, 2025** We are grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident. **Description:** Eptura Asset mobile WC app unavailable in the app store **Type of Event:** Outage **Services/Modules impacted:** Eptura Asset mobile WC app **Timeline:** Oct 27th 2025 8:24am Customers begin notifying that the WC App was missing from the Apple Store Oct 27th 2025 8:30am investigation into issue and confirmation Oct 27th 2025 8:45am JIRA Created and investigation began Oct 28th 2025 11:53pm App confirmed not visible but not removed from Apple Store Oct 29th 2025 3:53pm Root Cause Identified – investigation on solution Oct 29th 2025 9:19pm Confirmed fixed and Searchable in Apple Store again **Total duration of event:** 60 hours **Root cause:** On October 27, the Eptura Asset mobile app was temporarily unavailable in the App Store due to an unexpected issue, which affected the app's listing. **Remediation:** We promptly contacted the App Store support team and collaborated with them to identify and fix the problem, ensuring the app was restored to the App Store as quickly as possible. **Preventative action:** To prevent future occurrences, we have implemented a rigorous review process for app updates and metadata changes, and will continue to maintain regular communication with the App Store support team.

Read the full incident report →

Started Oct 22, 2025, 09:01 AM UTC· resolved Nov 04, 2025, 02:28 PM UTC

1. investigating Oct 22, 2025, 09:01 AM UTC

   Users encounter a 504 Gateway Timeout error when sending new invitation emails through Visitor (ProxyClick) or when recipients click the pre-registration link included in those emails. Our Engineering team is currently investigating to determine the cause of the disruption. Next update will be posted at 4 AM CST.

2. investigating Oct 22, 2025, 10:05 AM UTC

   We are actively continuing our investigation into the reported issue. To facilitate a thorough analysis, a support ticket has been raised with Microsoft, and their team is currently engaged in further diagnostics. We will provide the next update by 9:00 AM CST, or sooner if any significant developments occur.

3. investigating Oct 22, 2025, 02:00 PM UTC

   Our infra team has reviewed the relevant logs for email delivery failures, but no errors have been found. Our Engineering team is also involved in further investigation of the issue. We will provide the next update by 1:00 PM CST, or sooner if any significant developments occur.

4. investigating Oct 22, 2025, 07:24 PM UTC

   We are actively continuing our investigation into the reported issue. We will provide the next update by 6:00 PM CST, or sooner if any significant developments occur.

5. investigating Oct 22, 2025, 11:24 PM UTC

   We are actively continuing our investigation into the reported issue concerning users encountering a 504 Gateway Timeout error when sending new invitation emails through Visitor (ProxyClick) or when recipients click the pre-registration link included in those emails. We will provide the next update by 0400 UTC or sooner if any significant developments occur.

6. monitoring Oct 23, 2025, 04:17 AM UTC

   Our internal team has implemented a temporary workaround by disabling link tracking in SendGrid, which has positive results for the 504 Gateway Timeout error for newly sent invitation emails. Users should now be able to send and access new invitation and pre-registration links successfully. However, previously sent emails before this change may still encounter the error. We are in contact with the SendGrid team to identify and address the root cause within their link tracking service to ensure a permanent resolution. We will continue to closely monitor the system’s performance and provide the next update by 08:00 UTC or sooner if new information becomes available.

7. monitoring Oct 23, 2025, 08:09 AM UTC

   Our teams continue to work with SendGrid, while the temporary mitigation (disabling link tracking) remains effective for newly sent emails, we are still pursuing a permanent fix to fully resolve the issue. We appreciate your continued patience and will share the next update by 12:00 UTC, or sooner if any significant progress is made.

8. monitoring Oct 23, 2025, 12:16 PM UTC

   Our teams are actively collaborating with SendGrid to identify and implement a permanent resolution. In the meantime, the temporary mitigation continues to be effective for newly sent emails. While this measure has helped restore functionality, we recognize the importance of a long-term fix and remain focused on achieving a complete resolution. We will provide the next update by 12:00 UTC on 24th October, or earlier if we achieve any notable progress in our investigation.

9. monitoring Oct 24, 2025, 12:03 PM UTC

   Our teams are working in close coordination with SendGrid to establish a permanent solution. This collaborative effort remains our top priority as we aim to fully restore and stabilize email functionality. We will share the next update by 09:00 UTC on 27th October, or sooner should we make any significant progress in our investigation.

10. monitoring Oct 27, 2025, 09:00 AM UTC

    Our teams continue to work closely with SendGrid to implement a long-term, stable resolution. Restoring full email functionality remains our highest priority and we are committed to ensuring the solution is both effective and sustainable. In the meantime, the temporary mitigation remains in place and continues to be effective for newly sent emails. While this measure has helped restore functionality, we understand the importance of achieving a permanent resolution and remain fully focused on doing so. We will provide the next update by 09:00 UTC on 28th October or sooner if there is significant progress to share. We sincerely appreciate your patience and understanding as we work toward a complete resolution.

11. monitoring Oct 28, 2025, 09:00 AM UTC

    Our teams remain actively engaged with SendGrid as we continue implementing the long-term solution to fully restore and stabilize email functionality. The collaboration is ongoing and all efforts remain focused on ensuring the fix is robust and sustainable. The temporary mitigation continues to perform effectively for newly sent emails, helping to maintain functionality while we work toward the permanent resolution. We will share the next update by 09:00 UTC on 29th October or sooner should there be meaningful progress to report. We truly appreciate your continued patience and understanding as we work diligently toward a complete and lasting resolution.

12. monitoring Oct 29, 2025, 09:40 AM UTC

    Our teams continue to work closely with SendGrid to implement a permanent fix aimed at restoring full email stability. The joint effort is progressing with validation and reliability testing underway to ensure the long-term solution is both effective and sustainable. The temporary mitigation remains stable and continues to support the successful delivery of newly sent emails while permanent measures are finalized. We will provide the next update by 09:00 UTC on 30 October or sooner if there is significant progress to share. We appreciate your continued patience and understanding as we work to ensure a complete and durable resolution to this issue.

13. monitoring Oct 30, 2025, 09:00 AM UTC

    Our teams are actively collaborating with SendGrid to develop and implement a comprehensive, long-term solution that will fully restore the email functionality. This joint initiative is currently in the phase of rigorous validation and reliability testing to ensure that the final fix is not only effective but also resilient over time. In the meantime, the temporary mitigation continues to perform reliably. It is successfully facilitating the delivery of newly generated emails while we work toward deploying the permanent resolution. We are committed to keeping you informed and will share the next update by 09:00 UTC on 31 October, or earlier if we achieve any notable progress. We sincerely appreciate your ongoing patience and understanding as we work diligently to resolve this issue in a thorough and lasting manner.

14. monitoring Oct 31, 2025, 09:02 AM UTC

    We have not received any additional reports of email delivery issues since the temporary mitigation was implemented. This interim solution is currently ensuring the successful delivery of newly generated emails. Our engineering teams are actively working in close coordination with SendGrid to develop and implement a permanent resolution that will fully restore normal email functionality. We will provide the next update by 09:00 UTC on 3 November 2025, or sooner if significant progress is made. We appreciate your continued patience and understanding as we work toward a complete resolution.

15. monitoring Nov 03, 2025, 12:10 PM UTC

    We have not observed any further disruptions since implementing a temporary mitigation. This interim solution is currently ensuring that newly generated emails are being delivered successfully. Our engineering teams remain actively engaged and are working in close collaboration with SendGrid to identify and resolve the root cause of the issue. SendGrid has identified configuration anomalies within Azure Front Door (AFD) that may be contributing to the disruption. Our teams are now focused on developing and deploying a permanent fix to fully restore normal email functionality. We will share the next update by 09:00 UTC on 4 November 2025, or earlier should we make significant progress. We sincerely appreciate your continued patience and understanding as we work diligently toward a comprehensive resolution.

16. monitoring Nov 04, 2025, 09:01 AM UTC

    The temporary mitigation is functioning reliably and continues to ensure the successful delivery of newly generated emails. Once the Azure AFD issue is fully resolved, our infrastructure team will implement the necessary configuration changes and conduct thorough validation of the fix. This process will be carried out in close coordination with SendGrid support to confirm complete resolution. We will share the next update by 09:00 UTC on 5 November 2025, or earlier should we make significant progress.

17. resolved Nov 04, 2025, 02:28 PM UTC

    The temporary mitigation continues to perform reliably, and no further disruptions have been observed in the past 11 days. Newly generated invitation and notification emails are being delivered successfully, and users are no longer encountering 504 Gateway Timeout errors. Our internal teams remain closely engaged with SendGrid and Azure to validate the pending configuration changes once the ongoing Azure Front Door (AFD) issue is fully resolved. Given the current stability, we are moving this status page to Resolved. We will share an interim RCA within 10 days, followed by a full RCA after the permanent resolution has been implemented and validated. We sincerely appreciate your patience and understanding throughout this process.

Read the full incident report →

Started Oct 21, 2025, 01:34 PM UTC· resolved Oct 22, 2025, 07:12 PM UTC

1. investigating Oct 21, 2025, 01:34 PM UTC

   We are currently investigating an issue with EU Environment is Inaccessible. Our Engineering team is currently investigating to determine the cause of the disruption. Next update will be posted at 11 AM CST

2. monitoring Oct 21, 2025, 04:09 PM UTC

   We’ve identified that the issue is isolated to the bypass login method, while SSO login is functioning normally. Health checks are stable, and logs from EU customers confirm successful SAML authentications. The team continues to investigate the bypass login errors. No fix has been implemented yet, but system behavior has stabilized, and we are now in a monitoring phase. We’ll provide further updates as needed. Thank you for your patience.

3. investigating Oct 21, 2025, 11:40 PM UTC

   We are moving back into the investigation phase as customers in the EU region continue to experience issues accessing the platform via SSO. Our team is actively investigating and working to identify the root cause. We will provide updates as more information becomes available. Thank you for your continued patience.

4. monitoring Oct 22, 2025, 08:27 AM UTC

   Our engineering team has identified a configuration issue on one of the EU web servers, causing URL redirect failures for SSO logins. The affected sites have now been migrated to a new web server with corrected configurations. Access to all impacted instances for most clients has been restored. We are closely monitoring the environment to ensure continued stability. Thank you for your patience while we worked to resolve this issue.

5. resolved Oct 22, 2025, 07:12 PM UTC

   We have moved to the Resolved phase. Since migrating the affected sites to a new web server, we have not observed any further service disruptions. Thank you for your patience and understanding while we worked to resolve the issue. We sincerely apologize for any inconvenience this may have caused.

Read the full incident report →

Started Oct 20, 2025, 10:54 AM UTC· resolved Oct 21, 2025, 12:01 AM UTC

1. monitoring Oct 20, 2025, 10:54 AM UTC

   We are currently monitoring an ongoing incident reported by AWS, which you can follow here: https://health.aws.amazon.com/health/status. This incident has directly impacted several Eptura services, including Serraview and Archibus. While both services are now accessible, users may continue to experience intermittent issues as AWS progresses with its recovery efforts. Additionally, Eptura Visitor integrations with third-party services—such as SMS notifications—experienced disruptions. We appreciate your patience and will provide our next update by 15:00 UTC.

2. monitoring Oct 20, 2025, 03:01 PM UTC

   We are continuing to monitor an ongoing incident reported by AWS, which can be followed here: https://health.aws.amazon.com/health/status. This incident directly impacted several Eptura services, including Serraview, Archibus and SpaceIQ. Both Serraview and Archibus services are now accessible, however users may continue to experience intermittent issues as AWS progresses with its recovery efforts. SpaceIQ is reporting that the entire web and mobile application fails intermittently with customers receiving a time out error. Additionally, Eptura Visitor integrations with third-party services such as SMS notifications have experienced disruptions. We appreciate your patience and will provide our next update by 19:00 UTC.

3. monitoring Oct 20, 2025, 04:10 PM UTC

   We are continuing to monitor for any further issues.

4. monitoring Oct 20, 2025, 04:11 PM UTC

   We are continuing to monitor for any further issues.

5. monitoring Oct 20, 2025, 08:39 PM UTC

   We are continuing to monitor an ongoing incident reported by AWS, which can be followed here: https://health.aws.amazon.com/health/status. AWS continues to make progress toward pre-event levels in all Availability Zones and we are continuing to monitor Eptura services. This incident directly impacted several Eptura services, including Serraview, Archibus and SpaceIQ. Both Serraview and Archibus services are accessible. Users may continue to experience intermittent issues as AWS progresses with its recovery efforts. SpaceIQ has reported that the entire web and mobile application fails intermittently with customers receiving a time out error. AWS is working through the backlog of queued events at this time. Additionally, Eptura Visitor integrations with third-party services such as SMS notifications have experienced disruptions. We appreciate your patience and will provide our next update by 23:00 UTC.

6. monitoring Oct 20, 2025, 11:54 PM UTC

   The AWS incident we have been monitoring has been resolved and all AWS services have returned to normal operations. The current status can be followed here: https://health.aws.amazon.com/health/status. This incident directly impacted several Eptura services, including Serraview, Archibus, Visitor and SpaceIQ. We are continuing to monitor all Eptura services to ensure that everything returns to normal operations. Both Serraview and Archibus services are accessible. Users may continue to experience intermittent issues as AWS progresses with its backlog of queued events SpaceIQ has reported that the entire web and mobile application fails intermittently with customers receiving a time out error. AWS continues to work through the backlog of queued events at this time. Additionally, Eptura Visitor integrations with third-party services such as SMS notifications have experienced disruptions. We appreciate your patience and will provide our next update by October 21st, 13:00 UTC.

7. resolved Oct 21, 2025, 12:01 AM UTC

   The AWS incident we have been monitoring has been resolved and all AWS services have returned to normal operations. The current status can be followed here: https://health.aws.amazon.com/health/status. This incident directly impacted several Eptura services, including Serraview, Archibus, Visitor and SpaceIQ. We are continuing to monitor all Eptura services to ensure that everything returns to normal operations. Both Serraview and Archibus services are accessible. Users may continue to experience intermittent issues as AWS progresses with its backlog of queued events SpaceIQ has reported that the entire web and mobile application fails intermittently with customers receiving a time out error. AWS continues to work through the backlog of queued events at this time. Additionally, Eptura Visitor integrations with third-party services such as SMS notifications have experienced disruptions. We appreciate your patience and will provide our next update by October 21st, 13:00 UTC.

Read the full incident report →

Started Oct 07, 2025, 08:16 PM UTC· resolved Oct 13, 2025, 04:43 PM UTC

1. investigating Oct 07, 2025, 08:16 PM UTC

   Summary: We are currently investigating a change in behavior affecting some users of the Engage mobile app following the latest release over the weekend. What’s Happening: Some users are encountering an IntuneMAMEnrollment error pop-up during login when they open the Eptura Engage App. Our team is actively reviewing the reports and working to identify the root cause. Next Steps: While our team is investigating, as a workaround, please uninstall the Microsoft Edge browser from your mobile device and continue with logging into Engage. The next update on this issue will be at 11:30am UTC tomorrow October 8th. We appreciate your patience and will keep this page updated as progress is made.

2. investigating Oct 08, 2025, 11:30 AM UTC

   We are actively working on developing a fix for the issue impacting users of the Engage mobile app. Our engineering team is fully engaged in identifying the root cause and implementing a resolution. We recognize the disruption this may be causing and sincerely apologize for the inconvenience. Please be assured that resolving this issue remains our highest priority, and we are committed to restoring full functionality as quickly as possible. We will share the next update by October 9, 11:30am UTC, or earlier if we have significant progress to report. Thank you for your continued patience and understanding.

3. monitoring Oct 09, 2025, 12:12 PM UTC

   We’re pleased to inform you that our Engineering team has pushed a new version of the Engage app on the App Store. This update is part of our ongoing efforts to improve your experience and address the issue some users have been facing. The updated version will be available for download within the next 24-48 hours. Once available, we strongly recommend all users to update to the latest version of the app. Installing the update is essential to ensure that the reported issue is fully resolved and the app functions smoothly. We will share the next update by October 10, 11:30am UTC, or earlier if we have significant progress to report.

4. monitoring Oct 10, 2025, 03:57 PM UTC

   We’re pleased to share that our Engineering team has released the latest version of the Engage app on the App Store. This release continues our commitment to delivering an improved user experience and resolving the issue previously observed by some users. The updated version is now live and available for download. We request and encourage everyone to update their app at the earliest. We’re also glad to report that we have positive results from a client who has confirmed that the issue is no longer occurring after upgrading to the latest iOS version of the Engage app. Please let us know if you notice any further anomalies or have additional concerns. We will continue to monitor the situation and share updates as needed. We will share the next update by October 13, 11:30am UTC, or earlier if we have significant progress to report. Thank you for your continued patience and understanding.

5. resolved Oct 13, 2025, 04:43 PM UTC

   We’re happy to share that the issue with the Engage Mobile App has been successfully resolved. Our Engineering team has implemented the required fixes and confirmed that the service is now operating as expected. To provide further clarity, a detailed Root Cause Analysis (RCA) will be carried out and published on our Status Page within the next 10 days. We truly appreciate your patience and cooperation during this time.

6. postmortem Oct 17, 2025, 02:59 PM UTC

   We are grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident. ### **Type of event:** _S2 - Eptura Engage -_ iOS mobile app issue Users encountered an _IntuneMAMEnrollment_ error pop-up during login when opening the Eptura Engage iOS mobile app. ### **Services/modules impacted:** Eptura Engage iOS mobile app ### **Issue summary/background:** Following the deployment of **Eptura Engage iOS app version 4.14.2.8** over the weekend of **October 4–5**, users began experiencing an _IntuneMAMEnrollment_ error during the login process. This issue stemmed from an update introduced in this version to support **SSO login via Microsoft Edge**. Due to differing **Intune policy configurations** across clients, some environments experienced login failures, temporarily preventing access to the Eptura Engage mobile app. Because Intune policies differ by client, not all configurations could be replicated during internal testing. It is important to note that **Android users were not impacted**. ### **Root cause:** The Eptura Engage app requires both **Microsoft Edge** and **Eptura Engage** to be managed by **Microsoft Intune** for seamless operation. In certain client environments, the Edge browser was not Intune-managed, which caused a **policy mismatch**. As a result, redirection between Edge and Engage failed, triggering the error. ### **Remediation:** * The Engineering team **rolled back** the changes introduced in version 4.14.2.8 to restore previous login functionality. * A **new build \(version 4.14.3\)** was released to the App Store with the Edge browser-related updates reverted. ### **Timeline \(UTC\):** **06 Oct 2025 15:00** – Initial issue reported **07 Oct 2025 10:03** – Root cause identified and escalated to Eptura CloudOps team **07 Oct 2025 11:05** – Eptura Engineering team engaged due to increased impact and the suspected root cause being the new app rollout **08 Oct 2025 11:51** – The Engineering team confirmed the root cause and initiated rollback **09 Oct 2025 11:00** – Rollback completed successfully **09 Oct 2025 04:11** – Version 4.14.3 released to App Store with the fix ### **Total duration of event:** 68 hours ### **Preventive actions:** To help prevent similar issues in the future, the following actions are being implemented: * Development of a **revised SSO integration for Edge** to ensure compatibility with diverse Intune configurations. * **Expanded testing** will now include a wider range of Intune policy scenarios to better reflect client environments. * Future iOS builds will undergo **staged deployments** for validation prior to full release. We sincerely apologize for any inconvenience this issue may have caused and appreciate your understanding while we worked toward a resolution. Your experience and trust are extremely important to us, and we remain committed to delivering a reliable and seamless experience for all users.

Read the full incident report →

Started Oct 02, 2025, 07:31 PM UTC· resolved Oct 02, 2025, 11:13 PM UTC

1. monitoring Oct 02, 2025, 07:31 PM UTC

   We have implemented a solution for the issue affecting the Asset Service and are currently monitoring the situation to ensure stability and performance. Our Engineering team is overseeing the process to confirm that the issue has been fully resolved. The next update will be provided by 5pm CST.

2. resolved Oct 02, 2025, 11:13 PM UTC

   We are pleased to inform you that the issue with Asset Service has been resolved. Our Engineering team has completed the necessary actions and verified that the service is now functioning normally. A Root Cause Analysis (RCA) will be conducted to understand the incident in detail and will be made available on our Status Page within 10 days. Thank you for your patience and cooperation throughout this process. If you have any further questions or concerns, please feel free to reach out.

3. postmortem Oct 03, 2025, 07:22 PM UTC

   **Eptura Asset detailed Root Cause Analysis | Oct 2nd 2025** We are grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident. **Description:** Asset Service Unavailable **Type of Event:** Outage **Services/Modules impacted:** Asset Service **Timeline:** Oct 2nd 2025 1:55pm – Customers begin reporting that Asset Service is Unavailable Oct 2nd 2025 1:56pm - Eptura Asset Fire Alarm Activated Oct 2nd 2025 1:57pm – Engineering reports services back up Oct 2nd 2025 2:38pm – Customers contacted to confirm Eptura Asset service working again **Total duration of event:** 43 minutes **Root cause:** On Monday, while preparing our servers for a routine update, we executed standard commands to test and reload the configuration. This is a regular procedure that we follow, and it has not caused issues in the past. However, during this reload, active user sessions unexpectedly dropped. To quickly restore system functionality, we recycled all application pools across all servers. **Remediation:** We are currently continuing investigation into the exact reason for the session drop during the standard Nginx reload, as this behaviour has not been observed before. Our team is working diligently to understand and prevent the issue from occurring in the future. **Preventative action:** To prevent this from happening again, our Engineering team will be making the necessary changes this weekend during off-peak hours to minimize any potential disruption.

Read the full incident report →

Started Sep 23, 2025, 12:26 PM UTC· resolved Sep 23, 2025, 08:52 PM UTC

1. investigating Sep 23, 2025, 12:26 PM UTC

   We have received reports of issue wherein Secure Login button is grayed out. We are currently investigating the issue.

2. monitoring Sep 23, 2025, 02:00 PM UTC

   A fix has been implemented, and we shall continue to monitor the issue.

3. resolved Sep 23, 2025, 08:52 PM UTC

   We are pleased to inform you that the issue with HippoSecure Login has been resolved. Our Engineering team has completed the necessary actions and verified that the service is now functioning normally. A Root Cause Analysis (RCA) will be conducted to understand the incident in detail and will be made available on our Status Page within 10 days. Thank you for your patience and cooperation throughout this process. If you have any further questions or concerns, please feel free to reach out.

4. postmortem Sep 23, 2025, 10:54 PM UTC

   **Eptura Hippo detailed Root Cause Analysis | Sept 23 2025** We are grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident. **Description:** Hippo Secure Login Greyed Out **Type of Event:** Functionality issue **Services/Modules impacted:** Add services/modules impacted: Secure Login **Timeline:** Sept 23rd 2025 7:12am – Customers began reporting being unable to login Sept 23rd 2025 7:12am – JIRA created/Status Page Updated Sept 23rd 2025 7:14am – Engineering restarted restarted the hippo-prd-cmmsService and hippo-prd-cmmsService-ca services to resolve the issue Sept 23rd 2025 7:41am – Engineering conducting further investigation to identify the core issue. Sept 23rd 2025 9:46am – Issue resolved **Total Duration of Event:** 2\.25hrs **Root Cause:** An ongoing memory leak in the service that builds up over time leading to performance issues. A regularly scheduled service restart did not occur on schedule. **Remediation:** To resolve the issue we restarted the hippo-prd-cmmsService and hippo-prd-cmmsService-ca services to resolve the issue. **Preventative Action:** We’re committed to preventing similar issues in the future and will continue investigation to better understand the root cause. This will help us implement better monitoring and more robust procedures to ensure reliability and stability of our services.

Read the full incident report →

Started Sep 17, 2025, 01:50 PM UTC· resolved Sep 17, 2025, 06:50 PM UTC

1. investigating Sep 17, 2025, 01:50 PM UTC

   We are currently investigating an issue with sites that are experiencing errors while navigating through the site.. Our Engineering team is currently investigating to determine the cause of the disruption. Next update will be posted at 11 AM CST

2. identified Sep 17, 2025, 02:00 PM UTC

   We are moving into an identified phase. Our teams have identified that the issue is related to a release deployed last night. To mitigate the errors and restore stability, we are actively rolling back the release. We are now moving into the identified phase of our incident response and are working with urgency to resolve the issue. We understand how critical this is to your operations and appreciate your patience as we work through it. Further updates will be shared as progress continues.

3. monitoring Sep 17, 2025, 02:47 PM UTC

   We’ve completed the rollback of the release deployed last night, and all production servers have been successfully restored. At this time, your operations should be back to normal. We truly appreciate your patience and understanding during this disruption. Our teams will continue to monitor the environment closely over the next four hours to ensure full stability and performance. Thank you for your continued trust.

4. resolved Sep 17, 2025, 06:50 PM UTC

   We have moved to the Resolved Phase. As we have not seen further service disruptions after the rollback. Thank you for your patience and understanding while we worked to address the issue. We sincerely apologize for any inconvenience caused.

5. postmortem Sep 29, 2025, 10:19 PM UTC

   **Eptura Workplace detailed Root Cause Analysis | 9/17/2025** **S2 - End users are experiencing an error while navigating through Eptura Workplace** We are grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident. **Description:** Multiple customers experienced a temporary disruption in Eptura Workplace due to a release that was deployed to production on the evening of 09/16/2025 at approximately 10 PM CT. **Type of Event:** Intermittent Outage **Services/Modules impacted:** Eptura Workplace Production Environment **Timeline in CT time zone:** ` ` 2:46 AM – An initial support case was reported regarding difficulty accessing the platform. The customer was able to log into their account but could not access the reservations module. Attempts to refresh the page resulted in continuous loading without resolution. 6:30 AM – Two additional support cases were submitted, this time indicating complete site inaccessibility. Support escalated the issue by creating a Severity 2 JIRA ticket with Cloud Operations to begin investigation \(CM-98483\). 8:30 AM – Internal reports surfaced indicating that the AUP was down and the internal Server for the US region was experiencing issues. A new JIRA \(CM-98491\) was created, and Cloud Ops confirmed the issue was related to CM-98483. Internal discussions pointed to a recent release for Eptura Workplace deployed on the evening of 9/16 as the likely cause. A rollback was considered as a potential resolution. 8:50 AM – A fire alarm was triggered, and all customers were notified of the disruption via the Eptura Workplace Status Page. The message indicated that the issue was under active investigation. 9:00 AM – Customers were updated that the root cause had been identified. To resolve the issue, a rollback of the 9/16 release was required. 9:47 AM – The rollback was successfully completed. The system entered a monitoring phase, during which QA performed a sanity check. The check was completed successfully, confirming system stability. 1:50 PM – The incident was officially moved to the Resolved Phase. No further service disruptions were observed following the rollback. **Total Duration of Event:** 11 hrs 4 mins **Root Cause:** During a recent release, we introduced a code change that required specific data fields from customer feeds. These fields were present in our internal test environments, and as a result, the new code passed all internal testing successfully. However, upon deployment to production, we discovered that some customer feeds did not include these required fields. This discrepancy led to a data missing exception in the new code. Unfortunately, this exception triggered a pre-existing issue in the system, where database connections were not being properly released. As more exceptions occurred, these unreleased connections accumulated over time, eventually exhausting the available database connections and causing a temporary system outage. **Remediation:** To address the issue, we took the following steps: 1. **Resolved the underlying bug**: We fixed the bug that was causing database connections to remain open without being properly released. 2. **Deployed a patch**: We implemented a patch to ensure that the new code handles missing data fields more gracefully, preventing similar exceptions from cascading into system-level failures. 3. **Restarted affected services**: We restarted the affected services and cleared stale connections to restore system stability. These immediate actions helped mitigate the impact and restore normal operations. **Preventative Action:** To prevent this issue from recurring, we are implementing several key improvements: 1. **Enhanced validation processes**: We are improving our validation processes to ensure that all required data fields are present and accounted for before any new code is deployed. 2. **Expanded test coverage**: We are broadening our test coverage to better simulate production environments, including the variability in customer data feeds. 3. **Monitoring and alerting**: We are introducing new monitoring and alerting mechanisms to detect abnormal database connection usage early, allowing us to take proactive measures. 4. **Safeguards for connection leaks**: We are adding additional safeguards to detect and recover from connection leaks more effectively, ensuring greater resilience in future releases. These improvements are designed to enhance the reliability and stability of our system, and we are committed to delivering a seamless experience for all our customers.

Read the full incident report →

Started Sep 09, 2025, 08:00 PM UTC· resolved Sep 09, 2025, 08:00 PM UTC

1. resolved Sep 09, 2025, 10:09 PM UTC

   Between approximately 2:55 PM CT and 4:08 PM CT, Eptura Workplace customers using the V3 calendar service in the US region may have experienced latency with Exchange calendar sync and webhook delivery due to a third-party vendor issue. Services have since returned to normal, and performance has stabilized. Our teams continue to monitor to ensure full reliability moving forward. We appreciate your patience and understanding.

Read the full incident report →

Started Sep 09, 2025, 09:48 AM UTC· resolved Sep 09, 2025, 12:49 PM UTC

1. investigating Sep 09, 2025, 09:48 AM UTC

   We have received report of users not being able to login to the site. The issue impacts the sites hosted in Europe region. Our Infrastructure team is investigating the issue.

2. monitoring Sep 09, 2025, 11:10 AM UTC

   A fix has been implemented. We are continuing to monitor the situation.

3. resolved Sep 09, 2025, 12:49 PM UTC

   As we have not experienced further service disruptions since the fix was implemented, we have transitioned to the Resolved Phase. An RCA will be posted for this incident in 10 business days.

4. postmortem Sep 15, 2025, 07:14 PM UTC

   **Eptura Workplace Detailed Root Cause Analysis | 09/15/2025** **S2 - Sites hosted on the France Server are unable to access the site** ‌ We are grateful for your continued support and loyalty. We understand how disruptions like this can affect your operations, and we appreciate your patience and feedback as we worked to resolve the incident. ‌ **Description:** On September 9th, some clients encountered an error when attempting to log into their Eptura Workplace instance. ‌ **Type of Event:** Outage ‌ **Service/Modules Impacted:** Production environments ‌ **Timeline: \(Times are in CST\)** 3:29 AM Support received notification from the client not able to access the sites 3:30 AM Support began testing 3:40 AM Support created Jira CM-9814 3:46 AM Cloud Ops acknowledged and began investigating 4:07 AM Support pulled the Fire Alarm 4:48 AM Status page was updated 6:05 AM Support was informed that the site was back up 6:10 AM Status page was updated and moved to monitoring. 6:35 AM Support received confirmation from clients. 7:49 AM Status page was closed as resolved ‌ **Total Duration of Event:** 2 hours 36 minutes ‌ **Root Cause Analysis:** The issue occurred due to an update rolled out to enhance security. During this change, the security settings of our France SQL servers for minimum TLS \(Transport Layer Security\) was updated from version 1.0 to 1.2. This update caused a communication issue between our services and the servers, which were still trying to connect using the older TLS 1.0, resulting in the site temporarily going down. ‌ **Remediation:** To address the issue, we took these steps: * We immediately created a ticket with Microsoft Azure to roll back the change and help restore the sites temporarily. * We are implementing a configuration update in our system to support the new TLS 1.2 version. This update is currently being tested to ensure it functions properly. ‌ **Preventative Action:** To prevent this from occurring in the future, our Cloud Ops team is taking active steps to: 1. Ensure all significant changes to production are thoroughly reviewed at multiple points 2. Schedule these types of changes to weekends or other off-peak times to minimize any potential impact 1\. Involve management for approval on all such changes on Production for approval 2\. Move these changes to weekends to reduce impacts Thank you for your patience and understanding during this disruption. We assure you that we are dedicated to continually improving our services to better your experience.

Read the full incident report →

Started Sep 03, 2025, 04:46 PM UTC· resolved Sep 04, 2025, 03:55 PM UTC

1. investigating Sep 03, 2025, 04:46 PM UTC

   We’re currently investigating two issues that may impact your experience: - Employee Imports via SFTP are pending and not processing as expected. - Check-In Notifications are not being sent to end users. Our engineering team is actively investigating both issues and working to resolve them as quickly as possible. In the meantime, please note that end users can still successfully check in using either the desktop or mobile application. We sincerely apologize for the inconvenience and appreciate your patience as we work to restore full functionality. Our next update will be provided 3:45 PM CT.

2. identified Sep 03, 2025, 05:52 PM UTC

   We’ve identified the root cause of the current service disruption, and it appears both the Employee Imports via SFTP and Check-In Notification issues are related. Our engineering team is now actively working within the identified phase to implement a resolution. In the meantime, end users can continue to check in using the desktop or mobile application. We’ll provide our next update by 4:51 PM CT. Thank you for your continued patience and understanding.

3. monitoring Sep 03, 2025, 10:00 PM UTC

   We’ve implemented a solution for the issue affecting SpaceIQ Disruptions and have now moved into the monitoring phase. Booking notifications are coming through as expected, and all pending employee imports will be processed first thing on September 4, 2025. Our Support team will begin reaching out to confirm resolution and ensure everything is running smoothly. Engineering is closely monitoring system performance to validate stability and confirm the issue has been fully resolved. Thank you for your continued patience and understanding. Our next update will be provided by 8 PM CT.

4. monitoring Sep 04, 2025, 02:54 AM UTC

   A fix has been deployed to address the SpaceIQ disruptions. Booking notifications are now delivering as expected and pending imports will be processed shortly. Our Support team will reach out to confirm resolution with customers while Engineering continues to validate overall stability.

5. resolved Sep 04, 2025, 03:55 PM UTC

   As we have not experienced further service disruptions since the fix was implemented, we have transitioned to the Resolved Phase. An RCA will be posted for this incident in 10 business days.

6. postmortem Sep 12, 2025, 07:17 PM UTC

   **SpaceIQ by Eptura detailed Root Cause Analysis | 9/12/2025** **S2 | SpaceIQ Disruptions** We are grateful for your continued support and loyalty. We understand how disruptions like this can affect your operations, and we sincerely appreciate your patience and feedback as we worked to resolve the incident. **Description:** Two issues impacted SpaceIQ customers: · Employee Imports via SFTP were pending and not processing. · Check-In Notifications were not being sent to end users for reservations. **Type of Event:** Functionality Issue **Services/Modules impacted:** Production for Imports and Reservations **Timeline:** Timeline of Events \(MDT\) * Friday, 08/29/2025 – 5:45 AM Initial report of missing check-in email notifications. Escalated to engineering. No other customers were affected at this time. * Sunday, 08/31/2025 – 3:37 PM First report of pending employee imports. Also escalated to engineering. * Tuesday, 09/02/2025 Additional customers reported both issues. Engineering planned to consult with the Clops team during Bug Triage. * Wednesday, 09/03/2025 * 10:46 AM – Customers notified of service disruption affecting check-in notifications and employee imports. * 11:52 AM – Root cause identified. * 4:00 PM – Fix implemented; monitoring began. * Thursday, 09/04/2025 – 9:55 AM Customers confirmed resolution. Status page updated to reflect resolved state. **Total Duration of Event:** 2 days, 1 hour, 25 minutes **Root Cause:** The disruption was caused by an overloaded internal server, which temporarily blocked the system from sending check-in email notifications and processing employee imports via SFTP. **Remediation:** Our engineering team identified the issue and worked closely with Cloud Operations team to restart the internal server. This action immediately restored functionality. **Preventative Action:** To prevent recurrence, our Cloud Operations team will be upgrading our Redis system to a newer version. This upgrade will provide enhanced performance and stability, even under high load conditions.

Read the full incident report →

Started Sep 01, 2025, 05:00 AM UTC· resolved Sep 02, 2025, 10:55 AM UTC

1. investigating Sep 01, 2025, 05:00 AM UTC

   Our infrastructure team is actively investigating an intermittent issue where some users are being redirected to a logout page when attempting to access the application through the master.aspx URL. As a temporary Workaround: To continue operations without disruption, users are advised to log in using the application base URL https://xyz.condecosoftware.com/ (replace XYZ with your company name) in incognito/private mode. We recognize the inconvenience this may cause and want to assure you that resolving this issue is a top priority. Our teams are working diligently to identify and implement a fix. We will continue to provide updates as soon as more information becomes available. Thank you for your understanding and patience.

2. identified Sep 01, 2025, 10:47 AM UTC

   We've identified the issue causing the current outage and will be applying a configuration change to resolve it. This will result in approximately 10 minutes of downtime, starting at 11:30 AM UTC / 5:00 PM IST. We’ll post an update once the work is completed. Thank you for your understanding.

3. identified Sep 01, 2025, 02:11 PM UTC

   We are continuing to work on the fix Globally and it is targeted to be completed by 18:30 UTC/Midnight IST.

4. monitoring Sep 01, 2025, 04:19 PM UTC

   The fix has been implemented on the system. We shall keep this under monitoring till 0600 UTC on 2nd September 2025. Detailed root cause analysis shall be shared in 10 business days.

5. resolved Sep 02, 2025, 10:55 AM UTC

   This incident is now being marked as Resolved. Detailed root cause analysis will be shared on this page in 10 days.

6. postmortem Sep 09, 2025, 01:21 PM UTC

   We are grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident. **Type of Event:** _S2 - Eptura Engage - Users are unable to log in to Engage when accessing the web application via_ [_xyz.condecosoftware.com/master.aspx_](http://xyz.condecosoftware.com/master.aspx) _URL, as the page redirects them to a logout screen_ **Services/Modules Impacted:** Eptura Engage Web Application **Issue Summary/Background:** Eptura Engage Web Application — Login attempts made using bookmarked or redirected URLs \(e.g., `xyz.condecosoftware.com/master.aspx`\). Logins using the root URL \(`xyz.condecosoftware.com`\) were not impacted. **Root Cause:** During the Evergreen update applied over the weekend of **30–31 August 2025**, a security configuration change was introduced to strengthen login controls. This update restricted logins to only the root URL. As a result, users attempting to log in via bookmarked or redirected URLs were redirected to a logout screen instead of reaching the login page. **Remediation:** Engineering developed and validated a configuration fix that restored login support for non-root URLs. The fix was rolled out globally on **1 September 2025**, after successful testing in a controlled environment. Customers confirmed that login functionality was restored once the rollout was complete. ‌ **Timeline \(Timezone\):** **31 August 2025** 23:14 UTC – First customer reported login issues when accessing Engage via a bookmarked URL. 23:58 UTC – Support escalated the issue to CloudOps for investigation. **01 September 2025** 02:32 UTC – Ten customer cases confirmed with the same login failure symptoms. 04:48 UTC – Severity officially updated to S2 due to increasing customer impact. 05:00 UTC – Status Page created with initial outage details and guidance to access via the root URL. 06:28 UTC – Engineering team engaged to assist in root cause analysis. 06:45 UTC – Engineering identified a configuration change as the cause and provided a fix. The fix was tested successfully in a customer UAT environment. 09:52 UTC – Engineering confirmed the configuration fix could be safely deployed in bulk across environments. 11:30 UTC – CloudOps initiated the global deployment of the fix. 16:10 UTC – Global deployment completed; login functionality restored for bookmarked and redirected URLs. **02 September 2025** 10:55 UTC – Status Page updated to “Resolved” following confirmation that normal login access had been fully restored. **Total Duration of Event:** 16 hours 56 minutes **Preventive Actions:** To prevent recurrence, Eptura is implementing the following improvements: * **Enhanced release communication** – All changes with potential customer impact will be documented and highlighted in release notes. * **Extended testing coverage** – Login scenarios for both root and non-root/bookmarked URLs will be included in test suites. * **Change review enhancements** – Product, Engineering, and Support will jointly review security and configuration changes to assess customer impact before release. * **Updated customer guidance** – Documentation will be updated to clarify recommended login methods and best practices for bookmarking.

Read the full incident report →