Eptura incident

S2 - SpaceIQ Disruptions

Minor Resolved View vendor source →

Eptura experienced a minor incident on September 3, 2025 affecting SpaceIQ, lasting 23h 8m. The incident has been resolved; the full update timeline is below.

Started
Sep 03, 2025, 04:46 PM UTC
Resolved
Sep 04, 2025, 03:55 PM UTC
Duration
23h 8m
Detected by Pingoru
Sep 03, 2025, 04:46 PM UTC

Affected components

SpaceIQ

Update timeline

  1. investigating Sep 03, 2025, 04:46 PM UTC

    We’re currently investigating two issues that may impact your experience: - Employee Imports via SFTP are pending and not processing as expected. - Check-In Notifications are not being sent to end users. Our engineering team is actively investigating both issues and working to resolve them as quickly as possible. In the meantime, please note that end users can still successfully check in using either the desktop or mobile application. We sincerely apologize for the inconvenience and appreciate your patience as we work to restore full functionality. Our next update will be provided 3:45 PM CT.

  2. identified Sep 03, 2025, 05:52 PM UTC

    We’ve identified the root cause of the current service disruption, and it appears both the Employee Imports via SFTP and Check-In Notification issues are related. Our engineering team is now actively working within the identified phase to implement a resolution. In the meantime, end users can continue to check in using the desktop or mobile application. We’ll provide our next update by 4:51 PM CT. Thank you for your continued patience and understanding.

  3. monitoring Sep 03, 2025, 10:00 PM UTC

    We’ve implemented a solution for the issue affecting SpaceIQ Disruptions and have now moved into the monitoring phase. Booking notifications are coming through as expected, and all pending employee imports will be processed first thing on September 4, 2025. Our Support team will begin reaching out to confirm resolution and ensure everything is running smoothly. Engineering is closely monitoring system performance to validate stability and confirm the issue has been fully resolved. Thank you for your continued patience and understanding. Our next update will be provided by 8 PM CT.

  4. monitoring Sep 04, 2025, 02:54 AM UTC

    A fix has been deployed to address the SpaceIQ disruptions. Booking notifications are now delivering as expected and pending imports will be processed shortly. Our Support team will reach out to confirm resolution with customers while Engineering continues to validate overall stability.

  5. resolved Sep 04, 2025, 03:55 PM UTC

    As we have not experienced further service disruptions since the fix was implemented, we have transitioned to the Resolved Phase. An RCA will be posted for this incident in 10 business days.

  6. postmortem Sep 12, 2025, 07:17 PM UTC

    **SpaceIQ by Eptura detailed Root Cause Analysis | 9/12/2025** **S2 | SpaceIQ Disruptions** We are grateful for your continued support and loyalty. We understand how disruptions like this can affect your operations, and we sincerely appreciate your patience and feedback as we worked to resolve the incident. **Description:** Two issues impacted SpaceIQ customers: · Employee Imports via SFTP were pending and not processing. · Check-In Notifications were not being sent to end users for reservations. **Type of Event:** Functionality Issue **Services/Modules impacted:** Production for Imports and Reservations **Timeline:** Timeline of Events \(MDT\) * Friday, 08/29/2025 – 5:45 AM Initial report of missing check-in email notifications. Escalated to engineering. No other customers were affected at this time. * Sunday, 08/31/2025 – 3:37 PM First report of pending employee imports. Also escalated to engineering. * Tuesday, 09/02/2025 Additional customers reported both issues. Engineering planned to consult with the Clops team during Bug Triage. * Wednesday, 09/03/2025 * 10:46 AM – Customers notified of service disruption affecting check-in notifications and employee imports. * 11:52 AM – Root cause identified. * 4:00 PM – Fix implemented; monitoring began. * Thursday, 09/04/2025 – 9:55 AM Customers confirmed resolution. Status page updated to reflect resolved state. **Total Duration of Event:** 2 days, 1 hour, 25 minutes **Root Cause:** The disruption was caused by an overloaded internal server, which temporarily blocked the system from sending check-in email notifications and processing employee imports via SFTP. **Remediation:** Our engineering team identified the issue and worked closely with Cloud Operations team to restart the internal server. This action immediately restored functionality. **Preventative Action:** To prevent recurrence, our Cloud Operations team will be upgrading our Redis system to a newer version. This upgrade will provide enhanced performance and stability, even under high load conditions.