Eptura Asset - S2 - Server Connection Issue

Update timeline

1. investigating Mar 13, 2026, 08:48 PM UTC

   We are currently investigating reports of intermittent issues affecting some Eptura Asset customers who are receiving a 503 error message. Our engineering team is actively working to identify the root cause. We will continue to closely monitor the situation and provide further updates as they become available

2. monitoring Mar 13, 2026, 10:07 PM UTC

   Eptura Asset experienced a brief service degradation that caused some customers to encounter system slowness or receive intermittent 503 Connection Errors. The issue lasted for under 10 minutes and affected a subset of customers attempting to access Eptura Asset services. Our engineering team identified the cause immediately, and recent platform updates allowed for a rapid recovery. Normal performance was fully restored shortly after detection. We continue to monitor the system closely and are reviewing the incident to ensure resilience against similar issues in the future. We apologize for any inconvenience this brief disruption may have caused and appreciate your patience.

3. resolved Mar 14, 2026, 02:39 PM UTC

   We’re pleased to confirm that the issue has now been resolved. Our Engineering team has implemented a permanent fix, and all services are functioning as expected. We sincerely appreciate your patience and understanding while we worked to restore normal operations. A detailed Root Cause Analysis (RCA) is currently being prepared and will be published within the next 10 business days. Thank you for your continued trust and partnership.

4. postmortem Mar 18, 2026, 10:49 PM UTC

   We are grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident. **Description:** Eptura Asset 503 Server Connection Error **Type of Event:** Outage **Services/Modules impacted:** Asset Inventory Module **Timeline:** March 13th 20:48 UTC Customers reporting that they were experiencing 503 Server Connection Error March 13th 22:07 UTC Eptura team identified the cause, and recent platform updates allowed for a rapid recovery. March 14th 14:39 UTC Confirmation with customers that Asset Modules are loading as expected **Total Duration of Event:** 18 hours **Root Cause:** Our reporting service \(Exago\) was experiencing slow or unresponsive behavior. Because our application relied on this service for certain requests, these delays caused system‑wide slowdowns and, in some cases, 503 errors. When Exago became overloaded, it affected the performance of unrelated areas like Work Orders, Assets, and other core features. **Remediation:** We implemented several backend improvements to ensure reporting issues stay isolated and no longer impact the rest of the application. * **Asynchronous request handling** Reporting requests now run asynchronously, allowing the system to stay responsive even if individual reporting calls are slow. * **Circuit breaker pattern** If the reporting service becomes unhealthy, the system automatically detects this and temporarily stops sending new requests. This prevents slowdowns from spreading across the application. * **Session deduplication** Multiple tabs or browser refreshes no longer create duplicate reporting sessions. All repeated user actions are consolidated into a single backend request. * **Session caching** Newly created reporting sessions are briefly cached \(30 seconds\), reducing unnecessary repeat calls during login bursts or fast navigation. * **Strict timeout controls** All reporting requests follow a consistent end‑to‑end timeout \(up to 270 seconds\). This ensures system resources are promptly released even when Exago is slow or unresponsive. * **Improved telemetry and monitoring** New Prometheus metrics provide better visibility into thread usage, allowing the team to detect early warning signs before they impact customers. **Preventative Action:** Rate Limiting has been enabled on all the PROD instances. If an abnormal volume of Exago session requests originates \(e.g., login storms, retry floods\), the system will gracefully reject overflow requests while continuing to serve other users normally. These updates ensure that reporting issues no longer affect critical areas of the application. If Exago becomes slow, the impact is now contained, other features are expected to continue working normally. New Behavior Summary · Backend timeout: 270 seconds max per attempt · Frontend retries: 8 attempts over 3 minutes max · Session caching: 30 seconds \(reduces duplicate requests from multiple user tabs\) · Circuit breaker: Opens after 20% failure rate, pauses attempts for 30 seconds, Users get fast failure responses during this window rather than long waits