Deviniti experienced a critical incident on August 19, 2024 affecting Extension for JSM, lasting 3d 19h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 19, 2024, 01:14 PM UTC
We have detected there has been an active incident affecting the Extension for JSM app. We will continue to investigate the cause of this incident and provide another update within 2 hours.
- investigating Aug 19, 2024, 03:01 PM UTC
We’re continuing to investigate the incident. We will post updates within the next hours once we have more information to share.
- monitoring Aug 20, 2024, 06:06 AM UTC
The fix has been implemented and we will continue to monitor the issue to ensure it has been fully resolved.
- investigating Aug 20, 2024, 01:26 PM UTC
We have detected another incident. We are continuing to investigate the issue.
- monitoring Aug 21, 2024, 06:43 AM UTC
We have successfully restored the functionality of the application. and we will continue to monitor the issue to ensure it has been fully resolved. Please note that, due to our ongoing efforts to eliminate the root cause, you may experience some performance issues when loading panels from our application.
- investigating Aug 21, 2024, 02:13 PM UTC
We have noticed that the issue has reoccurred, and we are currently investigating it. We apologize for any inconvenience, and we will keep you updated on our progress.
- investigating Aug 21, 2024, 05:53 PM UTC
We are continuing to investigate this issue.
- monitoring Aug 21, 2024, 06:03 PM UTC
The solution has been implemented and we continue to monitor the performance. The incident was caused by an incorrect configuration of database connection limits.
- monitoring Aug 23, 2024, 08:48 AM UTC
We are continuing to monitor for any further issues.
- resolved Aug 23, 2024, 08:49 AM UTC
The issue has been resolved and the app is operating normally. We apologize for any inconvenience caused. Next week, we will add a post-mortem analysis describing the causes and the solution.
- postmortem Aug 27, 2024, 09:33 AM UTC
The primary cause of the application's performance issues was an insufficient database connection limit. This, combined with increased traffic on our server during peak hours and prolonged communication with Jira's REST API, resulted in issues with one of the application's nodes, which stopped processing client requests. The problem was temporary and affected a smaller portion of the application's traffic. As preventive measures, the database connection limit has been increased, and a detailed analysis of the longest query durations has been conducted. These queries will be optimized in the near future.