eazyBI Outage History

A change in the import structure introduced an unexpected error affecting project role dimension imports. The issue was identified quickly, and a fix has been deployed to prevent recurrence.

Changes in the structure of imported data caused unplanned DB performance issues. As a result, imports took longer than expected and larger queues accumulated. We are planning to optimize DB resources and performance.

We experienced a short outage caused by a hardware failure in Google Cloud infrastructure beyond our control. The incident has been resolved quickly, and we have implemented architectural changes.

eazyBI sites without an Atlassian subscription that use a development license extension had lost their license status, resulting in import errors across several eazyBI accounts. We have restored the license status validation for development extensions and verified successful import completion across affected sites.

The problem occurred due to a change in the REST API on the Atlassian side. Was fixed by Atlassian.

Atlassian incident impacted eazyBI app.

Service was unavailable due to a large number of unterminated SQL transactions, which exhausted available database connections. The issue was detected by monitoring systems; however, as it occurred outside working hours, response and remediation were delayed. To prevent recurrence, we are updating timeout configurations to reduce the risk of lingering transactions. Monitoring has been improved with severity-based alerting to ensure timely notification and response.

A third-party REST API problem

The incident was caused by an incomplete solution for Xray data import changes. Import via the Xray GraphQL API was added alongside backup processing, but unlike backups, the GraphQL API enforces project-level permissions — so instance admins who could access the full Xray backup were denied data for projects where they lacked explicit access.

Impact on the system from recently deployed changes.

Atlassian incident: [https://developer.status.atlassian.com/incidents/mc0d4jnrz122](https://developer.status.atlassian.com/incidents/mc0d4jnrz122)

SSL certificate expiration follow-up was dysfunctional.

Unexpected side effects of a newly deployed feature.

Jira REST API returned incorrect data. Fixed by Atlassian.

Changes in the Atlassian REST API made it impossible to detect whether the site has Assets and other apps.

An unexpected side effect of deployed changes.

Synchronization with Clickhouse DB did not work after Clickhouse upgrade.

Problem with eazyBI infrastructure that stuck the eazyBI import queue

The PDF generation engine failed after an automatic upgrade. Version control for the engine was implemented to use a stable version

A third party REST API problem

A problem with JSM Assets REST API.

A new bug from the last deployment

A bug in a new deployment

The problem occurred because auto-scale options for eazyBI servers were not correctly enabled during the last deployment. This resulted in an overload of our app servers.