Atlassian Outage History

Started May 27, 2025, 06:53 AM UTC· resolved May 27, 2025, 07:56 AM UTC

1. investigating May 27, 2025, 06:53 AM UTC

   We are aware of an issue where users are receiving a 403 error when attempting to reach Administration. Our team is looking into this issue with urgency and will provide an update as soon as possible.

2. monitoring May 27, 2025, 07:33 AM UTC

   The issues causing access problems to Administration have been resolved, and services are now functioning normally for all affected customers. We will monitor it closely for the next 30 minutes to ensure stability.

3. resolved May 27, 2025, 07:56 AM UTC

   The issue with the Administration accessibility has now been resolved, and the service is operating normally for all affected customers.

Read the full incident report →

Started May 26, 2025, 05:19 AM UTC· resolved May 26, 2025, 06:27 AM UTC

1. investigating May 26, 2025, 05:19 AM UTC

   We are aware of issues relating to authentication resulting in 503 gateway error messages. Our team is investigating with urgency and will provide an update when available.

2. monitoring May 26, 2025, 05:57 AM UTC

   A fix for the issue causing authentication errors across multiple apps has been rolled out. We will provide further update when can confirm the issue has been fully resolved.

3. resolved May 26, 2025, 06:27 AM UTC

   Monitoring has indicated that users are no longer receiving the error messages caused by this issue and it should now be fully resolved.

Read the full incident report →

Started May 26, 2025, 05:19 AM UTC· resolved May 26, 2025, 06:27 AM UTC

1. investigating May 26, 2025, 05:19 AM UTC

   We are aware of issues relating to authentication resulting in 503 gateway error messages. Our team is investigating with urgency and will provide an update when available.

2. monitoring May 26, 2025, 05:57 AM UTC

   A fix for the issue causing authentication errors across multiple apps has been rolled out. We will provide further update when can confirm the issue has been fully resolved.

3. resolved May 26, 2025, 06:27 AM UTC

   Monitoring has indicated that users are no longer receiving the error messages caused by this issue and it should now be fully resolved.

Read the full incident report →

Started May 26, 2025, 05:19 AM UTC· resolved May 26, 2025, 06:27 AM UTC

1. investigating May 26, 2025, 05:19 AM UTC

   We are aware of issues relating to authentication resulting in 503 gateway error messages. Our team is investigating with urgency and will provide an update when available.

2. monitoring May 26, 2025, 05:57 AM UTC

   A fix for the issue causing authentication errors across multiple apps has been rolled out. We will provide further update when can confirm the issue has been fully resolved.

3. resolved May 26, 2025, 06:27 AM UTC

   Monitoring has indicated that users are no longer receiving the error messages caused by this issue and it should now be fully resolved.

Read the full incident report →

Started May 25, 2025, 11:45 PM UTC· resolved May 26, 2025, 12:56 AM UTC

1. investigating May 25, 2025, 11:45 PM UTC

   We are aware of some users experiencing issues loading Bitbucket and may be receiving errors such as '500 internal server error'. Our team is investigating this issue with urgency and will provide an update as soon as possible.

2. monitoring May 26, 2025, 12:24 AM UTC

   Our team has mitigated the issue that caused error messages when loading Bitbucket. We are continuing to monitor at this time to ensure performance has been restored, and resolve the root cause.

3. resolved May 26, 2025, 12:56 AM UTC

   Our team has mitigated the issue that caused error messages when loading Bitbucket. Bitbucket should now function correctly for all previously impacted users.

Read the full incident report →

Started May 22, 2025, 12:25 AM UTC· resolved May 22, 2025, 01:37 AM UTC

1. identified May 22, 2025, 12:25 AM UTC

   RDS is currently experiencing an issue that has caused an outage in the production environment. AWS is investigating. In the meantime w will perform a database restore.

2. monitoring May 22, 2025, 01:12 AM UTC

   The database has been restored. We are monitoring the site to ensure there are no further connectivity issues. We have confirmed no data was lost during this outage.

3. resolved May 22, 2025, 01:37 AM UTC

   This incident has been resolved.

Read the full incident report →

Started May 20, 2025, 02:36 PM UTC· resolved May 20, 2025, 02:57 PM UTC

1. investigating May 20, 2025, 02:36 PM UTC

   We are currently investigating an issue impacting both the Compass UI and APIs. We are investigating this issue and will share an update once we have additional information.

2. identified May 20, 2025, 02:53 PM UTC

   The issue has been identified and a fix is being implemented. We have put some mitigation mechanisms in place and our systems are starting to recover.

3. resolved May 20, 2025, 03:38 PM UTC

   We have verified that the incident has been resolved.

Read the full incident report →

Started May 20, 2025, 02:14 PM UTC· resolved May 21, 2025, 12:42 AM UTC

1. investigating May 20, 2025, 02:14 PM UTC

   We are investigating cases of degraded performance for all Jira Work Management, Jira Service Management, Jira, and Jira Product Discovery Cloud customers. We will provide more details within the next hour.

2. investigating May 20, 2025, 04:37 PM UTC

   We are continuing to investigate issues with the Jira degradation and will provide updates as we learn more.

3. investigating May 20, 2025, 06:25 PM UTC

   We are continuing to investigate an issue that is disrupting access to core Jira functionality for a limited number of Jira customers. We will provide an update when we have identified a fix.

4. resolved May 21, 2025, 12:42 AM UTC

   Deployment of the fix for this issue has now completed rolling out and this should be resolved for all users.

5. postmortem Jun 02, 2025, 03:29 AM UTC

   ### Summary On May 20, 2025, between 13:10 UTC and 22:12 UTC, a small subset of Atlassian customers using Jira Software and Jira Service Management products were unable to access certain functionality, such as creating, viewing, or editing boards, issues, filters, and Jira Service Management requests. The root cause was data duplication in our software related to upgrades of \(non-faulty\) third party apps. The incident was detected within 4 minutes by Automated Monitoring, and mitigated through the subsequent 8 hours and 58 mins. Factors that prolonged the impact of the incident included: * An attempt to mitigate the issue by disabling the \(non-faulty\) apps was ineffective because the disablement mechanism was not integrated with the affected data source * Attempts to manually rectify the data in the database were hampered by the binary storage format used for the data \(necessary for performance reasons\) ### **IMPACT** During the impact period stated above, Jira Software and Jira Service Desk sites that had a specific version of the third party apps installed were impacted. For the duration of the incident, functionality such as creating, viewing, or editing of boards, issues, filters, and Jira Service Management requests was unavailable on those sites. ### **ROOT CAUSE** During the process of version upgrade for the third party apps with dynamic modules, duplicated data was left behind due to a bug in Atlassian systems triggered during a prior upgrade. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** We know that outages impact your productivity. While we have a number of testing and preventative processes in place, this specific issue wasn’t identified during development because of the specific sequence of events required to trigger the latent duplicate data to impact the product’s functioning. We are prioritizing the following improvement actions to avoid repeating this type of incident: * Detecting and repairing duplicated data at the source * Updating the faulty “write” subsystem to prevent further duplication of data Furthermore, we are taking steps to enable swifter mitigation should such an incident occur in the future. We are: * Enhancing app disablement tooling \(and its documentation\) to make it simple and effective to deploy against this class of issue * Enhancing validation of the app upgrade process * Developing an affordance for viewing, repairing, and modifying specific app data We apologize to customers whose services were impacted during this incident; we are taking immediate steps to improve the platform’s performance and availability. Thanks, Atlassian Customer Support

Read the full incident report →

Started May 20, 2025, 02:14 PM UTC· resolved May 21, 2025, 12:42 AM UTC

1. investigating May 20, 2025, 02:14 PM UTC

   We are investigating cases of degraded performance for all Jira Work Management, Jira Service Management, Jira, and Jira Product Discovery Cloud customers. We will provide more details within the next hour.

2. investigating May 20, 2025, 04:37 PM UTC

   We are continuing to investigate issues with the Jira degradation and will provide updates as we learn more.

3. investigating May 20, 2025, 06:25 PM UTC

   We are continuing to investigate an issue that is disrupting access to core Jira functionality for a limited number of Jira customers. We will provide an update when we have identified a fix.

4. resolved May 21, 2025, 12:42 AM UTC

   Deployment of the fix for this issue has now completed rolling out and this should be resolved for all users.

5. postmortem Jun 02, 2025, 03:29 AM UTC

   ### Summary On May 20, 2025, between 13:10 UTC and 22:12 UTC, a small subset of Atlassian customers using Jira Software and Jira Service Management products were unable to access certain functionality, such as creating, viewing, or editing boards, issues, filters, and Jira Service Management requests. The root cause was data duplication in our software related to upgrades of \(non-faulty\) third party apps. The incident was detected within 4 minutes by Automated Monitoring, and mitigated through the subsequent 8 hours and 58 mins. Factors that prolonged the impact of the incident included: * An attempt to mitigate the issue by disabling the \(non-faulty\) apps was ineffective because the disablement mechanism was not integrated with the affected data source * Attempts to manually rectify the data in the database were hampered by the binary storage format used for the data \(necessary for performance reasons\) ### **IMPACT** During the impact period stated above, Jira Software and Jira Service Desk sites that had a specific version of the third party apps installed were impacted. For the duration of the incident, functionality such as creating, viewing, or editing of boards, issues, filters, and Jira Service Management requests was unavailable on those sites. ### **ROOT CAUSE** During the process of version upgrade for the third party apps with dynamic modules, duplicated data was left behind due to a bug in Atlassian systems triggered during a prior upgrade. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** We know that outages impact your productivity. While we have a number of testing and preventative processes in place, this specific issue wasn’t identified during development because of the specific sequence of events required to trigger the latent duplicate data to impact the product’s functioning. We are prioritizing the following improvement actions to avoid repeating this type of incident: * Detecting and repairing duplicated data at the source * Updating the faulty “write” subsystem to prevent further duplication of data Furthermore, we are taking steps to enable swifter mitigation should such an incident occur in the future. We are: * Enhancing app disablement tooling \(and its documentation\) to make it simple and effective to deploy against this class of issue * Enhancing validation of the app upgrade process * Developing an affordance for viewing, repairing, and modifying specific app data We apologize to customers whose services were impacted during this incident; we are taking immediate steps to improve the platform’s performance and availability. Thanks, Atlassian Customer Support

Read the full incident report →

Started May 20, 2025, 02:14 PM UTC· resolved May 21, 2025, 12:42 AM UTC

1. investigating May 20, 2025, 02:14 PM UTC

   We are investigating cases of degraded performance for all Jira Work Management, Jira Service Management, Jira, and Jira Product Discovery Cloud customers. We will provide more details within the next hour.

2. investigating May 20, 2025, 04:37 PM UTC

   We are continuing to investigate issues with the Jira degradation and will provide updates as we learn more.

3. investigating May 20, 2025, 06:25 PM UTC

   We are continuing to investigate an issue that is disrupting access to core Jira functionality for a limited number of Jira customers. We will provide an update when we have identified a fix.

4. resolved May 21, 2025, 12:42 AM UTC

   Deployment of the fix for this issue has now completed rolling out and this should be resolved for all users.

5. postmortem Jun 02, 2025, 03:29 AM UTC

   ### Summary On May 20, 2025, between 13:10 UTC and 22:12 UTC, a small subset of Atlassian customers using Jira Software and Jira Service Management products were unable to access certain functionality, such as creating, viewing, or editing boards, issues, filters, and Jira Service Management requests. The root cause was data duplication in our software related to upgrades of \(non-faulty\) third party apps. The incident was detected within 4 minutes by Automated Monitoring, and mitigated through the subsequent 8 hours and 58 mins. Factors that prolonged the impact of the incident included: * An attempt to mitigate the issue by disabling the \(non-faulty\) apps was ineffective because the disablement mechanism was not integrated with the affected data source * Attempts to manually rectify the data in the database were hampered by the binary storage format used for the data \(necessary for performance reasons\) ### **IMPACT** During the impact period stated above, Jira Software and Jira Service Desk sites that had a specific version of the third party apps installed were impacted. For the duration of the incident, functionality such as creating, viewing, or editing of boards, issues, filters, and Jira Service Management requests was unavailable on those sites. ### **ROOT CAUSE** During the process of version upgrade for the third party apps with dynamic modules, duplicated data was left behind due to a bug in Atlassian systems triggered during a prior upgrade. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** We know that outages impact your productivity. While we have a number of testing and preventative processes in place, this specific issue wasn’t identified during development because of the specific sequence of events required to trigger the latent duplicate data to impact the product’s functioning. We are prioritizing the following improvement actions to avoid repeating this type of incident: * Detecting and repairing duplicated data at the source * Updating the faulty “write” subsystem to prevent further duplication of data Furthermore, we are taking steps to enable swifter mitigation should such an incident occur in the future. We are: * Enhancing app disablement tooling \(and its documentation\) to make it simple and effective to deploy against this class of issue * Enhancing validation of the app upgrade process * Developing an affordance for viewing, repairing, and modifying specific app data We apologize to customers whose services were impacted during this incident; we are taking immediate steps to improve the platform’s performance and availability. Thanks, Atlassian Customer Support

Read the full incident report →

Started May 15, 2025, 02:27 PM UTC· resolved May 15, 2025, 03:21 PM UTC

1. identified May 15, 2025, 02:27 PM UTC

   We are aware that Trello was temporarily inaccessible for all customers. The system has already been recovered, and we are monitoring the situation.

2. resolved May 15, 2025, 03:21 PM UTC

   A fix has been implemented, and the issue is now resolved.

3. postmortem May 27, 2025, 07:04 PM UTC

   ### **SUMMARY** On May 15, 2025, between 13:55 and 14:18 UTC, Atlassian customers using the Trello product experienced errors or slow loading times when attempting to view their cards and boards. The event was triggered by a database plan cache expiring and high resource usage caused by subsequent database query planning operations. The particular database shard that was impacted held data that was required for every card load. The incident was detected within two minutes by the automated monitoring system and mitigated by increasing resources available to the affected database shard, which put Atlassian systems into a known good state. The total time to resolution was about 23 minutes. ### **IMPACT** The overall impact was between May 15, 2025, 13:55 and May 15, 2025, 14:18 UTC on the Trello product. The incident caused service disruption for all Trello customers. ### **ROOT CAUSE** The issue was caused by a query plan expiring from the database cache, which caused incoming queries to go through a replanning operation. These queries had multiple plans that could satisfy them, and depending on the size of the query, one plan might be significantly more efficient than another. This caused the query planner to perform a great many more replanning operations than usual, which consumed all of the CPU on the server for a brief moment. Once the CPU was consumed, the planning operations themselves began taking too long and therefore required constant replanning in an effort to find more efficient options. This negative feedback loop could not be broken without intervention. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** We know that outages impact your productivity. While we have a number of testing and preventative processes in place, this specific issue wasn’t identified because it would only occur under very distinct conditions, including the amount of load and the order of database queries. We are prioritizing the following improvement actions to avoid repeating this type of incident: * Review our capacity planning thresholds and ensure that all shards have sufficient overhead to handle unexpected load. * Improve query planner performance by: * Implement hinting for known problematic query shapes to circumvent the query planner. * Investigate long-term generalized solutions to prevent query planner thrashing. Furthermore, we are prioritizing the following additional measures to reduce the impact of any future incidents: * Analyze and reduce single points of failure for loading Trello boards and cards. We apologize to customers whose services were impacted during this incident; we are taking immediate steps to improve the platform’s performance and availability. Thanks, Atlassian Customer Support

Read the full incident report →

Started May 15, 2025, 02:22 AM UTC· resolved May 20, 2025, 04:07 AM UTC

1. investigating May 15, 2025, 02:22 AM UTC

   We are aware of an issue where users of GitHub for Jira connector may be missing pull requests and experiencing other data pipeline issues. Our team is looking into this with urgency and an update will be provided when available.

2. identified May 15, 2025, 05:17 AM UTC

   Our team has identified the root cause of this issue and have reverted the change that was causing these data processing errors. All new updates from GitHub should now be flowing correctly. We are actively working to address a potential data gap for previously impacted events, and will provide an update when available.

3. identified May 19, 2025, 01:37 AM UTC

   Processing of previously impacted events is now underway. An update will be provided when these operations have been completed and historical processing has been completed.

4. resolved May 20, 2025, 04:07 AM UTC

   Existing events are continuing to be processed. Any updates to existing entities such as pull requests will also correct and update the data here as required. If there are any concerns about impact to your site with this connector please reach out to support and the team will address your concerns as soon as possible.

Read the full incident report →

Started May 15, 2025, 02:22 AM UTC· resolved May 20, 2025, 04:07 AM UTC

1. investigating May 15, 2025, 02:22 AM UTC

   We are aware of an issue where users of GitHub for Jira connector may be missing pull requests and experiencing other data pipeline issues. Our team is looking into this with urgency and an update will be provided when available.

2. identified May 15, 2025, 05:17 AM UTC

   Our team has identified the root cause of this issue and have reverted the change that was causing these data processing errors. All new updates from GitHub should now be flowing correctly. We are actively working to address a potential data gap for previously impacted events, and will provide an update when available.

3. identified May 19, 2025, 01:37 AM UTC

   Processing of previously impacted events is now underway. An update will be provided when these operations have been completed and historical processing has been completed.

4. resolved May 20, 2025, 04:07 AM UTC

   Existing events are continuing to be processed. Any updates to existing entities such as pull requests will also correct and update the data here as required. If there are any concerns about impact to your site with this connector please reach out to support and the team will address your concerns as soon as possible.

Read the full incident report →

Started May 12, 2025, 10:43 AM UTC· resolved May 15, 2025, 12:32 AM UTC

1. investigating May 12, 2025, 10:43 AM UTC

   We have identified an issue in our Data Lake that could potentially result in the last 6 days of data not being available in Atlassian Analytics dashboard for Identity, Atlas, Confluence entities in all regions. The root cause is identified and are working on mitigating the impacted entities. Next update will be shared in 6 hours.

2. identified May 12, 2025, 02:08 PM UTC

   The cause of the issue has been identified and we are working on restoring missing data. Next update will be in three hours.

3. identified May 12, 2025, 05:44 PM UTC

   Data replays are continuing and we are seeing up-to-date data appearing in some regions. We will continue to monitor progress and update again in 6 hours or at the next major change in status.

4. identified May 13, 2025, 12:22 AM UTC

   Identity data has been completely restored across all regions. Restoration of the remaining impacted data is expected to complete in the next 30 hrs. Next update will be shared in 6 hrs.

5. identified May 13, 2025, 06:13 AM UTC

   OpsGenie data has been completely restored across all regions. Remaining data is expected to complete in the next 24 hrs. Next update will in 24 hrs or upon completion of the next major data restoration.

6. identified May 14, 2025, 06:42 AM UTC

   Confluence data has been completely restored across all regions. Next update when the final data restoration completes.

7. monitoring May 14, 2025, 05:06 PM UTC

   All data has been restored. We are continuing to monitor as we conduct final validations on the data to ensure nothing was missed. Once all validations are completed, the incident will be marked as resolved.

8. resolved May 15, 2025, 12:32 AM UTC

   All data has been restored and validated and the issue has been resolved.

Read the full incident report →

Started May 10, 2025, 07:08 AM UTC· resolved May 10, 2025, 10:51 AM UTC

1. identified May 10, 2025, 07:08 AM UTC

   We identified degraded performance at alert search functionality for some Jira Service Management and Opsgenie Cloud customers due to the infrastructure issue from cloud provider. No impact has been observed at alert critical flows like notification. The team has taken actions to mitigate the issue and minimize the impact to search functionality

2. monitoring May 10, 2025, 09:49 AM UTC

   The problem is mitigated, and we are now monitoring closely.

3. resolved May 10, 2025, 10:51 AM UTC

   All search functionality is operational without any latency. Thank you for your patience.

Read the full incident report →

Started May 08, 2025, 03:41 PM UTC· resolved May 09, 2025, 11:17 AM UTC

1. investigating May 08, 2025, 03:41 PM UTC

   We are currently investigating this issue.

2. identified May 08, 2025, 03:45 PM UTC

   The issue has been identified and a fix is being implemented.

3. monitoring May 08, 2025, 03:57 PM UTC

   A fix has been implemented and we are monitoring the results.

4. monitoring May 08, 2025, 06:48 PM UTC

   We are continuing to monitor for any further issues.

5. monitoring May 08, 2025, 09:11 PM UTC

   We are continuing to monitor for any further issues.

6. monitoring May 08, 2025, 11:25 PM UTC

   We are continuing to monitor for any further issues.

7. monitoring May 09, 2025, 01:21 AM UTC

   We are continuing to monitor for any further issues.

8. monitoring May 09, 2025, 04:33 AM UTC

   We are continuing to monitor for any further issues.

9. monitoring May 09, 2025, 07:20 AM UTC

   We are continuing to monitor for any further issues.

10. resolved May 09, 2025, 11:17 AM UTC

    This incident has been resolved.

11. postmortem May 19, 2025, 04:42 PM UTC

    ### Summary On May 8, 2025, at 3:26pm UTC, Bitbucket Cloud experienced website and API latency due to an overloaded primary database. The event was caused by a backfill job running from an internal Atlassian service, which triggered an excessive call volume of expensive queries and pressure on database resources. As a result, the primary database automatically failed over, and Bitbucket services recovered in 15 minutes. Our real-time monitoring detected the incident immediately, and the high-intensity backfill job was stopped. However, following the failover, a backlog of retries from downstream services continued to impact overall database performance. Customers may have seen intermittent errors or website latency during this time. During this period following the failover, the engineering team implemented several strategies to further shed database load, successfully alleviating pressure on resources and improving performance. On May 9th at 11:19 AM UTC, Bitbucket Cloud systems were fully operational. ### **Impact** The overall impact occurred between May 8th, 2025, at 3:26 PM and May 9th at 11:19 AM UTC on Bitbucket Cloud. The incident resulted in increased latency and intermittent failures across Bitbucket Cloud services, including the website, API, and Bitbucket Pipelines. ### **Root cause** The issue was caused by an internal high-scale backfill job that triggered excessive load on certain API endpoints, which eventually impacted the database through resource-intensive queries and operations. This led to additional load from retries by dependent services, increasing the total recovery time. ### **Remedial action plan and next steps** We know that outages impact your productivity. While we have several testing and preventative processes in place, this specific issue wasn’t identified during our testing, as it was related to a specific high-scale backfill job run by an internal component, which impacted highly resource-intensive database queries. To prevent this type of incident from recurring, we are prioritizing the following improvement actions: * Improve database request routing so that more reads go to read replicas instead of the write-primary database. * Adjust rate limits for internal API endpoints with resource-intensive database operations. * Optimize database queries so that they can run more efficiently. * Tune retry policies from downstream services. We apologize to customers whose services were interrupted by this incident, and we are taking immediate steps to improve the platform’s reliability. Thanks, Atlassian Customer Support

Read the full incident report →

Started May 07, 2025, 08:40 PM UTC· resolved May 07, 2025, 10:24 PM UTC

1. investigating May 07, 2025, 08:40 PM UTC

   We are investigating an issue with pipeline builds not triggering that is impacting some Bitbucket customers. We will provide more details within the next hour.

2. identified May 07, 2025, 09:28 PM UTC

   The issue has been identified and a fix is being implemented.

3. monitoring May 07, 2025, 10:00 PM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved May 07, 2025, 10:24 PM UTC

   This incident has been resolved.

Read the full incident report →

Started May 06, 2025, 01:11 AM UTC· resolved May 06, 2025, 05:55 AM UTC

1. identified May 06, 2025, 01:11 AM UTC

   We are investigating reports of intermittent errors for Atlassian Intelligence Confluence Cloud customers. We have identified the root cause and are preparing a hotfix

2. monitoring May 06, 2025, 04:38 AM UTC

   We have identified the root cause of the intermittent errors for Smart Answers and AI definitions in Confluence Cloud Atlassian Intelligence customers are progressively rollout out a fix to customers, with a fix expected to reach all customers over the next 2 hours. We are monitoring closely to confirm all errors have resolved.

3. resolved May 06, 2025, 05:55 AM UTC

   Between 2025/05/03 21:38 UTC to 2025/05/06 05:26 UTC, we experienced an outage in Smart Answers in search and AI definitions for all Confluence Cloud Atlassian Intelligence customers. Rovo experiences were not impacted. The issue has been resolved and the service is operating normally.

Read the full incident report →

Started May 05, 2025, 03:08 PM UTC· resolved May 05, 2025, 05:37 PM UTC

1. investigating May 05, 2025, 03:08 PM UTC

   We've noticed that Trello is slow or unavailable for some users. This will be present in both the web and mobile apps. Our engineering team is actively investigating this incident and working to bring Trello back up to speed as quickly as possible. We'll keep you posted with further updates on this page.

2. monitoring May 05, 2025, 04:56 PM UTC

   We have identified and mitigated the issue causing Trello to be slow or unavailable for some users. We expect API traffic to return to normal within the next 30 minutes. We are now monitoring closely. We will update within the next 30 minutes.

3. resolved May 05, 2025, 05:37 PM UTC

   On May 5th, 2025 we identified a degradation for Trello. Trello is now back online and no further impact has been observed.

4. postmortem May 16, 2025, 12:09 AM UTC

   ### **SUMMARY** On May 5, 2025, between 2:08 p.m. and 4:29 p.m. UTC, some Atlassian customers using Trello were unable to view their boards or cards. The event was triggered by an unexpected error encountered by our infrastructure management tools, which resulted in an incorrect DNS configuration being deployed to a portion of our database. The incident was detected within four minutes by automated monitoring systems and mitigated by identifying the faulty portion of the database and performing a failover, which put Atlassian systems into a known good state. The total time to resolution was about two hours and 21 minutes. ### **IMPACT** The overall impact was on the Trello product on May 5, 2025, between 2:08 p.m. and 4:29 p.m. UTC. The incident caused service disruption to Trello customers whose accounts and boards contained or referenced data on the affected shard of our database. Additionally, some Trello customers would have experienced a service disruption due to our use of load-shedding tools during the incident to strategically block portions of our traffic to aid in recovery. ### **ROOT CAUSE** The day before the incident, on May 4, our infrastructure management tooling encountered an unexpected error when attempting to fetch the networking metadata on a particular host. This led to the host, which was a member of our database cluster, to incorrectly apply the default Operating System DNS configuration. This DNS configuration was not able to resolve internal domains, which led to a partial failure state of the node. The database continued to function normally and there was no immediate customer impact but in the background this incorrect DNS configuration led to the slow buildup of database sessions. These database sessions are usually short-lived and automatically expire when no longer needed, but the DNS misconfiguration prevented this automatic expiration. The database sessions eventually grew to the default maximum on this particular shard. At that point, the shard was unable to generate new sessions, which are required for all basic operations, and the Trello product began experiencing elevated error rates. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** We know that outages impact your productivity. While we have a number of testing and preventative processes in place, this specific issue wasn’t identified due to the isolated nature of the database session resource and monitoring gaps around this resource and around DNS resolution. We are prioritizing the following improvement actions designed to avoid repeating this type of incident: * Update our infrastructure management tool to use a safe fall-back DNS configuration in the case of unexpected errors. * Expand existing DNS monitoring to include the resolution of internal domains. * Expand existing database session count monitoring to include all database node types. Furthermore, we are prioritizing the following additional measures to reduce the duration of any future incidents: * Evaluate our incident response process to identify actions that can be streamlined for quicker resolution. We apologize to customers whose services were impacted during this incident; we are taking steps designed to improve the platform’s performance and availability. Thanks, Atlassian Customer Support

Read the full incident report →

Started Apr 28, 2025, 07:03 PM UTC· resolved Apr 29, 2025, 12:15 AM UTC

1. investigating Apr 28, 2025, 07:03 PM UTC

   We're seeing a recurrence of the earlier issue around components not showing up in our search indexing. We're working on restoring the service.

2. monitoring Apr 28, 2025, 08:05 PM UTC

   We've implemented a fix for this issue and are seeing systems recover. We will keep monitoring to make sure we don't see a recurrence of this issue.

3. resolved Apr 29, 2025, 12:15 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Apr 28, 2025, 11:59 AM UTC· resolved Apr 28, 2025, 04:12 PM UTC

1. investigating Apr 28, 2025, 11:59 AM UTC

   We are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms. Such as apps lifecycle (installation, removals), developer console usage (including tunnelling), and the creation and deployment of Forge apps. Our team is working diligently to resolve this issue and restore services quickly. We are in the process of identifying the root cause and implementing the necessary fixes. Next updates in 1 hour.

2. identified Apr 28, 2025, 12:55 PM UTC

   We are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms, such as apps lifecycle (installation, removals), developer console usage (including tunneling), and the creation and deployment of Forge apps. We have identified the root cause and are in the process of mitigating the impact. We'll keep you posted with further updates. Next updates in 1 hour.

3. monitoring Apr 28, 2025, 01:42 PM UTC

   The issues causing the unavailability of related Forge apps capabilities on multiple Atlassian platforms have been resolved, and services are now operating normally for all impacted customers. We will continue to closely monitor for stability.

4. resolved Apr 28, 2025, 04:12 PM UTC

   Between 10:55 AM UTC to 13:15 PM UTC on April 28, 2025, we experienced partial unavailability in managing Forge apps; Such as lifecycle developer console usage, and the creation and deployment of Forge apps for Atlassian Partners, Jira Work Management, Jira, Atlassian Developer, Jira Product Discovery, Compass, and Atlassian Migrations. We have deployed a fix to mitigate the issue and have verified that the services have recovered. The conditions that cause the bug have been addressed and we're actively working on a permanent fix. The issue has been resolved and the service is operating normally.

Read the full incident report →

Started Apr 28, 2025, 11:59 AM UTC· resolved Apr 28, 2025, 04:12 PM UTC

1. investigating Apr 28, 2025, 11:59 AM UTC

   We are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms. Such as apps lifecycle (installation, removals), developer console usage (including tunnelling), and the creation and deployment of Forge apps. Our team is working diligently to resolve this issue and restore services quickly. We are in the process of identifying the root cause and implementing the necessary fixes. Next updates in 1 hour.

2. identified Apr 28, 2025, 12:55 PM UTC

   We are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms, such as apps lifecycle (installation, removals), developer console usage (including tunneling), and the creation and deployment of Forge apps. We have identified the root cause and are in the process of mitigating the impact. We'll keep you posted with further updates. Next updates in 1 hour.

3. monitoring Apr 28, 2025, 01:42 PM UTC

   The issues causing the unavailability of related Forge apps capabilities on multiple Atlassian platforms have been resolved, and services are now operating normally for all impacted customers. We will continue to closely monitor for stability.

4. resolved Apr 28, 2025, 04:12 PM UTC

   Between 10:55 AM UTC to 13:15 PM UTC on April 28, 2025, we experienced partial unavailability in managing Forge apps; Such as lifecycle developer console usage, and the creation and deployment of Forge apps for Atlassian Partners, Jira Work Management, Jira, Atlassian Developer, Jira Product Discovery, Compass, and Atlassian Migrations. We have deployed a fix to mitigate the issue and have verified that the services have recovered. The conditions that cause the bug have been addressed and we're actively working on a permanent fix. The issue has been resolved and the service is operating normally.

Read the full incident report →

Started Apr 28, 2025, 11:59 AM UTC· resolved Apr 28, 2025, 04:12 PM UTC

1. investigating Apr 28, 2025, 11:59 AM UTC

   We are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms. Such as apps lifecycle (installation, removals), developer console usage (including tunnelling), and the creation and deployment of Forge apps. Our team is working diligently to resolve this issue and restore services quickly. We are in the process of identifying the root cause and implementing the necessary fixes. Next updates in 1 hour.

2. identified Apr 28, 2025, 12:55 PM UTC

   We are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms, such as apps lifecycle (installation, removals), developer console usage (including tunneling), and the creation and deployment of Forge apps. We have identified the root cause and are in the process of mitigating the impact. We'll keep you posted with further updates. Next updates in 1 hour.

3. monitoring Apr 28, 2025, 01:42 PM UTC

   The issues causing the unavailability of related Forge apps capabilities on multiple Atlassian platforms have been resolved, and services are now operating normally for all impacted customers. We will continue to closely monitor for stability.

4. resolved Apr 28, 2025, 04:12 PM UTC

   Between 10:55 AM UTC to 13:15 PM UTC on April 28, 2025, we experienced partial unavailability in managing Forge apps; Such as lifecycle developer console usage, and the creation and deployment of Forge apps for Atlassian Partners, Jira Work Management, Jira, Atlassian Developer, Jira Product Discovery, Compass, and Atlassian Migrations. We have deployed a fix to mitigate the issue and have verified that the services have recovered. The conditions that cause the bug have been addressed and we're actively working on a permanent fix. The issue has been resolved and the service is operating normally.

Read the full incident report →

Started Apr 28, 2025, 11:59 AM UTC· resolved Apr 28, 2025, 04:12 PM UTC

1. investigating Apr 28, 2025, 11:59 AM UTC

   We are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms. Such as apps lifecycle (installation, removals), developer console usage (including tunnelling), and the creation and deployment of Forge apps. Our team is working diligently to resolve this issue and restore services quickly. We are in the process of identifying the root cause and implementing the necessary fixes. Next updates in 1 hour.

2. identified Apr 28, 2025, 12:55 PM UTC

   We are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms, such as apps lifecycle (installation, removals), developer console usage (including tunneling), and the creation and deployment of Forge apps. We have identified the root cause and are in the process of mitigating the impact. We'll keep you posted with further updates. Next updates in 1 hour.

3. monitoring Apr 28, 2025, 01:42 PM UTC

   The issues causing the unavailability of related Forge apps capabilities on multiple Atlassian platforms have been resolved, and services are now operating normally for all impacted customers. We will continue to closely monitor for stability.

4. resolved Apr 28, 2025, 04:12 PM UTC

   Between 10:55 AM UTC to 13:15 PM UTC on April 28, 2025, we experienced partial unavailability in managing Forge apps; Such as lifecycle developer console usage, and the creation and deployment of Forge apps for Atlassian Partners, Jira Work Management, Jira, Atlassian Developer, Jira Product Discovery, Compass, and Atlassian Migrations. We have deployed a fix to mitigate the issue and have verified that the services have recovered. The conditions that cause the bug have been addressed and we're actively working on a permanent fix. The issue has been resolved and the service is operating normally.

Read the full incident report →

Started Apr 28, 2025, 11:59 AM UTC· resolved Apr 28, 2025, 04:12 PM UTC

1. investigating Apr 28, 2025, 11:59 AM UTC

   We are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms. Such as apps lifecycle (installation, removals), developer console usage (including tunnelling), and the creation and deployment of Forge apps. Our team is working diligently to resolve this issue and restore services quickly. We are in the process of identifying the root cause and implementing the necessary fixes. Next updates in 1 hour.

2. identified Apr 28, 2025, 12:55 PM UTC

   We are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms, such as apps lifecycle (installation, removals), developer console usage (including tunneling), and the creation and deployment of Forge apps. We have identified the root cause and are in the process of mitigating the impact. We'll keep you posted with further updates. Next updates in 1 hour.

3. monitoring Apr 28, 2025, 01:42 PM UTC

   The issues causing the unavailability of related Forge apps capabilities on multiple Atlassian platforms have been resolved, and services are now operating normally for all impacted customers. We will continue to closely monitor for stability.

4. resolved Apr 28, 2025, 04:12 PM UTC

   Between 10:55 AM UTC to 13:15 PM UTC on April 28, 2025, we experienced partial unavailability in managing Forge apps; Such as lifecycle developer console usage, and the creation and deployment of Forge apps for Atlassian Partners, Jira Work Management, Jira, Atlassian Developer, Jira Product Discovery, Compass, and Atlassian Migrations. We have deployed a fix to mitigate the issue and have verified that the services have recovered. The conditions that cause the bug have been addressed and we're actively working on a permanent fix. The issue has been resolved and the service is operating normally.

Read the full incident report →