Falcony Outage History

Started Aug 18, 2026, 06:18 AM UTC· resolved Aug 18, 2026, 06:18 AM UTC

1. resolved Aug 18, 2026, 06:18 AM UTC

   Microsoft had an issue between 2026-08-14 and 2026-08-17 where it refused to process some of the email messages we sent. The affected email messages were dropped. This issue affected customers that were using Microsoft 365 service for receiving email.

Read the full incident report →

Started Oct 28, 2025, 07:50 AM UTC· resolved Oct 28, 2025, 09:16 AM UTC

1. investigating Oct 28, 2025, 07:50 AM UTC

   We are currently investigating an outage on app.falcony.io. We will update once we know more details.

2. identified Oct 28, 2025, 08:06 AM UTC

   We have implemented mitigation measures to ensure app.falcony.io is available again. We will continue investigating the root cause and will work towards a permanent fix.

3. identified Oct 28, 2025, 08:29 AM UTC

   We have identified the scenario that caused the issue. We are working on resolving the situation permanently. Users might experience delays in receiving email notifications from Falcony, but app.falcony.io remains available.

4. monitoring Oct 28, 2025, 09:04 AM UTC

   A solution for the degraded performance has been implemented. Email delivery is restoring. We are monitoring the situation to fully resolve this incident.

5. resolved Oct 28, 2025, 09:16 AM UTC

   The solution resolved this incident. We will conduct a root cause analysis and plan further actions.

Read the full incident report →

Started Oct 20, 2025, 09:49 AM UTC· resolved Oct 20, 2025, 10:04 AM UTC

1. monitoring Oct 20, 2025, 09:49 AM UTC

   Amazon Web Services (AWS) is experiencing a service disruption on one of its regions. This service disruption has caused the following Falcony services to be unavailable: - Customer support platform - Help center - Status page The Falcony application is unaffected. Some Falcony services, such as email sending, might experience delays. Amazon has implemented a fix, and they are seeing positive outcomes. We will keep monitoring the situation.

2. resolved Oct 20, 2025, 10:04 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Sep 30, 2025, 07:26 AM UTC· resolved Sep 30, 2025, 12:27 PM UTC

1. investigating Sep 30, 2025, 07:26 AM UTC

   We are investigating slow response times and request timeouts. We will update the incident as we know more.

2. monitoring Sep 30, 2025, 07:49 AM UTC

   We have identified that the application encountered slow response times and request timeouts between 07:10 - 07:26 UTC. Users may have experienced slow page loads, and partial unavailability during this time. The situation has now resolved, and the application behaviour has normalised. We are working together with our hosting provider Heroku to find the root cause for the degraded performance. We will keep monitoring, and will provide updates once we learn more details.

3. investigating Sep 30, 2025, 10:36 AM UTC

   This issue has reappeared. Users may occasionally encounter slow page loads and partial unavailability. We are working on finding the root cause for the issue, and implementing mitigation measures.

4. monitoring Sep 30, 2025, 11:05 AM UTC

   A solution candidate for the degraded performance has been implemented. The application performance is back to normal again. We are monitoring the situation to fully resolve this incident.

5. resolved Sep 30, 2025, 12:27 PM UTC

   The solution candidate has resolved this issue. We will conduct a root cause analysis, and publish a post-mortem containing the results.

Read the full incident report →

Started Jul 18, 2025, 12:12 PM UTC· resolved Jul 18, 2025, 04:03 PM UTC

1. identified Jul 18, 2025, 12:12 PM UTC

   We've identified an issue that caused some columns to be changed from "NOT NULL" to "NULLABLE" on the BI database. We'll be reverting the schema modification and return the columns to NOT NULL. organization table: - uses_old_email_layout - use_malware_scanner - email_service_provider - intercom_enabled - reporter_name_help_text_translations

2. resolved Jul 18, 2025, 04:03 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jun 10, 2025, 07:01 AM UTC· resolved Jun 11, 2025, 07:49 AM UTC

1. investigating Jun 10, 2025, 07:01 AM UTC

   We are investigating an outage on app.falcony.io. We will update once we know more.

2. investigating Jun 10, 2025, 07:17 AM UTC

   We have identified that our service provider Heroku appears to have an outage. We will keep investigating the issue.

3. identified Jun 10, 2025, 07:54 AM UTC

   We have received confirmation from Heroku that they are facing an outage. Heroku has assured that the incident is being treated with the utmost priority by their team. We will post updates once the situation proceeds.

4. identified Jun 10, 2025, 08:39 AM UTC

   The issue is still ongoing, and is affecting multiple Salesforce services besides Heroku. The latest information of the incident can be obtained from https://status.salesforce.com/generalmessages/10001540.

5. identified Jun 10, 2025, 09:33 AM UTC

   The issue is still ongoing. Salesforce is providing updates every 30 minutes in https://status.salesforce.com/generalmessages/10001540.

6. identified Jun 10, 2025, 10:44 AM UTC

   The service has been available for short periods of time, but the outage still widely remains. We will keep posting updates of any advancements.

7. identified Jun 10, 2025, 11:33 AM UTC

   Salesforce is continuing to work on the incident together with a third-party cloud services provider. Updates on the incident can be found in https://status.salesforce.com/generalmessages/10001540.

8. identified Jun 10, 2025, 12:41 PM UTC

   Salesforce continues to investigate the incident

9. identified Jun 10, 2025, 12:59 PM UTC

   Salesforce has identified network configuration issues within Heroku environment. They will explore this further and remain focused on providing a long-term solution.

10. identified Jun 10, 2025, 01:55 PM UTC

    Salesforce continues to investigate an intermittent service disruption in the Heroku service.

11. identified Jun 10, 2025, 02:52 PM UTC

    Salesforce has restarted a subset of their Heroku instances. This has shown positive results, but the outage still remains for app.falcony.io. Further investigations by Salesforce have highlighted that an automated change on the Heroku environment was the potential trigger of this issue. Follow https://status.salesforce.com/generalmessages/10001540 for recent updates.

12. identified Jun 10, 2025, 03:33 PM UTC

    Salesforce is working to restart the rest of the Heroku instances to fully restore the services. We are also actively restarting Falcony instances, which is showing a positive impact on the availability. We will keep following the situation closely.

13. identified Jun 10, 2025, 03:41 PM UTC

    Falcony seems to be available for the time being. The incident on Salesforce's end still persists, so further outage and side effects are still possible. We will monitor the situation closely.

14. monitoring Jun 10, 2025, 04:54 PM UTC

    The outage has not reappeared in app.falcony.io, and the service has been available since 15:26 UTC. Salesforce has the incident still open, and has investigations ongoing. A permanent fix on Heroku's end is in progress, and it could potentially reintroduce an outage impact. We will thus keep the incident open and will actively monitor the situation.

15. monitoring Jun 11, 2025, 05:17 AM UTC

    The outage has not reappeared since Jun 10th 15:26 UTC. The application is fully operational and we do not expect any further disturbance. We deeply apologise for the inconvenience this has caused. We will report a post-mortem once all the details are clear to us.

16. resolved Jun 11, 2025, 07:49 AM UTC

    This incident has been resolved.

17. postmortem Jun 17, 2025, 09:32 AM UTC

    ## Incident overview **Severity** Critical **Time** June 10, 2025, 06:58 UTC – June 10, 2025, 15:26 UTC **Total duration** 8 hours, 28 minutes **Affected services** [app.falcony.io](http://app.falcony.io) \(all users and services\) **Root cause** Global platform outage at our hosting provider, Heroku, caused by a faulty automated system update that resulted in widespread network connectivity failure. ## Summary On June 10, 2025, the Falcony application \([app.falcony.io](http://app.falcony.io)\) was unavailable to all customers for 8 hours and 28 minutes. The outage was a direct result of a global, platform-wide incident at our hosting partner, Heroku \(a Salesforce company\). The Heroku outage also impacted their core management tools, including the dashboard, command-line interface \(CLI\), and support channels, which severely limited our team's ability to mitigate the issue or access status information initially. Our team monitored the situation through alternative channels provided by Salesforce. Once Heroku restored access to their management tools, we performed a full restart of our application, and normal service was restored. ## Timeline of events All times are in UTC. 06:58 - First reports received indicating [app.falcony.io](http://app.falcony.io) is unavailable. 07:15 - Incident confirmed. Investigation reveals that the application is unreachable at the hosting level. Our team attempts to access the Heroku Dashboard, CLI, Status page and Support page but finds them unavailable. 07:42 - Salesforce representative confirms that they are looking into the incident. 08:02 - Falcony team discovers an alternative Salesforce status page \([status.salesforce.com](http://status.salesforce.com)\) and begins actively monitoring it for updates. 08:00 - 15:00 - Falcony application remains unavailable. Our team is on standby, continuously monitoring the Salesforce status for any sign of service restoration. 15:06 - Access restored. The Falcony team successfully regains access to the Heroku CLI. The team begins a full restart of the Falcony application servers. 15:26 - Full service restoration. The application is available to all customers. 16:00 – 05:00 - Falcony team continues to monitor the Salesforce status for possible additional impact. 05:17 - Falcony regains full access to the remaining affected Heroku services. The incident is declared resolved. ## Root cause The direct root cause of this incident was a global infrastructure failure within the Heroku platform. According to Heroku's preliminary analysis, an automated update to a system package triggered critical network connectivity issues across their platform. This made both [app.falcony.io](http://app.falcony.io) and Heroku's core management tools unreachable. ## Next steps Our team is committed to learning from this incident and improving the resilience of our service. We’ve identified the following action items: ### **Completed** **Backup support channels** – We have established backup channels for contacting Heroku support when their primary help portal is unavailable. ### **In progress** **Monitor Salesforce root cause analysis** – We will be closely following updates on [Salesforce's full root cause analysis](https://help.salesforce.com/s/articleView?id=005036674&type=1) to identify any further preventative measures we can implement. Salesforce is currently implementing their actions according to the _Next Steps_ section of the analysis. ## Closing We sincerely apologise for the significant impact this outage had on you and your business. We thank you for your patience and understanding. Should you have any further questions, please do not hesitate to contact our support team at [[email protected]](mailto:[email protected]). **Falcony team** ## Additional resources * [Salesforce Status incident](https://status.salesforce.com/generalmessages/10001540) * [Heroku Status incident](https://status.heroku.com/incidents/2822) * [Salesforce Preliminary Root Cause Analysis](https://help.salesforce.com/s/articleView?id=005036674&type=1)

Read the full incident report →

Started May 28, 2025, 06:10 AM UTC· resolved May 28, 2025, 12:22 PM UTC

1. identified May 28, 2025, 06:10 AM UTC

   We have identified an issue that prevented BI database from updating. The BI Database ended up empty due to the issue. We have fixed the issue and are currently updating the BI database.

2. monitoring May 28, 2025, 09:32 AM UTC

   BI database data has been updated and we are monitoring the situation. The Bi database data should be up-to-date.

3. resolved May 28, 2025, 12:22 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Apr 17, 2025, 07:52 AM UTC· resolved Apr 17, 2025, 10:59 AM UTC

1. identified Apr 17, 2025, 07:52 AM UTC

   We've identified an issue that is preventing several customers new data from being copied to the Falcony BI database. The issue started on 15.4.2025 and is still ongoing. We're currently fixing the issue.

2. identified Apr 17, 2025, 09:07 AM UTC

   We have now fixed the underlying issue. The BI database update is scheduled and should finish in a few hours. We will post once the data has been updated.

3. resolved Apr 17, 2025, 10:59 AM UTC

   The update has completed and the data is up-to-date. The incident is now fully resolved.

Read the full incident report →

Started Mar 17, 2025, 10:19 AM UTC· resolved Mar 17, 2025, 09:10 PM UTC

1. investigating Mar 17, 2025, 10:19 AM UTC

   Our downstream provider, Mailgun, is experiencing issues that are affecting Falcony. As a result, Falcony is unable to send emails, and some data in the service may appear outdated. For more details, please visit the Mailgun status page: https://status.mailgun.com/

2. identified Mar 17, 2025, 10:20 AM UTC

   Our downstream provider is implementing a fix.

3. monitoring Mar 17, 2025, 01:14 PM UTC

   Our downstream provider has implemented a fix. We are monitoring the results.

4. monitoring Mar 17, 2025, 03:05 PM UTC

   We are no longer experiencing issues on our systems due to the downstream provider. However, the downstream provider has not yet marked the issue as resolved. We will continue to monitor the situation.

5. resolved Mar 17, 2025, 09:10 PM UTC

   The downstream provider has confirmed that the issue has been solved.

6. postmortem Mar 19, 2025, 08:53 AM UTC

   **Incident summary** On Monday, March 17th 2025, our email provider Mailgun experienced [an outage](https://status.mailgun.com/incidents/b60dthm3t8gk) that prevented emails from being sent out. This resulted in a backlog of emails in our task queue. The task queue being full in turn impacted other services that rely on the same system, including PDF generation, integrations, malware scanning, and user sessions. **Impact** * Emails were not sent as expected. Some were significantly delayed, while others failed to send altogether. * The task queue filled up, affecting other system functionalities beyond email delivery. * Users experienced disruptions in services utilising task queue, including document generation and integrations. **Resolution and mitigation** * To minimize further disruptions, we increased the memory allocated to the task queue to ensure continued processing. * Once the email provider resolved the underlying issue, we added additional resources to speed up the processing of the task queue. * On Tuesday and Wednesday, we manually sent out any remaining emails that had not been delivered due to earlier failures. **Next steps** We have identified areas in our system where we can improve resilience in case of similar provider outages in the future. Our key focus areas will be: * Ensuring that undelivered emails remain in the queue and can be retried once the service is restored * Implementing better isolation between the email task queue and other dependent services to prevent cascading failures. * Enhancing monitoring and alerting mechanisms to respond more quickly to issues related to email delivery. We appreciate your patience and understanding during this incident. Our team is committed to making improvements to prevent similar disruptions in the future. If you have any questions or concerns, please reach out to our support team at [[email protected]](mailto:[email protected]). **Falcony team**

Read the full incident report →

Started Oct 28, 2024, 06:48 PM UTC· resolved Oct 28, 2024, 07:41 PM UTC

1. monitoring Oct 29, 2024, 09:27 AM UTC

   Our service provider, Heroku, had an incident between 2024-10-28 18:48 UTC and 2024-10-28 19:41 UTC that caused HTTP redirect loops on audits.io. The application was unreachable during that time. Our system health checks failed to identify the issue.

2. resolved Oct 29, 2024, 09:30 AM UTC

   This incident has been resolved. We will continue to investigate how to improve our health checks to identify similar incidents.

Read the full incident report →