Boltrics Outage History

Started Oct 21, 2026, 11:50 AM UTC· resolved Oct 21, 2026, 10:54 PM UTC

1. identified Oct 21, 2026, 11:50 AM UTC

   Several hosting providers in the Netherlands have taken websites offline as a precaution because of recently disclosed PHP vulnerabilities. We have received questions about this from customers. We have reviewed our services. The following are **not affected** by these vulnerabilities: - Business Central (3PL Dynamics) - Web portal - App platform **Required action:** none. You can continue to use all Boltrics services as normal. We will keep monitoring the situation, but we do not expect any change to this assessment.

2. resolved Oct 21, 2026, 10:54 PM UTC

   The Boltrics website ([www.boltrics.com](https://www.boltrics.com)) is back online. The emergency form for a production environment that is down is available again. The PHP developers have released security updates for the recently disclosed vulnerabilities. We brought the website back online after the security updates were applied. We confirm again that no other Boltrics services were affected. Business Central (3PL Dynamics), the Web portal, the App platform and DataHub remained available throughout and were not affected by these vulnerabilities. **Required action:** none. Thank you for your patience while the website was offline. If you have any questions, please call Support or create a ticket.

Read the full incident report →

Started Oct 20, 2026, 06:17 AM UTC· resolved Oct 21, 2026, 01:03 PM UTC

1. investigating Oct 20, 2026, 06:17 AM UTC

   Since the night of October 19 to 20, Business Central sandbox environments (RAPP/TEST/WISH) in the Microsoft Cloud may get stuck in the status "Soft deleting" or "Preparing". This happens when a sandbox is deleted or when a new sandbox copy of production is created. During this time the affected sandbox environment is unavailable. **Production environments are not affected.** Your daily work in Business Central continues as normal. The issue lies with Microsoft and appears to affect all Business Central customers, not only Boltrics customers. We have reported it to Microsoft and are following up with them. **Required action** 1. Please do not delete or recreate an affected sandbox yourself. That does not solve the issue and may cause another stuck environment. 2. There is no need to report an affected sandbox. We are monitoring all environments.

2. investigating Oct 20, 2026, 08:07 AM UTC

   We have reported the issue to Microsoft and requested an urgent investigation, highlighting the scale of the impact on our customers. We have provided technical details and examples of the failed operations to support their investigation. We are following up with Microsoft and will update this page as soon as we receive further information about progress or a resolution.

3. investigating Oct 20, 2026, 08:51 AM UTC

   Microsoft has acknowledged the issue and is investigating it. Other Microsoft partners in the West Europe and North Europe regions report the same problem. **Production environments are still not affected.** The required action in our first update still applies: please do not delete or recreate an affected sandbox yourself. We will update this page as soon as Microsoft shares further information.

4. investigating Oct 20, 2026, 02:46 PM UTC

   Microsoft has confirmed that the issue is caused by a disruption in a shared platform service responsible for database management operations in the West Europe region. As a result, environment operations such as Copy, Restore, and Delete may fail or remain in a pending state. According to Microsoft, this is a high-priority incident and the responsible engineering team is actively working on a resolution. No estimated time of resolution (ETA) is available at this moment. Microsoft has indicated that once the underlying platform issue is resolved, any failed or pending environment operations will be processed accordingly. We will continue to monitor the situation and provide further updates as soon as new information becomes available from Microsoft.

5. identified Oct 21, 2026, 08:30 AM UTC

   Microsoft has identified an additional backlog of requests in a supporting service that is contributing to delays in sandbox environment operations, including Copy, Restore, and Delete. Microsoft is actively working to clear this backlog to mitigate the issue. The issue is not yet resolved, and no estimated time of resolution is available. Production environments remain unaffected. Please do not delete or recreate an affected sandbox yourself. We continue to monitor the situation and will provide further updates as soon as more information becomes available from Microsoft.

6. resolved Oct 21, 2026, 01:03 PM UTC

   Business Central sandbox operations (Copy, Restore and Delete) in West Europe and North Europe appear to be working normally again. The issue started in the night of October 19 to 20. Production environments were not affected. Please check whether your sandbox environments are available again. If a sandbox is still stuck in "Soft deleting" or "Preparing", or you still experience other issues, please call Support or create a ticket. We apologize for the inconvenience and thank you for your patience.

Read the full incident report →

Started Oct 06, 2026, 02:25 PM UTC· resolved Oct 06, 2026, 02:40 PM UTC

1. resolved Oct 06, 2026, 02:40 PM UTC

   PTV has resolved the issue and PTV Developer should be working again. Routing should now function as normal. If you applied the workaround, please restore your original settings in Transport Setup: 1. Set Default Activity Calculation Type back to its previous value. 2. Re-enter the values you noted down in the xServer xMap URL, xServer xLocate URL, xServer xRoute URL and xServer Profile fields. Trips created during the outage may be missing driving kilometres and times, and ETAs may not have been calculated. These trips can be recalculated once the settings are restored. If you still experience problems after restoring your settings, please contact Boltrics Support. We apologize for any inconvenience.

Read the full incident report →

Started Aug 28, 2026, 06:25 AM UTC· resolved Aug 28, 2026, 11:34 AM UTC

1. investigating Aug 28, 2026, 06:25 AM UTC

   An issue is affecting the processing of incoming customs messages in Boltrics Data Integration. Our team is investigating and will provide an update as soon as more information becomes available.

2. investigating Aug 28, 2026, 07:00 AM UTC

   Incoming customs messages are being processed again. If you are still missing a message, please resend it. We will continue to monitor the situation. We apologize for the inconvenience.

3. resolved Aug 28, 2026, 11:34 AM UTC

   The issue affecting the processing of incoming customs messages has been resolved. After monitoring the service, we can confirm that incoming messages are being processed normally again. The issue was related to a temporary connection disruption, which has since been restored. We apologize for the inconvenience and thank you for your patience.

Read the full incident report →

Started Jul 28, 2026, 06:30 AM UTC· resolved Jul 28, 2026, 07:03 AM UTC

1. resolved Jul 28, 2026, 07:03 AM UTC

   Our Data Integration Team has resolved the issue, and services are operating normally again. **Required action** If any DataHub flows failed during the incident, restart the affected flow runs. This should allow the processing to complete successfully. If you continue to experience issues after restarting the affected runs, please contact Boltrics Support.

2. resolved Jul 28, 2026, 07:37 AM UTC

   **Root Cause Analysis:** The issue was caused by a defect introduced in a recent DataHub update that included improvements to the efficiency of SFTP flow processing. Due to this defect, unused SFTP connections were not correctly released. Over time, these accumulated and exhausted the available connection pool, preventing new SFTP flows from being processed. The defect was identified and corrected immediately. The fix ensures that unused connections are properly cleared, restoring normal and efficient processing of SFTP flows. No further customer action is required beyond restarting any flows that failed during the incident.

Read the full incident report →

Started Jul 24, 2026, 06:30 AM UTC· resolved Jul 24, 2026, 10:37 AM UTC

1. investigating Jul 24, 2026, 08:10 AM UTC

   We are currently investigating an issue affecting the **DataHub 1 platform**. As a result, some incoming and outgoing messages may not be processed as expected. Our team has identified the cause of the issue and is actively working to restore normal message processing as quickly as possible. We will provide another update as soon as more information becomes available.

2. resolved Jul 24, 2026, 10:37 AM UTC

   The issue affecting the processing of incoming and outgoing messages has been resolved. All messages are now being received and processed correctly. Messages that were not processed during the incident have been successfully restarted and completed. If you believe any messages are still missing or have not been processed, please contact our Support team for assistance. **Root Cause** The incident was caused by an issue in the processing of XSLT 3.0 transformations. **Resolution & Mitigation** The underlying issue has been resolved, and all affected message flows have been restarted to ensure normal message processing has resumed.

Read the full incident report →

Started Jun 22, 2026, 06:00 AM UTC· resolved Jun 22, 2026, 08:29 AM UTC

1. investigating Jun 22, 2026, 06:32 AM UTC

   We are currently investigating an issue affecting Data Integration flows. Some customers may experience failures when integrations attempt to connect to Microsoft Entra ID (Azure AD). As a result, affected integrations may not process data as expected and could return authentication-related errors. Our team has identified that the issue is related to expired authentication credentials used by the integration service and is working on restoring normal operation as quickly as possible.

2. investigating Jun 22, 2026, 06:54 AM UTC

   The authentication issue affecting Data Integration flows has been resolved: The required authentication credentials have been renewed and new integration messages are being processed successfully. Our Data Integration team is currently restarting and processing messages that were impacted during the outage. We continue to monitor the service and will provide a further update once all backlog processing has been completed.

3. resolved Jun 22, 2026, 08:29 AM UTC

   The issue affecting Data Integration flows has been resolved. The root cause was an expired authentication credential used for Microsoft Entra ID access. This occurred due to a manual error during the credential renewal process, resulting in authentication failures for affected integration flows. The authentication credential has been renewed and Data Integration services are operating normally. New messages are being processed successfully. To prevent similar incidents in the future, we are implementing a new version that uses Microsoft-managed keyless authentication, eliminating the need for manual credential renewals. We apologize for the inconvenience and thank you for your patience.

Read the full incident report →

Started May 20, 2026, 08:41 PM UTC· resolved May 21, 2026, 07:15 AM UTC

1. investigating May 21, 2026, 07:08 AM UTC

   We are aware of an issue affecting customs related message flows through Boltrics Data Integration. Due to a DNS issue in the underlying Azure infrastructure, customs messages (including Transit Arrival / T1 flows) were temporarily not transmitted to Customs systems. The issue occurred between 20-05-2026 22:41 CET and 21-05-2026 08:03 CET. During this period, messages may have remained pending and were not delivered successfully.

2. resolved May 21, 2026, 07:29 AM UTC

   A DNS-related infrastructure issue in Azure temporarily impacted customs message flows through Boltrics Data Integration between 20-05-2026 22:41 CET and 21-05-2026 08:03 CET. As a result, customs messages may not have been delivered successfully during this timeframe. The issue has been resolved and message processing is functioning normally again. Customers can resend affected messages from Business Central if required.

Read the full incident report →

Started Apr 22, 2026, 08:35 AM UTC· resolved Apr 22, 2026, 01:05 PM UTC

1. investigating Apr 22, 2026, 08:35 AM UTC

   At the moment, there are issues with one or more DataHub platforms (datahub flow). Therefore, you might experience difficulties using Boltrics’ DataHub Platform. Our team is actively investigating the incident and will provide updates here soon. Note: It only impacts transaction processing, your logistics processes in Business Central will continue.

2. resolved Apr 22, 2026, 01:05 PM UTC

   We've now resolved the incident. Thanks for your patience.

Read the full incident report →

Started Apr 20, 2026, 03:40 PM UTC· resolved May 14, 2026, 04:03 AM UTC

No vendor updates were posted for this incident.

Read the full incident report →

Started Mar 31, 2026, 11:29 AM UTC· resolved Mar 31, 2026, 11:43 AM UTC

1. resolved Mar 31, 2026, 11:43 AM UTC

   The issue has been mitigated, and you should no longer receive any related emails. We will delete the newly created test environments. They will be recreated 7 days prior to your scheduled production update, in line with the normal process.

Read the full incident report →

Started Feb 10, 2026, 07:51 AM UTC· resolved Feb 10, 2026, 11:37 AM UTC

1. resolved Feb 10, 2026, 11:37 AM UTC

   All new incoming customs messages are being received and processed correctly. Messages that were received earlier (including DMS and EMCS messages) have been successfully restarted and processed. If you believe any messages are still missing, please contact Support for assistance. **Root cause:** The issue was caused by an authentication problem between Datahub and Business Central following a recent change by Microsoft. **Resolution & mitigation:** Authentication has been restored and the affected flows have been restarted. To prevent similar issues in the future, monitoring and alerts have been implemented to detect and address this situation at an early stage.

Read the full incident report →

Started Jan 27, 2026, 10:00 PM UTC· resolved Jan 28, 2026, 10:40 AM UTC

1. resolved Jan 28, 2026, 10:40 AM UTC

   All previously affected Dutch Customs (Douane) services are fully operational again as of 11:19 CET. The emergency and fallback procedures are no longer applicable.

Read the full incident report →

Started Jan 12, 2026, 11:15 AM UTC· resolved Jan 13, 2026, 08:17 AM UTC

1. resolved Jan 13, 2026, 08:17 AM UTC

   The earlier issue causing DMS messages to be stuck in a loop has been resolved. Message processing is now functioning as expected. However, it is possible that some messages were not processed. If you are missing a message, please contact Customs to request a resend. We apologize for the inconvenience and appreciate your patience while we worked to resolve the issue.

Read the full incident report →

Started Jan 07, 2026, 11:00 AM UTC· resolved Jan 08, 2026, 01:40 PM UTC

1. resolved Jan 14, 2026, 07:43 AM UTC

   ### **Post Incident Review Report** **Report Date:** Tuesday, January 13, 2026 **Report By:** Microsoft Dynamics 365 Business Central **Summary** On Thursday, January 8, 2026, at 2:00 AM UTC, an escalation reported that users were unable to access your environment. Investigation determined that the Azure Front Door service dependency entered an unhealthy state, leading to impact. Service traffic was routed to redundant infrastructure and, following diagnostic review, it was confirmed that environment access was restored on Thursday, January 8, 2026, at 1:30 PM UTC. **User Experience** Users were unable to access your environment and may have received 502 and/or 504 Azure Front Door Gateway timeout errors. **Incident Start Date and Time** Thursday, January 8, 2026, 2:00 AM UTC **Date and Time Service was Restored** Thursday, January 8, 2026, 1:30 PM UTC **Root Cause** We continue to investigate the root cause of the Azure Front Door service dependency entering an unhealthy state, preventing environment access on Thursday, January 8, 2026. Additional details and results from this investigation will be provided within three weeks.

Read the full incident report →

Started Nov 17, 2025, 08:00 AM UTC· resolved Nov 17, 2025, 09:51 AM UTC

1. resolved Nov 17, 2025, 09:51 AM UTC

   Dear Customer, We would like to inform you that the earlier reported system-wide outage at Cargonaut has been resolved. Thank you for your patience.

Read the full incident report →

Started Oct 30, 2025, 08:52 AM UTC· resolved Oct 30, 2025, 10:17 AM UTC

1. resolved Oct 30, 2025, 10:17 AM UTC

   Microsoft has resolved the connectivity issue affecting Business Central environments. All users should now be able to log in and access their environments as expected. If you continue to experience issues, please contact support for further assistance.

Read the full incident report →

Started Oct 29, 2025, 03:58 PM UTC· resolved Oct 29, 2025, 08:26 PM UTC

1. resolved Nov 11, 2025, 08:14 AM UTC

   **Microsoft Root Cause Analysis – Azure Front Door Connectivity Incident (29–30 October 2025)** On 29–30 October 2025, a global connectivity issue with Azure Front Door caused intermittent service disruptions for Boltrics and other Microsoft customers. Microsoft has published a Root Cause Analysis (RCA) detailing the cause, mitigation, and preventive actions. The summary is included below. Join one of our upcoming 'Azure Incident Retrospective' livestreams discussing this incident (to hear from our engineering leaders, and to get any questions answered by our experts) or watch a recording of the livestream (available later, on YouTube): https://aka.ms/AIR/QNBQ-5W8. **What happened?** Between 15:41 UTC on 29 October and 00:05 UTC on 30 October 2025, customers and Microsoft services leveraging Azure Front Door (AFD) and Azure Content Delivery Network (CDN) experienced connection timeout errors and Domain Name System (DNS) resolution issues. From 18:30 UTC on 29 October 2025, as the system recovered gradually, some customers started to see availability improve – albeit with increased latency – until the system fully stabilized by 00:05 UTC on 30 October 2025. Affected Azure services included but were not limited to: Azure Active Directory B2C, Azure AI Video Indexer, Azure App Service, Azure Communication Services, Azure Databricks, Azure Healthcare APIs, Azure Maps, Azure Marketplace, Azure Media Services, Azure Portal, Azure Sphere Security Service, Azure SQL Database, and Azure Static Web Apps. Other Microsoft services were also impacted, including Microsoft 365 (see: MO1181369), the Microsoft Communication Registry website, Microsoft Copilot for Security, Microsoft Defender (External Attack Surface Management), Microsoft Dragon Copilot, Microsoft Dynamics 365 and Power Platform (see: MX1181378), Microsoft Entra ID (Mobility Management Policy Service, Identity & Access Management, and User Management), Microsoft Purview, Microsoft Sentinel (Threat Intelligence), Visual Studio App Center, and customers’ ability to open support cases (both in the Azure Portal and by phone). **Read more about this issue below.**

Read the full incident report →

Started Oct 09, 2025, 07:50 AM UTC· resolved Oct 13, 2025, 09:26 AM UTC

1. resolved Nov 11, 2025, 07:53 AM UTC

   Final PIR - Azure Front Door - Connectivity issues across multiple regions: Incident ID: YKYN-BWZ Full Report available: https://app.azure.com/h/QNBQ-5W8/e7ac71

Read the full incident report →

Started Sep 04, 2025, 12:22 PM UTC· resolved Sep 04, 2025, 03:00 PM UTC

1. resolved Sep 04, 2025, 03:00 PM UTC

   Issues with Front Door haven been resolved.

Read the full incident report →

Started Jun 19, 2025, 05:16 AM UTC· resolved Jun 19, 2025, 11:26 AM UTC

1. resolved Jun 19, 2025, 11:26 AM UTC

   The performance issues affecting NL environments on major version 25 have now been resolved. Systems are operating normally again, and no further disruptions have been reported. We want to thank you for your patience and understanding during this incident. The Root Cause Analysis (RCA) will be shared as soon as we receive it from Microsoft. We will publish it on this page to keep you fully informed. If you continue to experience any unexpected behavior, please reach out to our support team.

Read the full incident report →

Started Jun 11, 2025, 03:30 AM UTC· resolved Jun 11, 2025, 03:15 PM UTC

1. resolved Jun 16, 2025, 10:46 AM UTC

   Dear customer, First of all, we would like to sincerely apologize once again for the inconvenience caused by the service outage on June 11th. We fully understand the significant impact this has had on your operations and greatly appreciate your patience and understanding. Microsoft has shared a detailed Root Cause Analysis, confirming that the issue resulted from capacity limitations within one of their hosting clusters (AS4556). While the root cause lies within Microsoft’s infrastructure, we take the consequences for our customers very seriously. We will continue our discussions with Microsoft to explore how we can help prevent similar issues in the future and improve both response and resolution times. Our goal remains to ensure the stability and performance of the 3PL Dynamics solution for all our customers. ### **Root Cause Analysis (RCA) Microsoft:** ### Summary Based on our investigation and performance data analysis, the service outage that primarily affected Dynamics 365 Business Central customers in the Netherlands region was caused by cluster resource exhaustion. This resulted from reaching maximum node capacity in the default host group (AS4556, this was the major/main affected one), leading to persistent high CPU utilization and insufficient memory allocation across the cluster nodes. ### Technical Root Cause **Primary Issue: Cluster Resource Saturation** The outage was fundamentally caused by the cluster reaching its maximum operational capacity. Our analysis confirmed that the majority of affected tenants were hosted on this specific cluster infrastructure. The cluster had exceeded its designed tenant-to-node ratio, creating a cascading performance degradation scenario. **Resource Utilization Patterns** The performance metrics from the such outage demonstrate the resource exhaustion pattern: - CPU Utilization: Nodes consistently operated at maximum capacity (approaching 100% utilization) with multiple production instances showing sustained high processor load - Memory Constraints: Available memory dropped to critically low levels (below threshold), forcing the system into resource contention scenarios - Query Performance Impact: Long-running SQL queries increased significantly, with execution times extending beyond normal operational parameters **Infrastructure Scaling Limitations** The cluster architecture had reached its horizontal scaling limits within the default host group configuration. When cluster systems approach maximum node capacity, several performance issues emerge simultaneously: 1. Resource Competition: Multiple tenants competing for limited CPU and memory resources 2. Memory Pressure: Low memory conditions forcing increased CPU usage for memory management operations 3. Query Bottlenecks: Database operations experiencing delays due to resource contention ### Performance Impact Analysis **Login and Authentication Issues** The resource exhaustion directly impacted the authentication infrastructure, causing: - Extended login response times - Authentication service timeouts - Session establishment failures **Application Performance Degradation** Users experienced significant performance issues including: - Unworkable slow response times across Business Central operations - Extended query execution periods - System responsiveness falling below acceptable service levels ### Resolution Strategy **Immediate Remediation** We implemented a cluster splitting solution to address the immediate capacity constraints: 1. Cluster Segmentation: The overloaded cluster was divided into two separate clusters 2. Tenant Redistribution: Existing tenants were redistributed across the newly created cluster infrastructure 3. Resource Rebalancing: The splitting reduced the tenant-to-node ratio, providing adequate CPU and memory resources per tenant **Technical Benefits of Cluster Splitting** The cluster division approach provided several immediate benefits: - Reduced Resource Contention: Lower tenant density per cluster node eliminated resource competition - Improved Memory Allocation: Each cluster segment could allocate sufficient memory resources without constraint - Enhanced Query Performance: Database operations returned to normal execution timeframes with reduced resource pressure ### Preventive Measures **Capacity Monitoring Enhancement** We implemented enhanced monitoring to prevent similar incidents: - Proactive Threshold Monitoring: Real-time tracking of cluster resource utilization metrics - Automated Scaling Triggers: Early warning systems for approaching capacity limits - Tenant Distribution Optimization: Improved algorithms for balanced tenant placement across available infrastructure **Infrastructure Scaling Improvements** The outage highlighted the need for more dynamic scaling capabilities: - Elastic Cluster Management: Enhanced ability to provision additional nodes before reaching capacity limits - Resource Pool Expansion: Increased default host group capacity to accommodate growth - Performance Baseline Monitoring: Continuous tracking of key performance indicators to identify degradation trends ### Conclusion The service outage was a direct result of infrastructure scaling limitations majorly within the AS4556 cluster, where tenant growth exceeded the cluster's designed capacity. The combination of maximum node utilization, high CPU usage, and memory constraints created an environment where normal Business Central operations could not function effectively. Our cluster splitting resolution successfully addressed the immediate capacity issues and restored service performance to acceptable levels.

Read the full incident report →

Started Mar 24, 2025, 07:00 AM UTC· resolved Mar 26, 2025, 09:12 PM UTC

1. resolved Mar 26, 2025, 09:12 PM UTC

   Microsoft has identified and resolved an issue impacting performance for multiple tenants on the same cluster. The root cause was related to resource starvation on the shared application service. As of March 26, 2025, we have been monitoring the telemetry and we can confirm that the issue has been mitigated, and performance has returned to normal.

Read the full incident report →

Started Feb 10, 2025, 08:00 AM UTC· resolved Mar 05, 2025, 08:23 AM UTC

1. resolved Mar 05, 2025, 08:23 AM UTC

   After a full week of monitoring, we can confirm that performance has remained stable across all environments. We continue to work closely with Microsoft to explore long-term solutions and will provide further updates as new information becomes available. Thank you for your patience and trust.

Read the full incident report →

Started Feb 10, 2025, 08:00 AM UTC· resolved Feb 24, 2025, 07:14 PM UTC

1. resolved Feb 24, 2025, 07:14 PM UTC

   Earlier today, we received reports of performance issues affecting multiple tenants running on the same cluster. Microsoft has investigated and identified that the cluster was running low on memory. To mitigate the issue, all service nodes were restarted, and performance should now be back to normal. We have been actively monitoring the telemetry for the affected tenants throughout the afternoon and have observed significant improvement. Since the situation appears to be stable, we will be closing all support requests related to the recent performance issues. If you are still experiencing problems, you can reopen the same support request, and we will follow up accordingly. We sincerely apologize for the inconvenience this has caused and appreciate your patience and cooperation. Boltrics Support Team

Read the full incident report →