- Detected by Pingoru
- Jun 17, 2026, 06:12 PM UTC
- Resolved
- Jun 17, 2026, 06:36 PM UTC
- Duration
- 23m
Affected: Hosted Mender USHosted Mender EU
Timeline · 3 updates
-
identified Jun 17, 2026, 06:12 PM UTC
We've identified a regression affecting users who access Mender with a custom role (rather than the built-in administrator role). As a result, we have rolled back to the previous version. When a user with a custom role attempts to list software when creating a deployment, the request is rejected with There was an error while listing software forbidden by role-based access control Only users assigned a custom role were affected. Users with the built-in admin role were not affected. We apologize sincerely for this issue.
-
monitoring Jun 17, 2026, 06:17 PM UTC
We rolled back the affected version and we're monitoring the results
-
resolved Jun 17, 2026, 06:36 PM UTC
This incident has been resolved, the rollback worked and we'll apply the fix in the next release.
Read the full incident report →
- Detected by Pingoru
- Apr 08, 2026, 01:16 PM UTC
- Resolved
- Apr 08, 2026, 02:13 PM UTC
- Duration
- 57m
Affected: Hosted Mender USHosted Mender EU
Timeline · 4 updates
-
investigating Apr 08, 2026, 01:40 PM UTC
We are currently investigating the issue
-
identified Apr 08, 2026, 01:44 PM UTC
The issue has been identified and a fix is being implemented
-
monitoring Apr 08, 2026, 01:58 PM UTC
A fix has been implemented and we are monitoring the results.
-
resolved Apr 08, 2026, 02:13 PM UTC
Mender UI was unavailable between 2026-04-08T13:16:15Z and 2026-04-08T13:42:09Z (26m) due to a breaking change to Google Analytics dependency leading to misconfiguration. The issue has been mitigated by temporarily disabling Google Analytics until a fix is deployed.
Read the full incident report →
- Detected by Pingoru
- Feb 20, 2026, 09:28 AM UTC
- Resolved
- Feb 20, 2026, 12:50 PM UTC
- Duration
- 3h 21m
Affected: Hosted Mender USHosted Mender EU
Timeline · 5 updates
Read the full incident report →
- Detected by Pingoru
- Nov 19, 2025, 05:08 AM UTC
- Resolved
- Nov 19, 2025, 07:55 AM UTC
- Duration
- 2h 46m
Affected: Hosted Mender US
Timeline · 4 updates
-
investigating Nov 19, 2025, 05:08 AM UTC
We just upgraded the hosted Mender US cluster to Kubernetes v1.33 with a blue/green approach. The new cluster (blue) right after the switch is experiencing issues with the rate limits for some customer. We are investigating the issue.
-
identified Nov 19, 2025, 06:12 AM UTC
We rolled back to the old green cluster, still having the same issue.
-
monitoring Nov 19, 2025, 06:12 AM UTC
We found the root cause, a fix has been implemented, and we're monitoring the results.
-
resolved Nov 19, 2025, 07:55 AM UTC
This incident has been resolved. However, a rate limit hot fix has been implemented, so we will schedule a new maintenance window soon, to apply the definitive fix.
Read the full incident report →
- Detected by Pingoru
- Oct 31, 2025, 08:58 PM UTC
- Resolved
- Nov 03, 2025, 01:19 PM UTC
- Duration
- 2d 16h
Affected: Hosted Mender USHosted Mender EU
Timeline · 5 updates
Read the full incident report →
Critical October 22, 2025 - Detected by Pingoru
- Oct 22, 2025, 10:13 AM UTC
- Resolved
- Oct 22, 2025, 11:31 AM UTC
- Duration
- 1h 18m
Affected: Hosted Mender EU
Timeline · 3 updates
-
monitoring Oct 22, 2025, 10:13 AM UTC
This is the same incident as https://mender.statuspage.io/incidents/v85h4c0whdtc, which is keeping track of it for the hosted Mender EU The issue has been identified. A migration triggered by an upgrade caused an index to be removed prematurely. This in turn caused data corruption. We have initiated a database restore and rolled back the upgrade. We apologize for the inconvenience.
-
monitoring Oct 22, 2025, 10:14 AM UTC
The restore to 08:10:00 UTC has completed and the server is scaled back up. We will continue to monitor the situation.
-
resolved Oct 22, 2025, 11:31 AM UTC
This incident has been resolved
Read the full incident report →
Critical October 22, 2025 - Detected by Pingoru
- Oct 22, 2025, 08:56 AM UTC
- Resolved
- Oct 22, 2025, 11:32 AM UTC
- Duration
- 2h 35m
Affected: Hosted Mender US
Timeline · 6 updates
Read the full incident report →
Critical October 20, 2025 - Detected by Pingoru
- Oct 20, 2025, 09:55 AM UTC
- Resolved
- Oct 20, 2025, 12:37 PM UTC
- Duration
- 2h 41m
Affected: Hosted Mender US
Timeline · 3 updates
-
identified Oct 20, 2025, 09:55 AM UTC
We identified an issue with an upstream cloud provider. Unfortunately, even our statuspage was affected and we could not open this status page before now. Among the issues, we identified: * issues reaching the hosted Mender API server * emails not being sent * UI login not possible. The incident started at about 07:50 UTC
-
identified Oct 20, 2025, 09:58 AM UTC
The cloud provider is fixing their issues, we can see that most of the hosted Mender services are back to full operation. We're continuing to monitor the issue. The email service is still not working.
-
resolved Oct 20, 2025, 12:37 PM UTC
This incident has been solved.
Read the full incident report →
- Detected by Pingoru
- Oct 08, 2025, 03:53 AM UTC
- Resolved
- Oct 08, 2025, 09:09 AM UTC
- Duration
- 5h 16m
Affected: Hosted Mender USHosted Mender EU
Timeline · 2 updates
-
identified Oct 08, 2025, 03:53 AM UTC
We are aware of an open incident with a Cloud Provider for the Artifacts Storage for both hosted Mender US and EU. You may experience slow artifact download or throttling. We're monitoring the upstream provider's status.
-
resolved Oct 08, 2025, 09:09 AM UTC
The upstream provider solved their issue.
Read the full incident report →
- Detected by Pingoru
- Oct 06, 2025, 08:26 AM UTC
- Resolved
- Sep 30, 2025, 01:30 PM UTC
- Duration
- —
Timeline · 1 update
-
resolved Oct 06, 2025, 08:26 AM UTC
On September 25th, we updated the internal Redis Cluster to a new provisioning model, but it turned out that this new deployment was applied without a proper Kubernetes Priority Class being set. So we found out that during a new regular hosted Mender deployment, the pods with higher priority and "preempt = true" could preempt Redis pods to be scheduled faster. We applied a fix on the 30th of September, but during the previous 5 days we had at least four Redis Cluster issues, which lasted 1-2 minutes each. We apologize for the issues this incident might have caused to you.
Read the full incident report →
- Detected by Pingoru
- Sep 26, 2025, 06:41 AM UTC
- Resolved
- Sep 26, 2025, 12:51 PM UTC
- Duration
- 6h 9m
Affected: Hosted Mender USHosted Mender EU
Timeline · 4 updates
-
investigating Sep 26, 2025, 06:41 AM UTC
Customers are reporting issues with the remote terminal (Deviceconnect Service) which is frequently dropping the remote connection. We are investigating this issue.
-
identified Sep 26, 2025, 06:43 AM UTC
The issue has been identified and we're working on a solution.
-
monitoring Sep 26, 2025, 10:39 AM UTC
We rolled back the Deviceconnect service to the previous known working version. We're monitoring the metrics and the results.
-
resolved Sep 26, 2025, 12:51 PM UTC
This incident has been resolved.
Read the full incident report →
Critical September 16, 2025 - Detected by Pingoru
- Sep 16, 2025, 01:58 PM UTC
- Resolved
- Sep 16, 2025, 04:07 PM UTC
- Duration
- 2h 9m
Affected: Hosted Mender US
Timeline · 5 updates
Read the full incident report →
- Detected by Pingoru
- Aug 26, 2025, 05:37 PM UTC
- Resolved
- Aug 27, 2025, 09:46 AM UTC
- Duration
- 16h 8m
Affected: Hosted Mender EU
Timeline · 4 updates
Read the full incident report →
- Detected by Pingoru
- Aug 16, 2025, 05:29 AM UTC
- Resolved
- Aug 16, 2025, 08:16 AM UTC
- Duration
- 2h 46m
Affected: Hosted Mender USHosted Mender EU
Timeline · 2 updates
-
investigating Aug 16, 2025, 05:29 AM UTC
When you try to upgrade your subscription in the Billing page, you can't see the Checkout button and so you cannot upgrade your subscription. We're investigating the issue.
-
resolved Aug 16, 2025, 08:16 AM UTC
We reverted a change for the billing page. Now the behavior should be the one before the version v4.1.0-saas.12 of hosted Mender.
Read the full incident report →
- Detected by Pingoru
- Aug 09, 2025, 10:06 AM UTC
- Resolved
- Aug 09, 2025, 10:17 AM UTC
- Duration
- 11m
Affected: docs.mender.io
Timeline · 3 updates
-
investigating Aug 09, 2025, 10:06 AM UTC
We acknowledged that the documentation website docs.mender.io is currently not serving traffic. We are investigating the issue
-
monitoring Aug 09, 2025, 10:08 AM UTC
The issue has been identified on the web server serving traffic; as a temporary workaround, we forcefully scaled out the web server and we're observing the results.
-
resolved Aug 09, 2025, 10:17 AM UTC
This incident has been resolved.
Read the full incident report →
- Detected by Pingoru
- Jun 23, 2025, 05:08 AM UTC
- Resolved
- Jun 23, 2025, 05:39 AM UTC
- Duration
- 31m
Affected: Hosted Mender US
Timeline · 4 updates
-
investigating Jun 23, 2025, 05:08 AM UTC
We are investigating an issue regarding Redis cluster and the Device Auth Service which is in degraded state.
-
monitoring Jun 23, 2025, 05:19 AM UTC
The issue has been identified: a new Redis pod was restarting because of OOMKill. More memory has been given to the Redis pool and now the services are up. We're monitoring the result.
-
resolved Jun 23, 2025, 05:39 AM UTC
This incident has been resolved.
-
postmortem Jun 23, 2025, 08:29 AM UTC
This morning, the operation team performed a planned Redis Cluster upgrade, starting at 04:40 UTC. Around 04:56 UTC, one of the Redis pod got killed because of Out of Memory issues, causing the Device Auth service to experience connection failure. To resolve this, the operation team increased the memory allocated to the Redis Cluster, starting at 05:05 UTC. The change was fully implemented by 05:14 UTC, and no more error log was seen from the Device Auth service, which was returned to normal operation.
Read the full incident report →
- Detected by Pingoru
- Mar 20, 2025, 09:25 AM UTC
- Resolved
- Mar 20, 2025, 11:02 AM UTC
- Duration
- 1h 36m
Affected: Hosted Mender EU
Timeline · 3 updates
-
investigating Mar 20, 2025, 09:25 AM UTC
We are currently investigating an issue regarding the Inventory service on hosted Mender EU: we got alerted by an unusual amount of 500 error.
-
monitoring Mar 20, 2025, 09:52 AM UTC
The issue has been identified and we forcefully scaled up the Inventory resources. Now all the metrics are fine but we're still continuing to monitor the issue.
-
resolved Mar 20, 2025, 11:02 AM UTC
This incident has been resolved: a memory tuning has been applied to the Inventory service.
Read the full incident report →
- Detected by Pingoru
- Mar 06, 2025, 03:01 PM UTC
- Resolved
- Mar 31, 2025, 07:17 PM UTC
- Duration
- 25d 4h
Affected: Hosted Mender USHosted Mender EU
Timeline · 2 updates
-
identified Mar 06, 2025, 03:01 PM UTC
We identified an issue regarding the Server-side generation of Delta Artifacts feature; the generation starts and reports success but the silently fails in the background. We are working on a fix.
-
resolved Mar 31, 2025, 07:17 PM UTC
A fix has been implemented and tested. This incident is closed.
Read the full incident report →
- Detected by Pingoru
- Feb 25, 2025, 04:05 PM UTC
- Resolved
- Feb 26, 2025, 02:13 PM UTC
- Duration
- 22h 8m
Affected: Hosted Mender USHosted Mender EU
Timeline · 3 updates
-
identified Feb 25, 2025, 04:05 PM UTC
We are aware that some customers cannot update their credit card information from the Organization and Billing page. The issue has been identified and a fix is under deployment. This issue is affecting some customers on basic and professional plans.
-
monitoring Feb 26, 2025, 02:04 PM UTC
A fix has been implemented and we are monitoring the results.
-
resolved Feb 26, 2025, 02:13 PM UTC
This incident has been solved: customers can now update billing details again.
Read the full incident report →
- Detected by Pingoru
- Feb 06, 2025, 08:39 AM UTC
- Resolved
- Feb 06, 2025, 10:33 AM UTC
- Duration
- 1h 54m
Affected: Hosted Mender USHosted Mender EU
Timeline · 4 updates
-
investigating Feb 06, 2025, 08:39 AM UTC
We are currently experiencing issues with the default Artifacts Storage provided by Cloudflare R2; we are investigating the issue. Deployment Service may be affected.
-
identified Feb 06, 2025, 08:51 AM UTC
The issue has been identified: the default Artifacts Storage provider is experiencing an outage.
-
monitoring Feb 06, 2025, 09:43 AM UTC
The upstream Storage provider implemented a fix and the storage seems available again. We see requests are now working. We'll continue to monitor the issue.
-
resolved Feb 06, 2025, 10:33 AM UTC
The upstream provider declared their incident is resolved, so declaring incident as resolved on our end as well.
Read the full incident report →
- Detected by Pingoru
- Feb 03, 2025, 06:57 PM UTC
- Resolved
- Feb 04, 2025, 05:27 PM UTC
- Duration
- 22h 29m
Affected: Hosted Mender USHosted Mender EU
Timeline · 7 updates
Read the full incident report →
- Detected by Pingoru
- Jan 13, 2025, 03:01 PM UTC
- Resolved
- Jan 13, 2025, 04:37 PM UTC
- Duration
- 1h 35m
Affected: Hosted Mender EU
Timeline · 4 updates
-
investigating Jan 13, 2025, 03:01 PM UTC
We are experiencing scalability issue: new Kubernetes worker nodes are rolled out very slow. We're checking with the cloud provider.
-
monitoring Jan 13, 2025, 03:13 PM UTC
Now the required load is matching the required number of Kubernetes worker nodes. We're still in contact with the cloud provider support to check the root cause. The incident is still open.
-
resolved Jan 13, 2025, 04:37 PM UTC
The cloud provider support is still checking the issue. In the meantime we managed to increase the minimum number of Kubernetes worker node to prevent further autoscaling issue.
-
postmortem Jan 29, 2025, 09:15 AM UTC
We discussed the incident with Azure support and decided to replace a problematic component \(an AKS Nodepool\). The new component is working fine and has no scalability issues, so we promoted it to production. No further actions are needed
Read the full incident report →
- Detected by Pingoru
- Jan 08, 2025, 10:06 AM UTC
- Resolved
- Jan 08, 2025, 02:07 PM UTC
- Duration
- 4h
Affected: Hosted Mender US
Timeline · 4 updates
-
investigating Jan 08, 2025, 10:06 AM UTC
Today between 09:26 UTC and 09:28 UTC we got notifications about a MongoDB node failure on the primary. We are investigating the issue, that seems to be already solved.
-
identified Jan 08, 2025, 10:07 AM UTC
We observed in the provider's log that it tried twice to roll it back, then the cluster gave up and elected a new primary. The cluster is self-healing and hosted Mender is operational again.
-
monitoring Jan 08, 2025, 10:07 AM UTC
We're monitoring the incident and the metrics to check for possible issues.
-
resolved Jan 08, 2025, 02:07 PM UTC
This incident has been resolved: the MongoDB cluster seems stable and no other issue has been reported.
Read the full incident report →
- Detected by Pingoru
- Dec 19, 2024, 10:15 AM UTC
- Resolved
- Dec 19, 2024, 01:41 PM UTC
- Duration
- 3h 26m
Affected: Hosted Mender US
Timeline · 4 updates
Read the full incident report →
- Detected by Pingoru
- Dec 13, 2024, 08:09 AM UTC
- Resolved
- Dec 13, 2024, 12:44 PM UTC
- Duration
- 4h 34m
Affected: docs.mender.io
Timeline · 4 updates
Read the full incident report →
