Mews Outage History

# Problem On April 9, 2026, properties in France and Germany were unable to reliably print or download invoices due to issues with fiscal record processing. * In France, printing bills and invoices was blocked. * In Germany, printing was delayed, with documents available after a two-minute fallback. # Causes A database performance issue caused fiscalization jobs in France and Germany to time out. The database selected a suboptimal execution plan for a key query, which prevented fiscal records from being processed and blocked invoice generation. This was an infrastructure issue and was not caused by a recent code release. # Action Our engineering team identified the affected queries and restored normal database performance, unblocking fiscalization jobs in both countries. All pending fiscal records for France and Germany were processed by 09:41 CEST on April 9, 2026. We continued monitoring to confirm stable processing before resolving the incident. # Solutions We are implementing the following measures to prevent recurrence: * **Decouple fiscal processing** * Introduce a new table to separate fiscal record ingestion from downstream processing, reducing the risk of database bottlenecks blocking fiscalization. * **Improve alerting** * Enhance fiscalization job alerts so repeated timeouts trigger immediate investigation, with a focus on markets where operations can be fully blocked. * **Strengthen monitoring and resilience** * Continue improving monitoring and query performance to detect and mitigate similar issues earlier.

**Problem** On April 1st during the morning, we identified that the city tax rates for Barcelona were not being correctly applied to reservations. The new tax rates, which were legally required to take effect from April 1st, had been configured in our system but had not yet been fully propagated to all services at the time reservations were being created. ‌ **Action** Our team was alerted to the issue through property reports and immediately began investigating. We expedited a system deployment to propagate the updated tax configuration and ran corrective processes to fix all reservations that had been created with the incorrect tax amounts. The issue was fully resolved and confirmed by affected properties within approximately one hour.

# Problem On 4 March 2026 at 15:15 CET, a change to room price calculations caused some complex dependent rates to ignore differences between room categories. For affected properties, prices for room types became flat. Between 14:00–17:30 CET, incorrect prices were shown in Mews Operations and sent to channel manager integrations, so some guests booking through online channels might have seen rooms at incorrect prices. Direct bookings in the system using these rate configurations were also affected during the same period. # Action The Mews team detected the issue at 15:15 CET after a customer report. The Mews team confirmed the impact, reverted the change, and at 16:35 CET pricing calculations were correct for all new updates. The Mews team then ran a recovery job to recreate and resend corrected prices to channel manager integrations in a controlled way until all inventory was synchronised on 6 March 2026. # Causes The issue came from a change in dependent rate pricing that was not fully separated from existing pricing behaviour. For some complex rate setups, this caused all room categories on an affected rate to use the same base value. # Solution All prices have been re-synchronised, and pricing services are fully operational. The Mews team continues to monitor pricing accuracy. We apologise for the disruption this incident caused to you and your guests. Pricing accuracy is critical, and we are committed to strengthening our testing, rollout safeguards, monitoring, alerting, and recovery tooling to reduce the risk of similar issues and resolve them faster.

### Problem On 19 February 2026, between 10:53 and 12:10 UTC, a code change caused two issues 1. Users were unable to open the detail page of individual rooms from the timeline, reservation preview, or the space status report. 2. Reservation synchronization via channel managers stopped processing, causing delays in updates from connected distribution channels. The total duration of customer impact was approximately 1 hour and 17 minutes. ### Action The engineering team identified the faulty code change and rolled back the affected release. The backlog of updates from channel managers was then fully processed within 18 minutes. ### Causes A recent code change caused failures in querying history of changes of individual rooms, making the pages displaying such data and reservation synchronization fail. ### Solutions The faulty code change was rolled back and the underlying issue is being fixed. We are also strengthening our deployment safeguards and quality gates to reduce the likelihood and impact of similar incidents in the future.

## Overview In February 2026, Mews experienced an outage lasting 47 minutes and two shorter periods of degraded performance that affected access to the Mews PMS app. There was no data loss or data corruption. The impact was on availability and speed, not on the correctness of your data. Even short interruptions are unacceptable, especially at critical times when your teams are checking guests in and out, running reports, and handling payments. In this postmortem we explain, in straightforward terms: * What happened * How you were affected * What caused it * What we are changing to prevent a repeat We sincerely apologize for the disruption and are treating the symptoms of these incidents with the highest priority to ensure they do not happen again. ## What happened ### 5 February 2026 – Major outage * Between 06:40–07:27 UTC, many users could not log in to Mews. Pages loaded very slowly or showed errors, and key functions were unavailable. * A shorter performance degradation occurred at around 10:00 UTC, affecting a smaller group of properties. * Behind the scenes, several of the servers that handle Mews PMS traffic became overloaded at the same time as a brief issue on our primary database in the Germany West Central region \(Azure Cloud\). Together, these made the app unstable until we restarted the affected servers and increased capacity. ### 12 February 2026 – Short performance drops * On 12 February, there were four short windows where Mews PMS was noticeably slower or briefly unavailable for some users. * These were much shorter than the 5 February outage, but still visible as slow page loads and occasional login problems. ### 17 February 2026 – Mews is slow * On the morning of 17 February, support received multiple reports from different regions that Mews was slow or not loading at all for short periods. * In our monitoring, one of the servers handling Mews PMS traffic was clearly struggling and needed to be restarted. After a restart, performance returned to normal. ## How you were affected Across the three events, properties experienced: * Difficulty logging in to Mews PMS * Slow pages, sometimes ending in timeouts or errors * The app being slow and/or unresponsive during the instances of performance degradation ## Why it happened \(high‑level\) Although the three incidents were triggered by different technical details, the overall pattern was the same. ### 1. The majority of our backend instances were degraded * Servers became overloaded by traffic; * A background database task put extra pressure on our main database; * Internal checks meant to verify that a server is healthy did more work than they should and themselves became slow. In each case, part of our system was under more strain than it could handle, and some servers started responding very slowly or failing requests. ### 2. The “recycling” of instances was not responsive enough We rely on a combination of: * Internal health checks inside our application; and * The cloud provider’s ability to stop sending traffic to unhealthy instances In this case, that combination did not work as well as it needs to: * Our health‑check endpoint was too complex and depended on responses from several other downstream dependencies. * It also continued to say “I’m OK” often enough that the cloud platform - which looks only at that signal - kept the struggling servers in rotation. * The platform behaved exactly as designed, but our own health signal and thresholds were not strict enough to protect you from partial failures - hence the degraded performance you were experiencing. ### 3. Our monitoring focused too much on averages Our automatic scaling and some of our dashboards look at averages across all servers. Those averages can look acceptable even when a few individual servers are in serious trouble. This made it harder to see, quickly and clearly, that “one or two machines are having a very bad time and need to be taken out of service now”. In simple terms: local problems on specific servers and supporting systems exposed weaknesses in how we check health and remove bad servers, and that turned into outages and slowness for you. ## What we did during the incidents During each incident, our teams: * Declared and managed a formal incident, with clear ownership and regular updates. * Restarted or removed problematic servers and temporarily increased capacity to stabilize performance. * Worked with our cloud provider where there were signs of underlying platform issues. * Collected detailed logs and metrics to reconstruct exactly what happened, confirm that there was no data loss, and identify the design gaps that need to be fixed. These actions restored service each time, but they also showed that we need deeper changes, not just tactical fixes. ## What we are changing We are now implementing structural improvements in three key areas. ### 1. Simpler, more trustworthy health checks We are redesigning the internal “health check” that decides whether a server should receive traffic: * The new check will be much lighter and will not depend on multiple other systems being fully healthy just to answer “can this server safely handle requests?”. * It will have strict time limits, so a slow response does not get mistaken for a healthy server. * When a server is clearly unable to serve traffic, the health check will say so clearly and consistently, so the platform can remove it from rotation quickly. This reduces the chance that a partially broken server continues to handle your traffic. ### 2. Faster “recycling” of unhealthy instances and smarter scaling We are improving how we detect and react to unhealthy servers: * Looking at each server individually \(errors, slow responses, health‑check results\), not just at global averages. * Tuning our automatic scaling rules so we react faster when a subset of servers is overloaded. * Adjusting how we use the cloud provider’s automatic recovery tools so that they help with diagnostics and healing, without introducing extra instability. Our goal is that if one server misbehaves, you do not notice it because it is taken out of service and replaced before it affects users. ### 3. Safer behavior around the database We are also tightening up how we work with the database: * For the 5 February event, we are working with our cloud provider on the formal root cause analysis for the brief database issue in our primary region, and we are improving our monitoring of that database so we have earlier and clearer warning signals. * For the 12 February event, we have changed an internal maintenance job so that it no longer makes heavy changes during busy hours. Future adjustments of that type will follow a controlled, manual process. ## Looking ahead These incidents underline how important predictable performance and availability are to your operations. Our focus is on: * Reducing the chance that local technical issues ever become visible to you * Limiting the impact and duration if something does go wrong * Giving ourselves better visibility and clearer signals so we can act quickly and confidently

### Overview Less than 0.5% of properties were unable to create payment requests. All affected customers had multicurrency enabled. The issue was caused by incorrect handling of multicurrency status checks introduced in the release. ### Action The issue was resolved by reverting the update that introduced the faulty logic. After the rollback, payment request creation returned to normal for all affected properties. ### Causes Due to the update, payment gateway accounts data did not load correctly. This caused an error when retrieving multicurrency status, which resulted in payment requests being blocked. ### Solution The hotfix was deployed by reverting the multicurrency status changes introduced in the update. Improvements to test coverage and alerting are underway to help prevent similar issues in the future.

## Overview In February 2026, Mews experienced an outage lasting 47 minutes and two shorter periods of degraded performance that affected access to the Mews PMS app. There was no data loss or data corruption. The impact was on availability and speed, not on the correctness of your data. Even short interruptions are unacceptable, especially at critical times when your teams are checking guests in and out, running reports, and handling payments. In this postmortem we explain, in straightforward terms: * What happened * How you were affected * What caused it * What we are changing to prevent a repeat We sincerely apologize for the disruption and are treating the symptoms of these incidents with the highest priority to ensure they do not happen again. ## What happened ### 5 February 2026 – Major outage * Between 06:40–07:27 UTC, many users could not log in to Mews. Pages loaded very slowly or showed errors, and key functions were unavailable. * A shorter performance degradation occurred at around 10:00 UTC, affecting a smaller group of properties. * Behind the scenes, several of the servers that handle Mews PMS traffic became overloaded at the same time as a brief issue on our primary database in the Germany West Central region \(Azure Cloud\). Together, these made the app unstable until we restarted the affected servers and increased capacity. ### 12 February 2026 – Short performance drops * On 12 February, there were four short windows where Mews PMS was noticeably slower or briefly unavailable for some users. * These were much shorter than the 5 February outage, but still visible as slow page loads and occasional login problems. ### 17 February 2026 – Mews is slow * On the morning of 17 February, support received multiple reports from different regions that Mews was slow or not loading at all for short periods. * In our monitoring, one of the servers handling Mews PMS traffic was clearly struggling and needed to be restarted. After a restart, performance returned to normal. ## How you were affected Across the three events, properties experienced: * Difficulty logging in to Mews PMS * Slow pages, sometimes ending in timeouts or errors * The app being slow and/or unresponsive during the instances of performance degradation ## Why it happened \(high‑level\) Although the three incidents were triggered by different technical details, the overall pattern was the same. ### 1. The majority of our backend instances were degraded * Servers became overloaded by traffic; * A background database task put extra pressure on our main database; * Internal checks meant to verify that a server is healthy did more work than they should and themselves became slow. In each case, part of our system was under more strain than it could handle, and some servers started responding very slowly or failing requests. ### 2. The “recycling” of instances was not responsive enough We rely on a combination of: * Internal health checks inside our application; and * The cloud provider’s ability to stop sending traffic to unhealthy instances In this case, that combination did not work as well as it needs to: * Our health‑check endpoint was too complex and depended on responses from several other downstream dependencies. * It also continued to say “I’m OK” often enough that the cloud platform - which looks only at that signal - kept the struggling servers in rotation. * The platform behaved exactly as designed, but our own health signal and thresholds were not strict enough to protect you from partial failures - hence the degraded performance you were experiencing. ### 3. Our monitoring focused too much on averages Our automatic scaling and some of our dashboards look at averages across all servers. Those averages can look acceptable even when a few individual servers are in serious trouble. This made it harder to see, quickly and clearly, that “one or two machines are having a very bad time and need to be taken out of service now”. In simple terms: local problems on specific servers and supporting systems exposed weaknesses in how we check health and remove bad servers, and that turned into outages and slowness for you. ## What we did during the incidents During each incident, our teams: * Declared and managed a formal incident, with clear ownership and regular updates. * Restarted or removed problematic servers and temporarily increased capacity to stabilize performance. * Worked with our cloud provider where there were signs of underlying platform issues. * Collected detailed logs and metrics to reconstruct exactly what happened, confirm that there was no data loss, and identify the design gaps that need to be fixed. These actions restored service each time, but they also showed that we need deeper changes, not just tactical fixes. ## What we are changing We are now implementing structural improvements in three key areas. ### 1. Simpler, more trustworthy health checks We are redesigning the internal “health check” that decides whether a server should receive traffic: * The new check will be much lighter and will not depend on multiple other systems being fully healthy just to answer “can this server safely handle requests?”. * It will have strict time limits, so a slow response does not get mistaken for a healthy server. * When a server is clearly unable to serve traffic, the health check will say so clearly and consistently, so the platform can remove it from rotation quickly. This reduces the chance that a partially broken server continues to handle your traffic. ### 2. Faster “recycling” of unhealthy instances and smarter scaling We are improving how we detect and react to unhealthy servers: * Looking at each server individually \(errors, slow responses, health‑check results\), not just at global averages. * Tuning our automatic scaling rules so we react faster when a subset of servers is overloaded. * Adjusting how we use the cloud provider’s automatic recovery tools so that they help with diagnostics and healing, without introducing extra instability. Our goal is that if one server misbehaves, you do not notice it because it is taken out of service and replaced before it affects users. ### 3. Safer behavior around the database We are also tightening up how we work with the database: * For the 5 February event, we are working with our cloud provider on the formal root cause analysis for the brief database issue in our primary region, and we are improving our monitoring of that database so we have earlier and clearer warning signals. * For the 12 February event, we have changed an internal maintenance job so that it no longer makes heavy changes during busy hours. Future adjustments of that type will follow a controlled, manual process. ## Looking ahead These incidents underline how important predictable performance and availability are to your operations. Our focus is on: * Reducing the chance that local technical issues ever become visible to you * Limiting the impact and duration if something does go wrong * Giving ourselves better visibility and clearer signals so we can act quickly and confidently

## Problem On February 3, 2026, properties using Mews Operations were unable to add new reservations to availability blocks configured to accept only a single booking group. Any attempt to do so resulted in an error. As a result, staff were unable to create new reservations within those availability blocks for the duration of the incident. In addition, approximately 2,000 reservations that should have been added to an existing booking group were instead incorrectly placed into separate, newly created groups. ## Action Our team was notified of the issue and began investigating immediately. We identified the source of the problem, verified a fix on our test environment, and restored Mews Operations to the previous working version on production. The issue was fully resolved by 15:03 UTC, with error rates returning to zero shortly after. ## Causes The incident was triggered by a software update released earlier that day. The update inadvertently caused Mews Operations to stop correctly identifying which booking group a new reservation belonged to. As a result, the system treated each new reservation as belonging to a brand new group, which conflicted with the single-group rule set on those availability blocks. ## Solutions We restored the previous working version of Mews Operations to immediately stop the impact. A targeted fix addressing the underlying cause has since been deployed. We are also introducing additional automated checks to this area of the system to prevent similar issues from reaching production in the future.